Data scrapers, help shape the API
elias
MOD
2 months ago
+2
Hi everyone,
I'm seeing quite a few bots regularly scraping data from the website. I'm totally fine with sharing the data but constant scraping is creating excessive load on the server, which can affect performance for regular users.
To make things easier and more efficient for everyone, I'm planning to release a public API with reasonable rate limits. This would be one of the next website updates, together with some minor usability improvements. This way you can get what you need without hammering the site.
I want to get it right. What specific data are you scraping? What endpoints would you need?
Hi everyone,
I'm seeing quite a few bots regularly scraping data from the website. I'm totally fine with sharing the data but constant scraping is creating excessive load on the server, which can affect performance for regular users.
To make things easier and more efficient for everyone, I'm planning to release a public API with reasonable rate limits. This would be one of the next website updates, together with some minor usability improvements. This way you can get what you need without hammering the site.
I want to get it right. What specific data are you scraping? What endpoints would you need?
itsmechinmoy
1 month ago
-1
havent scraped anything yet but pretty sure most people are scraping sequels/watch order
will be great if we could get info on watchorder/sequel directly using mal_id or al_id or any other services
havent scraped anything yet but pretty sure most people are scraping sequels/watch order
will be great if we could get info on watchorder/sequel directly using mal_id or al_id or any other services
hacker09
1 month ago
+0
The system needs to retrieve all anime titles associated with a franchise on chiaki.site. This includes extracting the anime title from the h2 element while removing “Watch Order” text, symbols, and extra whitespace.
All anime links for a franchise must be collected and stored in an array.
For each franchise, the system must determine whether at least one entry exists and if it's type is TV.
It must also retrieve all entry IDs used by a franchise in the chiaki.site franchise page website.
It must also retrieve the total number of links associated with a franchise.
It must also be able to query a single anime ID to return all related entries in the correct watch order for that franchise.
The system should support bulk actions on franchises, such as adding/deleting all entries in a franchise with a chosen status by updating the status of every entry within a franchise.
Authentication must support automatic MyAnimeList login, like the “Sign in with MyAnimeList” button using the MAL sign-in API.
The system should be able to detect whether a franchise contains only a single entry on chiaki.site.
It must also provide aggregated franchise statistics, allowing users to quickly see how long it would take to watch an entire franchise, along with total episode count and total entry count. For example, for https://chiaki.site/?/tools/watch_order/id/40454, the system displays total duration, total episodes, total entries, and a breakdown by type (OVA, ONA, Music, Special, Movie, TV).
Total Duration: 1 day(s) 12 hr(s) 24 min(s)
Total Episodes: 92
Total Entries: 13
3 OVA(s)
0 ONA(s)
0 Music(s)
2 Special(s)
1 Movie(s)
88 TV Episode(s)
Also, this kind of data is cool
https://chiaki.site/?/tools/watch_order_groups/type/popular
https://chiaki.site/?/tools/watch_order_groups/type/large
Would be nice to have those endpoints, preferably without the 5-page-only limitation.
The system must support exporting all franchise entries in a Chiaki.site franchise page in a CSV-like text format. Each entry should include the title, entry type, episode count, and duration, where applicable, formatted as shown in the example below.
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka,, 13 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka OVA,, 1 OVA of 24min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka Gaiden: Sword Oratoria,, 12 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka Movie: Orion no Ya,, 1 Movie of 1hr. 22min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka II: Past & Future,, 1 TV Special of 24min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka II,, 12 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka II OVA,, 1 OVA of 26min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka III,, 12 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka III OVA,, 1 OVA of 27min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka IV Episode 0,, 1 TV Special of 23min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka IV: Shin Shou - Meikyuu-hen,, 11 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka IV: Shin Shou - Yakusai-hen,, 11 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka V: Houjou no Megami-hen,, 15 eps,
The system needs to retrieve all anime titles associated with a franchise on chiaki.site. This includes extracting the anime title from the h2 element while removing “Watch Order” text, symbols, and extra whitespace.
All anime links for a franchise must be collected and stored in an array.
For each franchise, the system must determine whether at least one entry exists and if it's type is TV.
It must also retrieve all entry IDs used by a franchise in the chiaki.site franchise page website.
It must also retrieve the total number of links associated with a franchise.
It must also be able to query a single anime ID to return all related entries in the correct watch order for that franchise.
The system should support bulk actions on franchises, such as adding/deleting all entries in a franchise with a chosen status by updating the status of every entry within a franchise.
Authentication must support automatic MyAnimeList login, like the “Sign in with MyAnimeList” button using the MAL sign-in API.
The system should be able to detect whether a franchise contains only a single entry on chiaki.site.
It must also provide aggregated franchise statistics, allowing users to quickly see how long it would take to watch an entire franchise, along with total episode count and total entry count. For example, for https://chiaki.site/?/tools/watch_order/id/40454, the system displays total duration, total episodes, total entries, and a breakdown by type (OVA, ONA, Music, Special, Movie, TV).
Total Duration: 1 day(s) 12 hr(s) 24 min(s)
Total Episodes: 92
Total Entries: 13
3 OVA(s)
0 ONA(s)
0 Music(s)
2 Special(s)
1 Movie(s)
88 TV Episode(s)
Also, this kind of data is cool
https://chiaki.site/?/tools/watch_order_groups/type/popular
https://chiaki.site/?/tools/watch_order_groups/type/large
Would be nice to have those endpoints, preferably without the 5-page-only limitation.
The system must support exporting all franchise entries in a Chiaki.site franchise page in a CSV-like text format. Each entry should include the title, entry type, episode count, and duration, where applicable, formatted as shown in the example below.
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka,, 13 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka OVA,, 1 OVA of 24min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka Gaiden: Sword Oratoria,, 12 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka Movie: Orion no Ya,, 1 Movie of 1hr. 22min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka II: Past & Future,, 1 TV Special of 24min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka II,, 12 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka II OVA,, 1 OVA of 26min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka III,, 12 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka III OVA,, 1 OVA of 27min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka IV Episode 0,, 1 TV Special of 23min.,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka IV: Shin Shou - Meikyuu-hen,, 11 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka IV: Shin Shou - Yakusai-hen,, 11 eps,
Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka V: Houjou no Megami-hen,, 15 eps,
hacker09
20 days ago
+0
Letting users know the total number of franchises on the site would be cool, too!
Letting users know the total number of franchises on the site would be cool, too!
hacker09
19 days ago
+0
I would love to know, using the API, if the current franchise has any unfinished entries or entries that have not yet aired.
And another JSON key showing the last month/year that the last entry was released.
I would love to know, using the API, if the current franchise has any unfinished entries or entries that have not yet aired.
And another JSON key showing the last month/year that the last entry was released.
Topic reply
Sign in to reply
2019–2026 chiaki.site, contact