Back

Help Center

Ask questions and help others in this forum.
TOPIC | scraping info from the game database
hi! i am trying to make a spreadsheet with all drops tagged by venue. in the game database, each monster has a list of all drops and their frequencies, and this seems to be stored as an html table. i figured the fastest way to make my spreadsheet would be to make a list of urls for all the monsters in a venue and then use python to scrape the tables.

will i get in trouble for this? i am worried that this might associate my IP address with "bot activity".

(if my overall goal matters here, i am making a bingo board generator.)
hi! i am trying to make a spreadsheet with all drops tagged by venue. in the game database, each monster has a list of all drops and their frequencies, and this seems to be stored as an html table. i figured the fastest way to make my spreadsheet would be to make a list of urls for all the monsters in a venue and then use python to scrape the tables.

will i get in trouble for this? i am worried that this might associate my IP address with "bot activity".

(if my overall goal matters here, i am making a bingo board generator.)
dT9HQge.png
qjxrAwx.png
J8QoPxT.png
PEFrAZg.gif Ti1VB.gif
@oakfern If you wanted official answer you'd have to ask through contact us: https://flightrising.zendesk.com/hc/en-us/requests/new

But based on the Terms of Use generally anything that does something for you automatically wouldn't be allowed:
Quote:
12. The use of bots, scripts, or third party software that shortcut or alter site functions to run or appear in ways that are not intended by site design is not permitted.

13. The use of bots, scripts, or third party software that generate an excessive amount of concurrent connections or potentially damage site stability is not permitted. This can fall under the realm of using browser functions, add-ons, extensions, etc. to hit Company or Services’ server infrastructure in repetitive or disproportionate ways. These incidents will be reviewed thoroughly on a case-by-case basis.
https://flightrising.com/main.php?p=wiki&article=81
@oakfern If you wanted official answer you'd have to ask through contact us: https://flightrising.zendesk.com/hc/en-us/requests/new

But based on the Terms of Use generally anything that does something for you automatically wouldn't be allowed:
Quote:
12. The use of bots, scripts, or third party software that shortcut or alter site functions to run or appear in ways that are not intended by site design is not permitted.

13. The use of bots, scripts, or third party software that generate an excessive amount of concurrent connections or potentially damage site stability is not permitted. This can fall under the realm of using browser functions, add-ons, extensions, etc. to hit Company or Services’ server infrastructure in repetitive or disproportionate ways. These incidents will be reviewed thoroughly on a case-by-case basis.
https://flightrising.com/main.php?p=wiki&article=81
cg2G1xD.gif
tumblr_inline_ngqv6bbPaQ1rvrguk.pngtumblr_inline_n1581tzliY1rvrguk.png
@oakfern
Anything involving a bot performing actions is against the Terms of Use. May I suggest using Maki's text-based drop list instead? Everything is already sorted by venue, which makes it easier to manage.
@oakfern
Anything involving a bot performing actions is against the Terms of Use. May I suggest using Maki's text-based drop list instead? Everything is already sorted by venue, which makes it easier to manage.
wTl0axg.png
@Lonin @Naeryl gotcha, thank you! the text-based drop list doesn't contain drop frequencies, which was my reason for going to the game database. i'll share whatever i end up making in the forums for sure
@Lonin @Naeryl gotcha, thank you! the text-based drop list doesn't contain drop frequencies, which was my reason for going to the game database. i'll share whatever i end up making in the forums for sure
dT9HQge.png
qjxrAwx.png
J8QoPxT.png
PEFrAZg.gif Ti1VB.gif
@oakfern
Drop frequencies aren't really that relevant I feel. If you look at any non-neutral monster's database page you'll see that battlestones and eggs are both under "Very Rarely" drops. However, looking at any avid-Coliseum player's hoard will tell you that they both don't have the same droprate.

If you're making a bingo board that distributes squares more fairly in the sense of one doesn't have four eggs while another only has food items, you're better off assigning appearance rates by hand. Though, of course, I don't know the details of what you're trying to make, so apologies if I'm off-base!
@oakfern
Drop frequencies aren't really that relevant I feel. If you look at any non-neutral monster's database page you'll see that battlestones and eggs are both under "Very Rarely" drops. However, looking at any avid-Coliseum player's hoard will tell you that they both don't have the same droprate.

If you're making a bingo board that distributes squares more fairly in the sense of one doesn't have four eggs while another only has food items, you're better off assigning appearance rates by hand. Though, of course, I don't know the details of what you're trying to make, so apologies if I'm off-base!
wTl0axg.png
@Naeryl that's true - some items i'm just leaving out of the bingo board for that reason. it feels very unfair to ask someone to find an egg for bingo. the current code i have (just for harpy's roost) actually uses item rarity to determine probabilities, since i feel like that has a greater impact on drop rate just based on my own coli experience. e.g. 6-star eliminate and 5-star eggs feel similarly rare, but 1-star scratch drops all the time. i've been testing the output of my current generator by pasting it into one of my dragon's bios, if you're interested

i also just want this as a reference - i feel like there's a lot of disconnected lists on the forums/wiki, and i just think it would be good to have everything that could possibly be useful in one place! and i'm sure i'll think of more coding projects in the future. i'm just a data nerd lol
@Naeryl that's true - some items i'm just leaving out of the bingo board for that reason. it feels very unfair to ask someone to find an egg for bingo. the current code i have (just for harpy's roost) actually uses item rarity to determine probabilities, since i feel like that has a greater impact on drop rate just based on my own coli experience. e.g. 6-star eliminate and 5-star eggs feel similarly rare, but 1-star scratch drops all the time. i've been testing the output of my current generator by pasting it into one of my dragon's bios, if you're interested

i also just want this as a reference - i feel like there's a lot of disconnected lists on the forums/wiki, and i just think it would be good to have everything that could possibly be useful in one place! and i'm sure i'll think of more coding projects in the future. i'm just a data nerd lol
dT9HQge.png
qjxrAwx.png
J8QoPxT.png
PEFrAZg.gif Ti1VB.gif