Guides

10771

April 26, 2015 09:08:59

@Sylvandyr No, even if the programming is fine. But I should probably think through it more.

So, what's lets you figure out the drops sometimes is that the enemies drop stuff in the order they're in, and the currency is always the last thing from a given enemy, is that right?

@nika - If I programmed it, the neutral and water enemies would have a totally independent chance. It would take the % chance from a lookup table based on the zone, and there would be one % for water and one % for neutral. The fact that the enemy appears in the same pack would be irrelevant. If you're worried about their design, though, I understand. (Our data shows that it indeed seems to go by zone instead of monster, with the Storm Seeker in Harpy's Roost). It's possible they did something similar to what I would do, but it was bad for performance, and they opted for manually set values. (In that case maybe the Storm Seeker was set to the Harpy's Roost value, I dunno - regrettably did not get enough Sandswept Delta data.). But even if manually set, if they have a zone drop setup for festivals, the neutral and water monsters would have their own independent drops regardless of pack.

I'm not convinced that the currency always appears last, because I used to think food always dropped first but I've seen trinkets drop before or after food. But, take the Mammertee / Jeweled Octoflyer / Kelp Tender pack. If I see an Arcane octopus food and then a token, I know that token is the Kelp Tender's. Drops go in order by enemy, even if not always in the same order by type (food/battle stone etc.)

Sylvandyr

16695

April 26, 2015 09:23:24

@nika - If I programmed it, the neutral and water enemies would have a totally independent chance. It would take the % chance from a lookup table based on the zone, and there would be one % for water and one % for neutral. The fact that the enemy appears in the same pack would be irrelevant. If you're worried about their design, though, I understand. (Our data shows that it indeed seems to go by zone instead of monster, with the Storm Seeker in Harpy's Roost). It's possible they did something similar to what I would do, but it was bad for performance, and they opted for manually set values. (In that case maybe the Storm Seeker was set to the Harpy's Roost value, I dunno - regrettably did not get enough Sandswept Delta data.). But even if manually set, if they have a zone drop setup for festivals, the neutral and water monsters would have their own independent drops regardless of pack.

I'm not convinced that the currency always appears last, because I used to think food always dropped first but I've seen trinkets drop before or after food. But, take the Mammertee / Jeweled Octoflyer / Kelp Tender pack. If I see an Arcane octopus food and then a token, I know that token is the Kelp Tender's. Drops go in order by enemy, even if not always in the same order by type (food/battle stone etc.)

Bonsai pixels (tofu and tea motif) by miirshroom

Last edited on Apr 26, 2015 09:32:17 by Sylvandyr

@Sylvandyr Yeah, idk if they programmed it the way you would (I'm not sure if per-zone drop rates are a thing in addition to per-enemy ones), but I'm not really that worried about them doing something as weird as pack-dependent drop rates, since that would be a lot of extra work.

Okay, the simplest problem with this, as far as I can see, is that if both enemies or neither enemy drop currency, then you can always be sure of the drops, but if only one does, than you can sometimes be sure and sometimes not, depending on what other enemy-specific stuff you see. So, for example, say that the Neutral has a 20% drop chance and Water has 40%, and the chance of being sure when only one drops is 50%.
Then out of 1000 battles, you get:
- 80 cases where both drop
- 120 cases where only Neutral drops, out of which you can be sure of 60
- 320 cases where only Water drops, out of which you can be sure of 160
- 480 cases where neither drop.
So if you then only count the cases you're sure of, which there's 780 of, you get:
- 80+60 = 140 drops from Neutral, 140/780 = 18%
- 160+80 = 240 drops from Water, 240/780 = 31%
Both the drop rates look lower than they are, because some cases are easier to be certain of than others. (Or I'm failing at math. Let me know if I am. I'm feeling pretty brain-scrambled today.)

I suppose this could be fixed by only counting half the both-drop and neither-drop cases as well, but for that purpose you'd have to for each pack keep track of how often you can vs can't be sure of the drops, and it gets even more complicated with something like Water-Neutral-Water packs, which have more possible drop combination outcomes...

Basically, this kind of statistics gets screwy really easily if you start only including some part of the data based on the result, even if you're not -trying- to introduce bias.

(There may also be more complicated problems which I'm still having trouble thinking through.)

nika

10771

April 26, 2015 09:39:03

@Sylvandyr Yeah, idk if they programmed it the way you would (I'm not sure if per-zone drop rates are a thing in addition to per-enemy ones), but I'm not really that worried about them doing something as weird as pack-dependent drop rates, since that would be a lot of extra work.

Okay, the simplest problem with this, as far as I can see, is that if both enemies or neither enemy drop currency, then you can always be sure of the drops, but if only one does, than you can sometimes be sure and sometimes not, depending on what other enemy-specific stuff you see. So, for example, say that the Neutral has a 20% drop chance and Water has 40%, and the chance of being sure when only one drops is 50%.
Then out of 1000 battles, you get:
- 80 cases where both drop
- 120 cases where only Neutral drops, out of which you can be sure of 60
- 320 cases where only Water drops, out of which you can be sure of 160
- 480 cases where neither drop.
So if you then only count the cases you're sure of, which there's 780 of, you get:
- 80+60 = 140 drops from Neutral, 140/780 = 18%
- 160+80 = 240 drops from Water, 240/780 = 31%
Both the drop rates look lower than they are, because some cases are easier to be certain of than others. (Or I'm failing at math. Let me know if I am. I'm feeling pretty brain-scrambled today.)

I suppose this could be fixed by only counting half the both-drop and neither-drop cases as well, but for that purpose you'd have to for each pack keep track of how often you can vs can't be sure of the drops, and it gets even more complicated with something like Water-Neutral-Water packs, which have more possible drop combination outcomes...

Basically, this kind of statistics gets screwy really easily if you start only including some part of the data based on the result, even if you're not -trying- to introduce bias.

(There may also be more complicated problems which I'm still having trouble thinking through.)

Last edited on Apr 26, 2015 09:40:35 by nika

@nika - I'm confused. I was not going to include the non-sure drops. Why is this an issue? Wouldn't it be an issue also by not including those packs at all? Why is there a difference?

Sylvandyr

16695

April 26, 2015 10:25:36

@nika - I'm confused. I was not going to include the non-sure drops. Why is this an issue? Wouldn't it be an issue also by not including those packs at all? Why is there a difference?

Last edited on Apr 26, 2015 10:27:26 by Sylvandyr

@Sylvandyr That's the problem - if you don't include the non-sure drops, you get the wrong drop rates (as I showed in my previous post, hopefully - did the math there make sense?).

If we just ignore all mixed packs, then we're ignoring 100% of all possible drop cases, which is fine because we're treating them all equally. But if we try to count the ones we're sure of, then we ignore different percentages of different drop cases, because different drop cases have different chances of being sure vs not-sure. That's what ends up skewing the data.

This kind of thing is generally pretty confusing, which is why the easiest thing to do is just to never count/ignore data based on the result, because that tends to carry a high risk of unintentionally introducing bias.

Btw @Drezdin you may want to look at the last few posts if you want a more detailed explanation of why I didn't want you to count mixed packs, since Sylvandyr had the same idea.

nika

10771

April 26, 2015 10:52:43

@Sylvandyr That's the problem - if you don't include the non-sure drops, you get the wrong drop rates (as I showed in my previous post, hopefully - did the math there make sense?).

If we just ignore all mixed packs, then we're ignoring 100% of all possible drop cases, which is fine because we're treating them all equally. But if we try to count the ones we're sure of, then we ignore different percentages of different drop cases, because different drop cases have different chances of being sure vs not-sure. That's what ends up skewing the data.

This kind of thing is generally pretty confusing, which is why the easiest thing to do is just to never count/ignore data based on the result, because that tends to carry a high risk of unintentionally introducing bias.

Btw @Drezdin you may want to look at the last few posts if you want a more detailed explanation of why I didn't want you to count mixed packs, since Sylvandyr had the same idea.

Last edited on Apr 26, 2015 10:53:40 by nika

@nika - But we're not treating it equally if we ignore all the mixed drop cases. That's where I'm confused. Ignoring mixed drop cases is throwing away data too.

I understand that we can't be sure which did and which didn't drop tokens in some of the mixed cases, but why is there no bias if we throw away all that data? Mixed cases are part of the overall data. Won't there be some bias anyway if we ignore it all?

(I'm probably in support of ignoring the mixed packs for simplicity, but I don't understand why it's okay to ignore some data and not others.)

Sylvandyr

16695

April 26, 2015 11:00:35

@nika - But we're not treating it equally if we ignore all the mixed drop cases. That's where I'm confused. Ignoring mixed drop cases is throwing away data too.

I understand that we can't be sure which did and which didn't drop tokens in some of the mixed cases, but why is there no bias if we throw away all that data? Mixed cases are part of the overall data. Won't there be some bias anyway if we ignore it all?

(I'm probably in support of ignoring the mixed packs for simplicity, but I don't understand why it's okay to ignore some data and not others.)

Last edited on Apr 26, 2015 11:25:43 by Sylvandyr

@Sylvandyr The problem with ignoring data comes in if you ignore different results with different frequencies. Like... if you arbitrarily ignore data from Tuesdays, and the drop rates don't change between days, you're fine. But if you ignore data from "unlucky" days, then you're not fine. These cases are obvious, our case is more complicated, but that's the basic principle.

If we assume that the drop rates are the same for all Neutral and all Water monsters regardless of whether they're in a mixed or unmixed pack, then by ignoring mixed packs we're ignoring the same fraction of drops and non-drops, so the data isn't biased. It's just like ignoring data from all Tuesdays.
(Of course it's possible that our assumption is wrong - maybe drop rates are pack-dependent, or maybe different enemies have different drop rates, in which case if some enemies show up in mixed packs and other ones in unmixed, the drop rates we're ignoring could be different from the ones we're counting and we'd be in trouble. I hope their programming isn't that excessively complicated, but if it is, then really all we can do is just count drops per battle and ignore enemy identities/elements - which tbh might not be a bad way to go. Hmm.)

On the other hand counting the sure drops from mixed packs pretty much inevitably leads to ignoring some results more often than others (because some results are always sure and others are non-sure with varying chances), which is what causes bias in the data - see my post with the math. (It might be possible to come up with some way to compensate for that, but it seems really complicated and almost certainly more effort than it's worth.)

nika

10771

April 26, 2015 12:36:40

@Sylvandyr The problem with ignoring data comes in if you ignore different results with different frequencies. Like... if you arbitrarily ignore data from Tuesdays, and the drop rates don't change between days, you're fine. But if you ignore data from "unlucky" days, then you're not fine. These cases are obvious, our case is more complicated, but that's the basic principle.

If we assume that the drop rates are the same for all Neutral and all Water monsters regardless of whether they're in a mixed or unmixed pack, then by ignoring mixed packs we're ignoring the same fraction of drops and non-drops, so the data isn't biased. It's just like ignoring data from all Tuesdays.
(Of course it's possible that our assumption is wrong - maybe drop rates are pack-dependent, or maybe different enemies have different drop rates, in which case if some enemies show up in mixed packs and other ones in unmixed, the drop rates we're ignoring could be different from the ones we're counting and we'd be in trouble. I hope their programming isn't that excessively complicated, but if it is, then really all we can do is just count drops per battle and ignore enemy identities/elements - which tbh might not be a bad way to go. Hmm.)

On the other hand counting the sure drops from mixed packs pretty much inevitably leads to ignoring some results more often than others (because some results are always sure and others are non-sure with varying chances), which is what causes bias in the data - see my post with the math. (It might be possible to come up with some way to compensate for that, but it seems really complicated and almost certainly more effort than it's worth.)

@nika - Ahhh, okay, now it makes sense to me. I'd looked at the math, but the distinction didn't click until now. I make a practice of not ignoring "unlucky" data, and it was weird to me that we ignore all the mixed packs, but now I get the explanation. I think the missing piece for me (in case you have to explain it to other people) was that italicized bit you just posted, plus the previous line in the math post (some results are easier to be sure of than others).

I think we should go with the pure packs and pray that we don't have to deal with individual pack drops. :P

Sylvandyr

16695

April 26, 2015 12:54:47

@nika - Ahhh, okay, now it makes sense to me. I'd looked at the math, but the distinction didn't click until now. I make a practice of not ignoring "unlucky" data, and it was weird to me that we ignore all the mixed packs, but now I get the explanation. I think the missing piece for me (in case you have to explain it to other people) was that italicized bit you just posted, plus the previous line in the math post (some results are easier to be sure of than others).

I think we should go with the pure packs and pray that we don't have to deal with individual pack drops. :P

@Sylvandyr Yay! I'm glad we got there. Thanks for the tip - yeah, I definitely need to get better at explaining this. I need to do similar stuff at work with some frequency, and it's always tough.

I wish there was some sensible way to count mixed-pack data, but I don't think there is, if we want to maintain the distinction between Neutral and elemental enemies. So yeah, let's just stick with what we're doing.

nika