+1% Crit Arrow Myth

OMGLAZERZ - Heavens Tear

EDIT: I'm linking my last post in the thread, so you can skip to that if you want. b:pleased

http://pwi-forum.perfectworld.com/showpost.php?p=1934221&postcount=109

******************************************************
******************************************************

I was curious to see if the crit arrow worked, so I went and tested the theory that it works. I mean, everyone says it does, so it has to, right?

So, I set out to verify what seemed to be common-sense logic : if it says it's giving you +1% crit that it is giving you just that: +1% crit.

Common sense, of course.

So, I started my testing, and then they patched PWI, so I started my testing over again which was done in both tiger form and human form on my barbarian.

Over about 6,760 hits I had exactly 138 crits which comes out to a 2.04% crit rate with the arrow equipped, and a base crit rate of 2%. Having the arrow equipped should have boosted that to a 3% crit rate. I also tested the Christmas Blessing against it and over 1020 hits had 29 crits which comes out to about a 3% crit rate.

As a baseline to compare it to, I unequipped the arrow and did a run of 1025 hits where I ended up getting 22 crits, or about a 2.15% crit rate which was the same with the arrow equipped.


What does this tell us? That the Crit arrow doesn't actually give you +1% crit rate with a non-bow equipped.

Kingpiccolo - Heavens Tear

But it sure looks impressive.

damani

A mathematician will quickly tell you that your sample size is way too small to correctly derive anything from.

However, this does need to be solved by a GM or a Developer, b/c I just found out about it today and really either need to get on board, or save some time by it being debunked from the start.

Thanks for the experiment tho

Swifti - Heavens Tear

tbh i also thought it was just a display bug at first, considering theres lots of similar bugs in other games(in diablo 2 they call it the lying character screen or LCS for short). should be fixed either way though.

OMGLAZERZ - Heavens Tear

damani wrote: »

A mathematician will quickly tell you that your sample size is way too small to correctly derive anything from.

However, this does need to be solved by a GM or a Developer, b/c I just found out about it today and really either need to get on board, or save some time by it being debunked from the start.

Thanks for the experiment tho

Except, it isn't too small to tell us nothing.

If the arrow actually did something, it would start to show a higher crit rate then it did. Consider that using the Christmas Blessing to push his crit rate to 3% showed a 3% crit rate at ~1000 hits which was well below the 6700+ with the crit arrow equipped, and not even using the crit arrow or the blessing produced a 2% crit rate at ~1000 hits.


I am fairly confident that the numbers would continue to produce the same results even at higher numbers.

jemima

damani wrote: »

A mathematician will quickly tell you that your sample size is way too small to correctly derive anything from.

However, this does need to be solved by a GM or a Developer, b/c I just found out about it today and really either need to get on board, or save some time by it being debunked from the start.

Thanks for the experiment tho

Assuming it's an actual binomial distribution (we don't actually know whether crit chances are independent of each other) 138 out of 6760 with a CL of 95% gives a CI from 1.73% to 2.41%.

Seems accurate enough to me.

How many trials would you consider necessary?

DeliaLotus - Lost City

I think it's fair to assume critical hits are independent, and as shown above, n=6700 is a statistically reasonable sample size. n=1000 is what they use for television ratings I believe.

Edit: Calculation:

binomcdf(6760,.03,138) = 6.3 * 10^(-7).

That means if the crit arrow worked, there's a 99.9999% chance this would not happen.

I think it's fair to say the arrow doesn't work.

Rundora - Lost City_1243871147

Hm...


WTS 1 Crit Arrow. PM Offers in-game b:avoid

VonTerror - Lost City

Did you really jot down on a notepad every hit you took and how many crits there were? Seems like a waste of your time but hey, to each his own

Obsessed - Heavens Tear

VonTerror - Lost City wrote: »

Did you really jot down on a notepad every hit you took and how many crits there were? Seems like a waste of your time but hey, to each his own

That's the first question I asked o.o. How accurate are these hit counters. Anyways, even if it doesnt add anything I'll keep wearing it, doesn't hurt and makes me feel better looking at my status window.b:laugh

Solandri - Heavens Tear

DeliaLotus - Lost City wrote: »

I think it's fair to assume critical hits are independent, and as shown above, n=6700 is a statistically reasonable sample size. n=1000 is what they use for television ratings I believe.

For a 95% confidence interval, the margin of error is

1.96 * sqrt [p*(1-p) / n]

where p is the rate at which the event occurs, n is the total sample size.

For frequent phenomena (stuff that occurs near 50% of the time), the p*(1-p) term cancels with the 1.96. 1.96*sqrt(0.5*0.5) = 0.98 which most everyone approximates as 1. And you get the familiar margin of error of 1/sqrt(n) used in polls and things like television ratings. People and the media like this because you can calculate the margin of error using only the sample size.

For infrequent phenomena, the p*(1-p) term does not cancel, and you have to take into account the incident rate p. If you don't do this, you get a misleading error margin which is bigger than the phenomenon you're trying to measure.

For p=0.02, and a sample of 6760, the margin of error with a 95% confidence interval is:

1.96 * sqrt(0.02*0.98/6760) = +/-0.33% avg error. That is, 2.04% +/- 0.33%, which is where jemima's 1.73% to 2.41% comes from. It's basically a more accurate version of 2% +/-0.33% which calculates the error in both directions separately using math which I would rather not get into here.

The second test with the Christmas blessing had an expected 3% incident rate and sample size of 1020. That gives a margin of error of 1.04%. So with a 95% confidence interval, the critical rate was 2.84% +/- 1.04%. So that sample is not big enough to draw a conclusion. Essentially, the critical rate with the Christmas blessing was anywhere from 2%-4% with this sample with a 95% CI.

The third test without arrorw or blessing had an expected 2% incident rate and sample size of 1025. This works out to 2.15% +/- 1.05%. So again, this sample is not big enough. It places the critical rate at between 1%-3%.

So really the only thing you can conclude from all this is that the +crit arrows aren't raising the critical rate above 2%. We can't be sure if the critical rate sans arrow was 1%, 2% or even 3%; and we can't be sure if the critical rate with the Christmas blessing was 2%, 3%, or 4%.

VonTerror - Lost City

b:shocked b:shockedb:shocked

Must of paid attention in Calculus

jemima

Stats, not calculus.

Solandri: As regards the third test with 22 out of 1025, even with a CL of 95%, I still get about 1.35% to 3.23% using Clopper-Pearson, and we can reasonably* assume that the arrow doesn't decrease our chances of critting, so we're still quite justified in saying that the crit rates with or without arrow are pretty much the same.

*It's certainly possible that someone decided to make the arrow a trap, and penalise people who equipped it, but I feel it's unlikely, especially as if it were the case the display would need to be incorrect even without the arrow, and correct with.*

Darksorrow - Heavens Tear

VonTerror - Lost City wrote: »

b:shocked b:shockedb:shocked

Must of paid attention in Calculus

For those who fell asleep in math class (or reading the long post);

6700ish was a sufficient sample size to base a conclusion on.
1000ish was an insufficient sample size to base base anything on.
In order to prove with 95% likely hood, the 2 later experiements would need a sample size signifigantly higher similar to the 1st experiment.

OMGLAZERZ - Heavens Tear

Darksorrow - Heavens Tear wrote: »

For those who fell asleep in math class (or reading the long post);

6700ish was a sufficient sample size to base a conclusion on.
1000ish was an insufficient sample size to base base anything on.
In order to prove with 95% likely hood, the 2 later experiements would need a sample size signifigantly higher similar to the 1st experiment.

The later 2 checks don't need more to prove anything since at a base crit rate of 2% you would end up with about 138 crits (give or take 1-2) over that many (crit arrow data) hits which is the number of crits I ended up with when the crit arrow was equipped.

Unless you want to call your base crit % into question as well.


Edit :: the Christmas Blessing actually had 2655 data points with 76 of them being crits.


I have a rather large amount of data....

walkingthepath

I'm having trouble figuring out why anyone would think that an arrow would boost damage for any weapon other than a bow.

My question is...does it work with a bow?

Obsessed - Heavens Tear

@walkingthepath it was concieved because it changes the crit number in the status window, even without a bow equipped.

Your second question is also very intriguing, I'd like to know as well.

Renara - Heavens Tear

Now we know why the GMs don't ban players for this particular "exploit"! b:laugh

Zoe - Heavens Tear

If it does anything or not I fail to see why it would be a bad thing. It's not like it unbalances anything, every class can use it after all.

DeliaLotus - Lost City

I don't personally mind it (if a BM wants 1% extra crit, they can have it), but no, not every class can use it equally. Everyone besides archers pay 2.5k once, archers would have to pay 2.5k every shot, hence why you never see an archer using them (at least I haven't).

Zoe - Heavens Tear

Why pay that much when you can just craft them?

DeliaLotus - Lost City

Zoe - Heavens Tear wrote: »

Why pay that much when you can just craft them?

Either way, they have a significant cost, well above that of a standard arrow. Even if we say it's only 100 coin per arrow (and with the cost and chance involved in crafting, I think this is an extremely low estimate), for 1000 attacks an archer has to pay 100k coin, while any other class pays only 100 coin.

It's similar to if charms never expired for a certain class - other classes would be reasonably upset. The 1% crit isn't as major a benefit as an everlasting charm would be, but it's a similar situation on a much smaller scale.

Smexxyfox - Heavens Tear

Archers already have such a high crit rate that 1% wouldnt make a SIGNIFICANT difference.

Its not like it suddenly gives all the other classes 20% crit rate.

Aesthor - Heavens Tear

walkingthepath wrote: »

I'm having trouble figuring out why anyone would think that an arrow would boost damage for any weapon other than a bow.

My question is...does it work with a bow?

The amount of time and money it takes to fire 6000 crit arrows is insane. Those things are worth like 200-300 coin but are often sold for over 1K. That's EACH. Getting 6K of them is ludicrous (but not as ludicrous as getting 9999 feathers for phoenix I guess). Also it takes me a while to burn out 6K arrows firing at a constant rate. You can't fire constant rate killing mobs and the only boss that's gonna last 6K arrows are those level 150 bosses on the field. Except for soul hunter I don't think you want to be counting data fighting those, and if you're fighting Soul Hunter you're gonna be shooting EXTRA SLOW.

tank4ever

try test it as an melee archer and use the 10% accuracy bolt ?

10% hit will give you more accurate analizes and undoubtly results.

Obsessed - Heavens Tear

Aesthor - Heavens Tear wrote: »

The amount of time and money it takes to fire 6000 crit arrows is insane. Those things are worth like 200-300 coin but are often sold for over 1K. That's EACH. Getting 6K of them is ludicrous (but not as ludicrous as getting 9999 feathers for phoenix I guess). Also it takes me a while to burn out 6K arrows firing at a constant rate. You can't fire constant rate killing mobs and the only boss that's gonna last 6K arrows are those level 150 bosses on the field. Except for soul hunter I don't think you want to be counting data fighting those, and if you're fighting Soul Hunter you're gonna be shooting EXTRA SLOW.

Your missing the point of his question, its not whether or not its feasible or cost effective. He's wondering if the arrow even works at all. To test it against melees, but not even see if it works on archers is kinda ludicrous.

Bobncut - Sanctuary

@OMGLAZERZ: How many misses did you have during this exercise? I'm assuming that when you say 6760 "hits" you are not counting misses so the estimates are consistent. However, the truncation of missed crits will affect confidence intervals. If we do not know the number of misses, we can at least approximate how significant this might be from knowing your miss percentage against the mobs you used.

Mosabi - Heavens Tear

this thread is giving me a headache close it please LALALALALLALALALA b:cryb:shockedb:cry

Grgumsh - Heavens Tear

ok... anyone that is a math major raise your hand, those of you who are just pretending leave the room now. *walks out*

Bobncut - Sanctuary

Mosabi - Heavens Tear wrote: »

this thread is giving me a headache close it please LALALALALLALALALA b:cryb:shockedb:cry

You believed it when classmates joked that you would never need math in the rest of your life, didn't you? Gotcha!

OK, now that we added drama to make this uniquely useful discussion more like the rest of the posts on this forum, can we continue without hurting anyone?

Psynopsis - Heavens Tear

OK, yes, the sample size is a little small. But that would depend on the mean, standard deviation, and median of the data, none of which were listed in your posts.

In addition, there are a lot of factors that could have screwed it up. Did you ever change equipment? Did you ever level in between hits? Did you get buffs from a passing player, or did they disappear during that time? If the answer was yes to any of those questions, the statistics you listed are 100% invalid, because we cannot tell which data are accurate.