you are free to believe what you want, however the fact is his figures are not absolute. they are anecdotal at best.
Sure i wont use his stats it to risk my life circum-navigating the moon and back. However, determining whether to risk "open the box! open the box!" or get some sparks to get the sharandar campaign progressed a lot faster on an alt based on the the results of 1111 openings of it, that is most likely sufficient in this case i think.
Statistics without context.... yeah i dont need the cotton wool thanks.
you have statistical data only. so you have 43/1111 = 0.0387 = 3.87%
if there were more players that had done this test with shared results, you still would have a drop rate based on statistical data only. this is not absolute and no one should put stock in these figures.
Nevertheless, statistics are a well known and respected way of converting the unknown into the known. 1111 is a large sample size, and I am happy to go with it. A simple calculation will yield a 95 and 99% confidence interval, but until someone does, I am happy to go with 4%.
0
syka08Member, Neverwinter Beta UsersPosts: 0Arc User
you are free to believe what you want, however the fact is his figures are not absolute. they are anecdotal at best.
But more valid than most. I know I was calling for people to try and post higher attempts than 50 and 100. I was quite surprised when I saw those attempts. That being said, you do have some good points that I hadn't thought of at that time and after speaking with others regarding statistical testing of things.
One, is that with any experiment, you want to do things multiples of times and in sets. So 1 person doing 11 sets of 100 (more or less in this case) does hold more validity than others doing much smaller batches. Obviously getting even 10 people to do 11 sets of 100 and comparing and compiling results would make the argument that much stronger. Continuing a pattern like this would make it less necessary to know how many sides the dice has or if part of their program deals with selecting the interstellar coordinates of a randomly selected planet at any given moment.
But then we also have the problem of not knowing whether or not the drop rate has been tweaked since that test was done. As you pointed out and many can relate, there are a lot of fixes and tweaks that go on behind the scenes between patches that never wind up in the patch notes. Plus, as a prime example of things changing between preview and live we have the Skeleton from the Halloween event. On preview it was purple and cost more, on live it was green and cost about 3 times as less.
In essence, I guess it would be just as foolish to assume that the % drop chance has not been tweaked since the test was done as it would be to assume that it has. It's an unknown-unknown, but one that could be put to rest, in some sense, if the test were repeated every patch. We still have the irk that we can't test it in such large quantities on the Live server, obviously, but I suppose that's where something like trust rolls in. Or, y'know... we all just migrate to the PTS and play over there full time instead of on live =P.
Fun stuff to think about while I sit here at work.
If j0shi82 performed the test as he described, and if the devs made no further changes to the drop rate before implementing it on live, then his result of p=3/87% is very strong evidence.
A 95% confidence interval around his result would be (2.74%, 5.01%). Further, if the null hypothesis tested were that the original drop rate was 10%, then his results rejects that hypothesis with a p-level of about 0.0000000000048.
If it was 10% or higher before, it wasn't on the test server when j0shi82 did his experiment.
If j0shi82 performed the test as he described, and if the devs made no further changes to the drop rate before implementing it on live, then his result of p=3/87% is very strong evidence.
A 95% confidence interval around his result would be (2.74%, 5.01%). Further, if the null hypothesis tested were that the original drop rate was 10%, then his results rejects that hypothesis with a p-level of about 0.0000000000048.
If it was 10% or higher before, it wasn't on the test server when j0shi82 did his experiment.
that's a very big if. the sample section of 1111 coffers were "bugged" in that they were originally the old coffers showing up on the test shard as the new coffers. there's no evidence that show either way that the coffers that we have now are the same or different. as a matter of fact, if that sampling was done today, it would provide better statistical data as the build on the test shard is closer to what is on the live shard. but it would still be anecdotal evidence.
My opnion is i'll take j0shi's testing over your opinions any day.
He provided far more detail than "it might not have changed" or "1111 isnt enough". Sure is for me.
We believe what we choose.
I choose to believe 5%.
I'm another person who would take j0shi's test results over anyone's anecdotal and blindly optimistic "might have changed" any day. Mathematical results > opinion.
Furthermore there's just as much of an argument that the devs might well have tweaked the drop rate down further, it's not as if anyone can tell without a huge number of alts now the coffers are account bound.
RIP Neverwinter 26/06/2014
0
reagenlionel1Member, Neverwinter Beta UsersPosts: 0Arc User
edited December 2013
Since module 2. I've not gotten 1 C ward on 10 of my characters. And many times, I havent even gotten a preservation ward at all.
your testing on the preview server is null and void as anything on there is subject to change prior to it going live. and since you haven't done this test on the live shard, it cannot be proven. plus, wasn't this done via a bug where existing (old) coffers were showing up as new coffers?
also your probability % is faulty based on the fact that every time you open a coffer, the chance resets. had multiple people opened 1111 coffers, each of their results would have been different. just like other people can come here and report that they've opened 7 coffers with 0 coal wards dropping, i've opened about 5 coffers and have had 2 coal wards drop. what is the probability that i'll never get another coal ward in the next 20-30 coffers i open? according to you, it's probable. while you could take this accumulated data of coffer drops and create a statistical table based on your findings, it would not be absolute.
statistically speaking you have limited anecdotal data from the preview shard and nothing more.
You and numbers are certainly not besties. My goodness...
The only thing true to your post is the fact that my test is indeed a couple weeks old and the percentage could since have been tweaked. The rest is empiric nonsense.
Btw.: It's perfectly possible my test is useless but for completely other reasons than the ones mentioned.
You and numbers are certainly not besties. My goodness...
The only thing true to your post is the fact that my test is indeed a couple weeks old and the percentage could since have been tweaked. The rest is empiric nonsense.
Btw.: It's perfectly possible my test is useless but for completely other reasons than the ones mentioned.
there's no need to get personal.
there was hardly anything mathematical in my post for you to glean such an accusation.
just like there's hardly any mathematical proof that your test isn't anything more than statistical based on questionable evidence and subject matter. all you did was copy characters with existing coffers over to the test shard prior to the module going live where they were "changed" into the new coffers--an action that was changed and stated as such by the devs in the thread where you reported your findings--and then you opened them and reported your findings as fact: the drop rate is around 5%. and you did it again in this thread. based on the fact that you have no internal information to back your claims, your test is only statistical data of a sampling done within questionable settings.
if i were to roll a 20 sided dice once, what is the probability that i would roll a 4? and what is the % of probability if i roll the dice 1000 times? 10000 times?
how about if i were to roll an unknown sided dice 1,111 times? what is the probability % that i'd roll a 4?
All statistics have to be taken with a grain of salt for sure and yeah for that matter your are right: The numbers are not absolute but indicate that the drop rate was around 5% at the time I made the test. But as others have stated the test, within its context, is very robust so saying the drop rate actually IS 5% is not that far off.
If you're saying the setting was questionable, fine. That's a legit concern and absolutely debatable. I disagree though that the "bugged" coffers have produced other numbers than the "new" coffers. But I soon will have enough coffers to repeat the test under the current build. We'll see.
Until then based on the evidence I'd set the droprate around 5%. Bring on a test that proofs otherwise and we'll talk again.
It can also be said that the larger the number of samples the more confident the estimate will be.
Making assumptions about the underlying distribution etc. it will also be possible to give a confidence
interval for a given number of samples. But I'm far from being an expert in statistics and don't know
how to do it.
If opening 1,000 coffers gives a probabality of 5%, it is unlikely - though not impossible - that
opening 1,000,000 coffers will give a totally different probability (say 15%) .
Going by what I have received from my first 11 coffers since the patch I'm thinking that J0shi82's numbers are still valid , not one single coalescent ward OR preservation ward just trash , 5% or lower sounds right to me , or at least feels right , pray 7 days to get a peridot and 1 or 2 rank 4 enchantments? makes the effort seem really worthwhile...
Nowadays: 10 chars invoking, 0 wards since Mod2. ALL empirical evidence points from EVERYONE posting numbers indicates that j0shi82's numbers are still accurate.
16 characters on my account, and ZERO Blue wards so far in module 2 - got three green though. I used to get 3 Blues and several greens each week before on average, so it does seems highly likely that the drop rate has changed (downwards).
0
baylen76Member, Neverwinter Beta Users, Neverwinter Guardian UsersPosts: 0Arc User
16 characters on my account, and ZERO Blue wards so far in module 2 - got three green though. I used to get 3 Blues and several greens each week before on average, so it does seems highly likely that the drop rate has changed (downwards).
Mconosrep: While both yours and my current coal ward drop rate is below 5%, we've both not even covered the postulated range once (5% = 1 in 20, we both sit below 20 openened coffers) - if you get a single coal ward on your 17th coffer you'd be above 5% already (1/17=5,88%).
Our sample sizes are simply still too small to matter. J0shi82 covered his deducted range of 20 more than 55 times (1.111/20=55,55), making his test a lot more statistically significant (in layman's terms, may the math guys forgive me).
Personally the wards don't irk me as much as the marks anyway, it's like barking at yesterday's tree. They've lowered the drop rate, I've upped the invoking chars. They put the bar higher still and there'll be a point where I won't bother any more. Still have a couple of games unplayed from the Steam autumn sale.
Mconosrep: While both yours and my current coal ward drop rate is below 5%, we've both not even covered the postulated range once (5% = 1 in 20, we both sit below 20 openened coffers) - if you get a single coal ward on your 17th coffer you'd be above 5% already (1/17=5,88%).
Our sample sizes are simply still too small to matter. J0shi82 covered his deducted range of 20 more than 55 times (1.111/20=55,55), making his test a lot more statistically significant (in layman's terms, may the math guys forgive me).
Actually I have opened the coffers twice each so it is 16*2 = 32 tried which is the (very) bottom limit for statistical significance for testing probabilities of 5% or more. (n>=30 tests being enough to roughly approximate the normal distribution).
Of course you would ideally want at least 100, but even at 32 tries it is possible to tell with ~75% accuracy if the base probability is 5% or 15%.
if i were to roll a 20 sided dice once, what is the probability that i would roll a 4? and what is the % of probability if i roll the dice 1000 times? 10000 times?
It's exactly this sort of question which statistical estimation is designed to answer. It doesn't answer it absolutely, that's an unknowable, but it can answer it to a very high, even an astronomically high degree of confidence.
When we roll the d20 mentioned above 1000 times or 10000 times, this may sound like the probabilities of these must be unknowable, but in fact, these are easily computed absolute probabilities from the Binomial Distribution, which I think someone linked the wiki article from in this thread somewhere. Because they're computable, we can actually test for that die being fair, or even being a d20, and return a very confident answer with a bound on how far the result might plausibly be off.
We use a sample of results, in this case success or failure for whether a "4" was rolled. From the sample, we compute the rate of success. Perhaps we only rolled it once, and got a "Not a 4" result. From that we'd estimate the rate to be zero. Which is certainly possible -- maybe it's a 2-sided die (a coin). We don't know, but the rate is still consistent with a d20, since on one roll a d20 frequently won't roll a 4. With only one roll, we have a very wide confidence interval -- it doesn't rule much of anything out on a likelihood basis. However, after we run 1111 trials, and we get 43 "It's a 4" results, then we can establish that if, say, it's a d10, then this result from 1111 rolls is wildly improbable. In the case of the experiment here, it's basically one in a trillion. The evidence doesn't support it being a d10. If the true rate were then shown to be 10%, then this would be a freak result more rare than having a meteor smash down into your room.
This sort of statistical testing takes on many forms and can deal with enormously complex problems; and the one here isn't any more difficult than ones first year statistics students would deal with in their homework. We can wonder if the rate was changed by the devs after the experiment was run; we can wonder if the experiment was run correctly, we can even wonder if the tester gave us all the test results, or even if the tester ever actually ran the experiment, but the statistics here are so strong it's not reasonable to argue against them on a mathematical basis.
My conclusion: don't use coalescent wards on anything but weapon and armor enhancements. Worst case price for preservation wards is 5k AD, and you need only 10 on average even for R10 enchantments. You only have 5% chance of wasting more than coalescent's worth on making R10, and even that will be true not for long because wondrous chests content is BOP now and coals price gonna skyrocket.
Well, goodness, I seem to burn streaks of ten green wards on a 40% chance more often than not. Your conclusion would be useful info, with a reasonable RNG, with proper distribution. However, it seems horribly clumpy, or possibly something more sinister.
not really. coal wards from the coffer of augmentation are still here. they're just BoA now. let's not get people upset over incorrect information. thanks.
Still there with an extremely low drop rate.
They don't want us creating 50 characters just for invoking, yet here we are....
You Mods/Devs don't get it. Grind is fine...Grinding until your eyes bleed is NOT fine.
The bottleneck to end game enchants is and always has been coal wards.
I'm thinking 5% drop rate is too high...it should be .0001%. If you are wanting to drive folks off then do it right! /sarcasm
They don't want us creating 50 characters just for invoking, yet here we are....
This. Reducing the drop rate of coal wards to discourage an army of invocation chars is asinine. That has completely the opposite effect. I only needed my 5 chars (1 of each class) in order to average 1 coal a week. Now I have had ZERO in the last 2 weeks, so apparently I need MORE chars to maintain the same 1 a week average.
Also, lets not mince words here. Did not one of the mods post about how the intent was originally to remove the 7 day coffer all together, so that means Devs really did want to kill off free coal wards. If I am wrong there, please show me where I am wrong.
Foundry - Fight Club? (nw-dluqbofu7)
- JailBreak (in development)
0
beckylunaticMember, NW M9 PlaytestPosts: 14,231Arc User
Also, lets not mince words here. Did not one of the mods post about how the intent was originally to remove the 7 day coffer all together, so that means Devs really did want to kill off free coal wards. If I am wrong there, please show me where I am wrong.
Nothing was said in so many words. I'm not about to go dig it up, but the official response to, "What is up with this Sharandar treasure BS in the Vault of Piety?" was "Oops, we're sorry we changed that and didn't tell you. We're putting in a revised coffer as a 7-day reward again."
I don't think anything has ever been said about intent one way or the other.
well this topic has been beaten to death and serves to slam one aspect of changes post m2. the fact is, a rough estimate is just that. if every player came here to report their individual experience, they would all be different. pre m2, the feedback thread was opened and touched on all of these points. the devs heard your feedback then and with the continuation of these arguments in this thread, i'm sure they're still hearing them. your estimations may be right, but they may also be wrong and since there won't ever be a confirmation one way or the other, the point is moot.
Comments
Sure i wont use his stats it to risk my life circum-navigating the moon and back. However, determining whether to risk "open the box! open the box!" or get some sparks to get the sharandar campaign progressed a lot faster on an alt based on the the results of 1111 openings of it, that is most likely sufficient in this case i think.
Statistics without context.... yeah i dont need the cotton wool thanks.
Nevertheless, statistics are a well known and respected way of converting the unknown into the known. 1111 is a large sample size, and I am happy to go with it. A simple calculation will yield a 95 and 99% confidence interval, but until someone does, I am happy to go with 4%.
But more valid than most. I know I was calling for people to try and post higher attempts than 50 and 100. I was quite surprised when I saw those attempts. That being said, you do have some good points that I hadn't thought of at that time and after speaking with others regarding statistical testing of things.
One, is that with any experiment, you want to do things multiples of times and in sets. So 1 person doing 11 sets of 100 (more or less in this case) does hold more validity than others doing much smaller batches. Obviously getting even 10 people to do 11 sets of 100 and comparing and compiling results would make the argument that much stronger. Continuing a pattern like this would make it less necessary to know how many sides the dice has or if part of their program deals with selecting the interstellar coordinates of a randomly selected planet at any given moment.
But then we also have the problem of not knowing whether or not the drop rate has been tweaked since that test was done. As you pointed out and many can relate, there are a lot of fixes and tweaks that go on behind the scenes between patches that never wind up in the patch notes. Plus, as a prime example of things changing between preview and live we have the Skeleton from the Halloween event. On preview it was purple and cost more, on live it was green and cost about 3 times as less.
In essence, I guess it would be just as foolish to assume that the % drop chance has not been tweaked since the test was done as it would be to assume that it has. It's an unknown-unknown, but one that could be put to rest, in some sense, if the test were repeated every patch. We still have the irk that we can't test it in such large quantities on the Live server, obviously, but I suppose that's where something like trust rolls in. Or, y'know... we all just migrate to the PTS and play over there full time instead of on live =P.
Fun stuff to think about while I sit here at work.
A 95% confidence interval around his result would be (2.74%, 5.01%). Further, if the null hypothesis tested were that the original drop rate was 10%, then his results rejects that hypothesis with a p-level of about 0.0000000000048.
If it was 10% or higher before, it wasn't on the test server when j0shi82 did his experiment.
that's a very big if. the sample section of 1111 coffers were "bugged" in that they were originally the old coffers showing up on the test shard as the new coffers. there's no evidence that show either way that the coffers that we have now are the same or different. as a matter of fact, if that sampling was done today, it would provide better statistical data as the build on the test shard is closer to what is on the live shard. but it would still be anecdotal evidence.
Furthermore there's just as much of an argument that the devs might well have tweaked the drop rate down further, it's not as if anyone can tell without a huge number of alts now the coffers are account bound.
Really not liking these new coffers.
You and numbers are certainly not besties. My goodness...
The only thing true to your post is the fact that my test is indeed a couple weeks old and the percentage could since have been tweaked. The rest is empiric nonsense.
Btw.: It's perfectly possible my test is useless but for completely other reasons than the ones mentioned.
there's no need to get personal.
there was hardly anything mathematical in my post for you to glean such an accusation.
just like there's hardly any mathematical proof that your test isn't anything more than statistical based on questionable evidence and subject matter. all you did was copy characters with existing coffers over to the test shard prior to the module going live where they were "changed" into the new coffers--an action that was changed and stated as such by the devs in the thread where you reported your findings--and then you opened them and reported your findings as fact: the drop rate is around 5%. and you did it again in this thread. based on the fact that you have no internal information to back your claims, your test is only statistical data of a sampling done within questionable settings.
if i were to roll a 20 sided dice once, what is the probability that i would roll a 4? and what is the % of probability if i roll the dice 1000 times? 10000 times?
how about if i were to roll an unknown sided dice 1,111 times? what is the probability % that i'd roll a 4?
If you're saying the setting was questionable, fine. That's a legit concern and absolutely debatable. I disagree though that the "bugged" coffers have produced other numbers than the "new" coffers. But I soon will have enough coffers to repeat the test under the current build. We'll see.
Until then based on the evidence I'd set the droprate around 5%. Bring on a test that proofs otherwise and we'll talk again.
The law of large numbers (http://en.wikipedia.org/wiki/Law_of_large_numbers) states that
the estimate will converge at (or near) the actual probability.
It can also be said that the larger the number of samples the more confident the estimate will be.
Making assumptions about the underlying distribution etc. it will also be possible to give a confidence
interval for a given number of samples. But I'm far from being an expert in statistics and don't know
how to do it.
If opening 1,000 coffers gives a probabality of 5%, it is unlikely - though not impossible - that
opening 1,000,000 coffers will give a totally different probability (say 15%) .
16 characters on my account, and ZERO Blue wards so far in module 2 - got three green though. I used to get 3 Blues and several greens each week before on average, so it does seems highly likely that the drop rate has changed (downwards).
Mconosrep: While both yours and my current coal ward drop rate is below 5%, we've both not even covered the postulated range once (5% = 1 in 20, we both sit below 20 openened coffers) - if you get a single coal ward on your 17th coffer you'd be above 5% already (1/17=5,88%).
Our sample sizes are simply still too small to matter. J0shi82 covered his deducted range of 20 more than 55 times (1.111/20=55,55), making his test a lot more statistically significant (in layman's terms, may the math guys forgive me).
Personally the wards don't irk me as much as the marks anyway, it's like barking at yesterday's tree. They've lowered the drop rate, I've upped the invoking chars. They put the bar higher still and there'll be a point where I won't bother any more. Still have a couple of games unplayed from the Steam autumn sale.
Actually I have opened the coffers twice each so it is 16*2 = 32 tried which is the (very) bottom limit for statistical significance for testing probabilities of 5% or more. (n>=30 tests being enough to roughly approximate the normal distribution).
Of course you would ideally want at least 100, but even at 32 tries it is possible to tell with ~75% accuracy if the base probability is 5% or 15%.
It's exactly this sort of question which statistical estimation is designed to answer. It doesn't answer it absolutely, that's an unknowable, but it can answer it to a very high, even an astronomically high degree of confidence.
When we roll the d20 mentioned above 1000 times or 10000 times, this may sound like the probabilities of these must be unknowable, but in fact, these are easily computed absolute probabilities from the Binomial Distribution, which I think someone linked the wiki article from in this thread somewhere. Because they're computable, we can actually test for that die being fair, or even being a d20, and return a very confident answer with a bound on how far the result might plausibly be off.
We use a sample of results, in this case success or failure for whether a "4" was rolled. From the sample, we compute the rate of success. Perhaps we only rolled it once, and got a "Not a 4" result. From that we'd estimate the rate to be zero. Which is certainly possible -- maybe it's a 2-sided die (a coin). We don't know, but the rate is still consistent with a d20, since on one roll a d20 frequently won't roll a 4. With only one roll, we have a very wide confidence interval -- it doesn't rule much of anything out on a likelihood basis. However, after we run 1111 trials, and we get 43 "It's a 4" results, then we can establish that if, say, it's a d10, then this result from 1111 rolls is wildly improbable. In the case of the experiment here, it's basically one in a trillion. The evidence doesn't support it being a d10. If the true rate were then shown to be 10%, then this would be a freak result more rare than having a meteor smash down into your room.
This sort of statistical testing takes on many forms and can deal with enormously complex problems; and the one here isn't any more difficult than ones first year statistics students would deal with in their homework. We can wonder if the rate was changed by the devs after the experiment was run; we can wonder if the experiment was run correctly, we can even wonder if the tester gave us all the test results, or even if the tester ever actually ran the experiment, but the statistics here are so strong it's not reasonable to argue against them on a mathematical basis.
Well, goodness, I seem to burn streaks of ten green wards on a 40% chance more often than not. Your conclusion would be useful info, with a reasonable RNG, with proper distribution. However, it seems horribly clumpy, or possibly something more sinister.
Still there with an extremely low drop rate.
They don't want us creating 50 characters just for invoking, yet here we are....
You Mods/Devs don't get it. Grind is fine...Grinding until your eyes bleed is NOT fine.
The bottleneck to end game enchants is and always has been coal wards.
I'm thinking 5% drop rate is too high...it should be .0001%. If you are wanting to drive folks off then do it right! /sarcasm
Also, lets not mince words here. Did not one of the mods post about how the intent was originally to remove the 7 day coffer all together, so that means Devs really did want to kill off free coal wards. If I am wrong there, please show me where I am wrong.
- JailBreak (in development)
Nothing was said in so many words. I'm not about to go dig it up, but the official response to, "What is up with this Sharandar treasure BS in the Vault of Piety?" was "Oops, we're sorry we changed that and didn't tell you. We're putting in a revised coffer as a 7-day reward again."
I don't think anything has ever been said about intent one way or the other.
Neverwinter Census 2017
All posts pending disapproval by Cecilia
so this thread is closed.