@commodoreshrvk I would put forth the observation, you reaction a bit Conformational Bias.. the OP's sample size as other have point out is too small to tell if the RNG is biased or now.
My statements or test do not suffer from confirmation bias so please read that article Borticus put up again. What I did is test the observed data to the expected pattern and even completed a sample size determination of the power of the data. There is no emotion present whatsoever in the statistical tests I put forth or the probabilities I have provided. The tests I conducted are routine for testing such distributional questions. Now, given the numerical, statistical, and probablistic evidence provided, consoles, regardless of quality, are not dropping uniformly and a sample size of 300 is sufficient to state that.
Whether people want to "believe" it or not in the face of the statistical and probabilistic fact is, well up to them. The only assumption we make in the whole analysis is that the OP's data are accurate, i.e. his sample is representative of the population. That assumption is made in ANY statistical analysis not just this one.
A lot of things that the players have had a gripe with.... STF drops, lock boxes, console fabbing... they all hinge on that RNG. Every-time I have complained about this stuff, I always mentioned the random number generator. I have never thought it was random enough.
You can easily get a 'sense' of it with the lock boxes. I stopped opening them quickly because of my suspicion of the RNG. A lot of RNG's use the current time as a seed and to be sure, I open a lock box every few minutes or open one in between running STFs and the like. I find that I get a better chance of getting a larger variety of stuff vs getting 10 Gamma Quadrant mini-doff packs it less than 3 seconds.
For example, I assume that all the mini-packs have the same drop chance from a lock box. If not that would counter my perception... but anyway, whenever I open like 5 or 6 boxes all at once, I'll get the same mini-doff pack over and over where if I wait a few seconds or a minute I'll get a mixture of doff packs, weapon packs and whatever. I know it could be just chance, but I think it's a crappy RNG.
Just try it on a test shard, give yourself 100 keys and 100 boxes and open them as fast as you can click and then do it again while waiting 10, 15, or 30 seconds or so. You'll notice you get large groups of the same item over and over while taking the fast approach. Taking the slow approach might net you some mid level items and a better mix of low level items.
Now I know you guys use C++ and you probably don't have your RNG in a loop like this example, but see here:
Even something as silly as where the RNG is initialized made a difference for the unrelated case above. What if STO suffers from such a small, almost unnoticeable oversight like the case above? I am glad you have reached out to your peers on the topic of the RNG. Perhaps you convince them to schedule a 15 min code review of the RNG?
join Date: Sep 2009 - I want my changeling lava lamp!
But what really convinces me is the fact that not only are the tactical energy type consoles low in occurence, but the 2 most popular and expensive ones (phaser relay and AP mag) are the lowest of them all. Same with the Field Gens, Neutroniums and Monotaniums. All the most popular consoles of their type, and all with the lowest occurence. I appreciate Bort taking the time to look into the drop table, but this is an undeniably rigged drop system.
I get these consoles quite often. As I stated before, only frustrated people will tend to post in such threads. The console i get the most? The shield one! Blue, purple, it likes me.
@jam062307 getting 3 Phaser consoles and 1 Disruptor Console would tend to disprove the theory the RNG is biased. And @commodoreshrvk I would put forth the obseration, you reaction a bit Conformational Bias.. the OP's sample size as other have point out is too small to tell if the RNG is biased or now.
The system folks could very easy do tests using the game engine to prove bias one way or other. Put a huge amount of NPCs on the map, code them to use the DOFF assignment loot tables make them give lots of loot drops using said tables, and use an "I Win Button" ability which kills everything on the map at once, and then collect and analyze the drops. Once a sufficient sample size is achieved (preferably several times) any RNG bias should be obvious.
What are you talking about? This thread is about crafting Powered Alien artifacts from the Protoype Console DOFF mission you get from your Eng officer. If a Dev wants to give my Tribble account the ability to run this mission with no cooldown and iinfinite Powered Alien Objects and 5 purple Eng DOFFS with the required Crit traits for %24-%51-%24. I will spen a week and do a few thousand runs and document the results for everyone to see. Or I know they are busy but can they Just created a test account with a macro that runs this mission over and over with a bank large enough to hold a few thousand items and the review the results. Since my OP several other players have stated that they are seeing the same junk consoles like Bio function/Pre fire chambers/stealth modules/ over and over again. The consoles that drop the most are the generic beam/cannon/ torp for Tac, For SCI it is Bio/Stealth/Power insulator and for Eng,Plasma Dist manafold/Field Emitter/Injector Assembly
A lot of things that the players have had a gripe with.... STF drops, lock boxes, console fabbing... they all hinge on that RNG. Every-time I have complained about this stuff, I always mentioned the random number generator. I have never thought it was random enough.
You can easily get a 'sense' of it with the lock boxes. I stopped opening them quickly because of my suspicion of the RNG. A lot of RNG's use the current time as a seed and to be sure, I open a lock box every few minutes or open one in between running STFs and the like. I find that I get a better chance of getting a larger variety of stuff vs getting 10 Gamma Quadrant mini-doff packs it less than 3 seconds.
For example, I assume that all the mini-packs have the same drop chance from a lock box. If not that would counter my perception... but anyway, whenever I open like 5 or 6 boxes all at once, I'll get the same mini-doff pack over and over where if I wait a few seconds or a minute I'll get a mixture of doff packs, weapon packs and whatever. I know it could be just chance, but I think it's a crappy RNG.
Just try it on a test shard, give yourself 100 keys and 100 boxes and open them as fast as you can click and then do it again while waiting 10, 15, or 30 seconds or so. You'll notice you get large groups of the same item over and over while taking the fast approach. Taking the slow approach might net you some mid level items and a better mix of low level items.
Now I know you guys use C++ and you probably don't have your RNG in a loop like this example, but see here:
Even something as silly as where the RNG is initialized made a difference for the unrelated case above. What if STO suffers from such a small, almost unnoticeable oversight like the case above? I am glad you have reached out to your peers on the topic of the RNG. Perhaps you convince them to schedule a 15 min code review of the RNG?
I also noticed that if I start another Prototype Console crafting as soon as one is finished (doing them back to back) I will get the same console as list time or another junk console) but when i allow at least a 2-4 hour cooldown I have better results. And the guy who thinks that nothing is wrong beacuse he has gotten a few good consoles obviously has not done this mission a much as myself and some of the other posters that have done this mission hundreds of times. There is a member of my fleet that did it only twice with blue and green doffs and got purple Antiproton Consoles both times. So yes good consoles do drop but after doing the mission several hundred times I see, and many other see a distinct pattern that is showing that it is not random and that the lesser value consoles drop at a much hight rate than the high value consoles do.
I get these consoles quite often. As I stated before, only frustrated people will tend to post in such threads. The console i get the most? The shield one! Blue, purple, it likes me.
Well the easy solution to this if you "feel" (and I mean feel because the stats are there) that the OP's results are somehow overly biased on only represent that very edge case which does not represent the population is to post your suite of data which can then be pooled with the OP's and the analysis can be re-done.
Well the easy solution to this if you "feel" (and I mean feel because the stats are there) that the OP's results are somehow overly biased on only represent that very edge case which does not represent the population is to post your suite of data which can then be pooled with the OP's and the analysis can be re-done.
This guy is just trolling.
There were 1 or 2 other people that posted here that have crafted as many if not more than I have. I ask them to please start recording your results and post them here. If a test account of some kind on tribble cannot be created to test this then I would hope the Devs would run a test account on this mission and do 4 or 5 hundred runs of it to see the honest results of it for them selves.
A lot of RNG's use the current time as a seed and to be sure, I open a lock box every few minutes or open one in between running STFs and the like. I find that I get a better chance of getting a larger variety of stuff vs getting 10 Gamma Quadrant mini-doff packs it less than 3 seconds.
I have also noticed this. Not just opening boxes. I have found doff missions that finish within a few seconds of each other often have the same crit success/falure resault.
Well the easy solution to this if you "feel" (and I mean feel because the stats are there) that the OP's results are somehow overly biased on only represent that very edge case which does not represent the population is to post your suite of data which can then be pooled with the OP's and the analysis can be re-done.
What's the most credible possibility? Borticus lying to us for evil purposes or some players being incredibly unlucky? At some point, even something almost impossible happens. It would be abnormal if we didn't have any exceptions, "weird" and unexpected outcomes and so on.
The RNG might be broken. I'm not trolling when i say i get the shield console quite often. I'm a bit curious to see the results of the investigations about it. It would affect more than this assignment if it was flawed though, unless the game has several RNGs for different parts of the game, which would be weird and quite inefficient by design (spreading RNG rolls across players and systems would likely increase the 'randomness'). It would affect damage dealt by ships, other assignments, and so on. If it was true, i'm quite sure that the pvp community would be screaming about it, with threads like "way too much/not enough crits", because those guys are pretty good at monitoring such flaws.
I'm definitely not saying it doesn't require further studies, but only that you should expect - and accept - a "working as intended" answer. There are exceptions all the time. People win at lotteries everyday. You will find dozens of people having 20 or more coins of the same year in their wallets across the world. Not to be mean but you may also very well be the victim of the common belief that if the odds are extremely low it shouldn't happen. But it does. Everywhere. All the time.
Let's quote Sir Conan Doyle for posterity.
"When you have eliminated the impossible, whatever remains, however improbable, must be the truth".
Well the easy solution to this if you "feel" (and I mean feel because the stats are there) that the OP's results are somehow overly biased on only represent that very edge case which does not represent the population is to post your suite of data which can then be pooled with the OP's and the analysis can be re-done.
I'll start recording mine this weekend. I just rounded up 24 Strange Artifacts from NADORC assignments earlier this week and started crafting on 2 different toons this morning.
There were 1 or 2 other people that posted here that have crafted as many if not more than I have. I ask them to please start recording your results and post them here. If a test account of some kind on tribble cannot be created to test this then I would hope the Devs would run a test account on this mission and do 4 or 5 hundred runs of it to see the honest results of it for them selves.
I would have to agree. This thread has attracted some trolls that aren't really contributing with anything but their own opinion. More people need to be collecting hard data and posting it here. Here's a fact. The Devs have been known to make errors when trying to track down the reasons why certain things behave the way they do. Anyone that's been around this game for awhile knows how true this is. That's why threads like these get started and it's a good thing too.
I was thinking about this last night a little bit, and I asked myself some things like what makes X so valuable, is it rarity or demand?
I wonder if anyone has documented the pre-S7 drop rate of STF gear. How often did you get an AP console? Pretty freaking rare right? Phaser consoles? I can't even remember. So we always felt those were weighted, what if they are using the same mechanic?
I have also noticed this. Not just opening boxes. I have found doff missions that finish within a few seconds of each other often have the same crit success/falure resault.
I remember a thread a while back in either the PvP or PvE forum about how weapons fire would sometimes have a long string of crits for no obvious reason. The speculation in that thread is/was the same as here. They thought that a flaw in the RNG was causing it to re-use an RNG roll instead of loading a new one.
"Incredibly unlucky" really does not come close to explaining the OP's results in the light of a uniform drop rate. Right now, the most probable event is something odd in the code chain (not necessarily the RNG). So let us go through the logical decision process so everyone can see where we currently stand. If you want to follow the hyethetico-deductive model, we can do that. We have before us five hypotheses that we have to decide on and unfortunately, they are not mutually exclusive.
1 - The OP has misrepresented his data - This is likely not the case because the OP has expressed a genuine concern and took a great deal of time and effort to collect the data.
2 - Borticus has misunderstood the table or the reward process - Again, we can say with certainty this is not the case. Borticus is familiar with DOFF system and their loot table structure.
3 - The observed results are spurious resulting purely from random chance (i.e. the sample does not reflect the population) - This cannot be denied, however the probability of such an event from the analyses provided is extremely improbable. Remember we have disproven the null hypothesis that the observed distribution and the expected distribution are the same. They are different and the results of those tests provide us the probability of that difference occurring by random chance alone.
4 - The observed results represent a sampling inadequacy (i.e. the sample size is not large enough to detect a difference) - Again, I have gone through some quite basic statistical power analyses that disprove this option. At this point whether people wish to believe it or not, the sample size issue is no longer a valid argument. We have a sufficient sample size to make a determination on the observed differences. Those still arguing a sample size issue are doing so from belief, feelings, or emotions and thus have no logical grounds anymore.
5 - Somewhere in the game code lies an issue that is skewing the observed results from the expected results (the drop rate in the loot table) - Somewhere in the chain of code from the jobs results to the draw from the loot table, the results are getting skewed. This could be anything and it does not matter if we specify it or not. We know it lies in that path because the initial roll on quality is working and we know that the values for the consoles in the loot table are equal (uniform). Something is breaking down in the process from point A----> point B.
We CAN rule out hypotheses 1 and 2 based on the character and knowledge of the two parties in question.
We CANNOT rule out hypothesis 3, but we can say it is highly improbable.
We CAN rule out hypothesis 4 because we have already determined ~300 consoles is enough to make a statement on the observed difference.
We CANNOT rule out hypothesis 5 because none of the parties know the code process.
We have two options here to decide definitively between the two:
1 - Obtain more data of similar sample sizes (replicates) to see if the observed pattern the OP has holds true or is falsified.
2 - Examine the code path for potential factors skewing the results.
If we are to make a determination, now between the last two competing hypotheses without any additional information, we have to invoke Occam's Razor. That invocation would determine hypothesis 5 as the most likely, because it is the simplest and most parsimonious. Given the evidence, your fictional sleuth would also logically deduce hypothesis 5.
Now, again just because you or anyone else is getting consoles you feel are valuable, your statement does not improve the position that the results fit hypothesis 3 or 4. Such arguments are from an emotional standpoint using a qualitative assessment as fact. Such arguments are definitely suffering from a conformational bias, as the focus is on the consoles that are deemed valuable rather than the sample of consoles as a whole. Second, such statements are purely qualitative. Those qualitative metrics are erroneously being used to afford judgment on whether consoles are being obtained in equal proportion and against the quantitative measures (which is the crux of this entire argument). Third, the qualitative statements of, "I regularly...", "I got three of...", etc... also suffer from a bias because they are only providing a partial result and cannot be used to confirm or refute the quantitative data. Our process has moved beyond the qualitative now and we must focus on the quantitative.
The best course of action we can do now is provide more data to confirm or refute hypothesis 3 and pursue an investigation to confirm or refute hypothesis 5. Both are underway.
I remember a thread a while back in either the PvP or PvE forum about how weapons fire would sometimes have a long string of crits for no obvious reason. The speculation in that thread is/was the same as here. They thought that a flaw in the RNG was causing it to re-use an RNG roll instead of loading a new one.
I recall that thread. I think the way that the game engine "if one crits they all crit" behavior exaggerated player conclusions. And then getting lucky with two or more damage groupings in a row getting crits. That would making a highly suspect lucky streak under the assumption of 1 crit roll per tick of damage.
PS And I'll miss the if one crits they all crit behavior. Seeing a Breen Torpedo Cluster crit with "Attack Pattern Alpha", "Go Down Fighting" and "Fire on my Mark" running is GLORIOUS! :eek:
I recall that thread. I think the way that the game engine "if one crits they all crit" behavior exaggerated player conclusions.
Just so we're clear, the core cause of this bug (as I understand it - but I'm not a programmer) is that a single RNG roll is being propogated through multiple combat events, not several separate rolls that produce the same figure.
In other words, blaming a "sticky" RNG on this behavior is a false assumption.
Jeremy Randall
Cryptic - Lead Systems Designer
"Play smart!"
The best course of action we can do now is provide more data to confirm or refute hypothesis 3 and pursue an investigation to confirm or refute hypothesis 5. Both are underway.
Indeed these are the last two options.
If we make such data collections, i'd also like to include the conventional console craft (experimental console upgrade) to see if it's an isolated issue or not. I remember seeing odd results with that one but completely forgot it since i've stopped doing it for this specific reason. By weird i mean a lot of countermeasure systems and an eng console i forgot.
Ok, assuming there's no hidden value(s) and the reported numbers are true and all consoles have equal likelihood of dropping... Then the question becomes: Should all the consoles have an equal chance of dropping?
What I mean is:
From a lore/story standpoint, what's essentially happening is that some alien trading partner stumbles upon some gadget, doesn't know what it is, and sells it to you as a curiosity piece. Your science/engineering teams fiddle with it, make it work, and lo and behold it's some piece of alien technology that provides dramatic to (insert name here) system. Right?
So, what, are we to believe that all these highly advanced mysterious alien ships were loaded up with stealth modules, power insulators, and the like?
No, it's far more realistic that most of them had, you know, useful consoles on their ship. Therefor, there should be a weighted chance of acquiring a useful console from this assignment chain. The most commonly used consoles should be the most commonly generated consoles.
Just so we're clear, the core cause of this bug (as I understand it - but I'm not a programmer) is that a single RNG roll is being propogated through multiple combat events, not several separate rolls that produce the same figure.
In other words, blaming a "sticky" RNG on this behavior is a false assumption.
I wasn't thinking of mines but energy weapons.
But that's a good point about relevence. The RNG isn't being used the same way. Sticky or not, the rolls are days apart in the data.
I applaud your efforts in trying to reason it as a streak of bad luck but the probability of the OP not getting some of the consoles with 300 tries is (1/42)^300 = 1.05x10-487!!!
That is a 1 in 9.43x10^486 chance!!
Actually, this is incorrect. There are 42 different consoles that can be crafted using this mission. Assuming they are all equally likely, there is a 1 in 42 chance to get a specific one. So, there is a 1 in 42 chance of getting a Phaser Relay. This means there is a 41 in 42 chance of NOT getting a Phaser Relay.
The math becomes (41/42)^300 = 0.000725. Thus there is a 0.07% chance of not getting a Phaser Relay. This works out to a 1 in 1379 chance.
Actually, this is incorrect. There are 42 different consoles that can be crafted using this mission. Assuming they are all equally likely, there is a 1 in 42 chance to get a specific one. So, there is a 1 in 42 chance of getting a Phaser Relay. This means there is a 41 in 42 chance of NOT getting a Phaser Relay.
The math becomes (41/42)^300 = 0.000725. Thus there is a 0.07% chance of not getting a Phaser Relay. This works out to a 1 in 1379 chance.
I stand corrected and will graciously retract my statements pertaining to the probability of obtaining a phaser relay as I have calculated the chances of getting 300 phaser relays. It still does not change the probability that the OP observed distribution differs from the expected distribution. Also, it does not change that the observed distribution happening from "bad luck" is small.
Oh and P.S., the odds then of getting at least one Phaser Relay in 300 tries then becomes 1 - (41/42)^300 =0.999275 or 1 in 1.000725...yet that highly probable result did not happen.
I am continuing to Craft every day and will update by numbers on the OP to keep track of what I am getting. ( I got a Blue Antiproton Mag regulator and a purple Emergency force field today!!!!)
I have also noticed this. Not just opening boxes. I have found doff missions that finish within a few seconds of each other often have the same crit success/falure resault.
Yes me too and I even considered mentioning that, but the post was getting to long.
join Date: Sep 2009 - I want my changeling lava lamp!
I think you're definitely running into the problem of a very low sample size. While crafting 300 consoles requires a large investment of in-game resources, it's really a tiny number. I've crafted a few hundred of these things myself, and while I haven't kept a nice spreadsheet like you have to track the results, I've received two purple phaser relays, two purple antiproton mag regulators, and at least two of the purple tetryon consoles. There was even a purple field generator in there, too. And, yes, a staggering number of worthless consoles of all types and rarities.
Should it be weighted towards tactical consoles? But what if I *want* a biofunction monitor? Who decides what has value- what doesn't- for the weighting? Borticus? He doesn't want that job.
*Edit: I'll add that there may very well be a problem with the RNG in this game. I have noticed that if I open two reward packs together very quickly, they have a seemingly improbable chance of containing the same thing. But, again, that could just as easily be a sample size problem.
The math becomes (41/42)^300 = 0.000725. Thus there is a 0.07% chance of not getting a Phaser Relay. This works out to a 1 in 1379 chance.
Which means that if 2000 people craft 300 consoles apiece, you expect at least one guy who never gets a phaser array. Of course, there's plenty of other things he didn't get, either, but I point out that nobody(except me) bothered to mention that he didn't get a Biofunction Monitor either. I'm sure there's a lot more console crafters out there than merely 2000, so this has to happen to somebody.
Quite simply, I am not convinced that anything shady or interesting is occurring here. It's clear why the expected distribution of rarities matches the results: There are only 3 possible outcomes, with a roughtly 1:2:1 ratio of occurrence. 300 is a fairly sizeable sample size for testing this. However, there are 42 possible consoles...300 is NOT a big sample size to test something like that.
As for theory of sticky RNGs, I call bollocks on this. There are several reasons why RNGs become sticky: Failure to seed the RNG will result in an identical sequence being generated every single time. RNGs are therefore typically seeded with a value, generally some value derived from server time. The other possible way for an RNG to become sticky is if it this identical value is used to seed multiple instances of the same RNG. This would occur only if the time value had a low resolution (say, seconds), and the RNG was reseeded in several different threads that launched simultaneously. This, of course, cannot possibly describe the behavior of "waiting several hours" or even "several minutes" when it comes to opening boxes or queue doff missions. Hours and minutes are pretty much an eternity on a computer's timescale, and even if the resolution was as bad as "seconds", it would matter because of how doff missions resolve.
Now, how do doff missions resolve? An interesting thing occurs here: A doff mission does not resolve itself and determine its outcome until you look at it! How do I know this? Well, awhile back, I queued some doff missions. As is typical after one queues a doff mission, I buggered the hell off and switched to another toon. Hours later, the doff mission had finished, but as I didn't really consider it important, I hadn't bothered to look at it. I needed my toon for something on Tribble. I copied him over, did my test, and noted that my doff missions had ended badly. Curious about how this worked, I deleted the toon off Tribble, and copied myself again: Opening up my doff missions, I had gotten entirely different outcomes...outcomes which were again completely different when I finally opened the real version on Holodeck. In short, doff missions do not even resolve themselves until you look at them. There is no mechanism by which even a sticky RNG could possibly affect doff missions, unless every time you opened a sequence of the same doff mission, you always got the same result. This doesn't happen, so this sticky-RNG theory is bollocks.
Now, the two-reward-packs thing, THAT could very probably be a sticky RNG, IF every query submitted to open a pack was handled by a different thread spawned with the same seed, such as the second...but that would be very noticeable, very fast. Realistically, getting garbage out of a box is such a likely outcome that it should be no surprise at all when it repeats itself multiple times in a row. At least 99% of everything in a box is garbage, and there aren't all that many items to choose from.
If (great big IF) anything is truly amiss with this process, THIS would be the culprit.
All the additional hooplah over "hidden weighting" is, quite frankly, a pile of ... stinky things. The data is, what the data is, and I've already publicly stated exactly what that data is.
So, if (again, this is a hypothetical ONLY) there is anything off-kilter about the overall distribution of rewards from these tables, the only remaining culprit is the core RNG, about which I have approximately zero knowledge or input.
I'll send out a feeler to investigate, though, just for peace of mind. Don't get your hopes up about results, though.
The numbers are what they are, but I think that at least some of the outcry being posted here is largely motivated by an emotional reaction to those figures, due to the effect they have on your in-game currencies (including Time as a more abstract currency).
Ive had 5 Purple Tellarites since the artifacts started so I here is what I have noticed over the past year.
The Quality ratio seems to be fine. Ive always gotten blues and purples in about the right proportion cant really complain about that.
What i have noticed is that I do seem to get the same few over and over, in my case I get some that others have mentioned they dont see. I get a lot of phaser relays and a lot of bio function monitors.
Something I have noticed with a few doff related rewards is that the RNG seems off and tend to repeat the same rewards. This seems different per character. Lets use Instigate Defection. On three characters I tend to get the same one about every other time on each character. Sometimes it feels like everytime, So one charater might get Sizzta (energy weapons guy) over and over and another might get Alenia (?..the diplomat) over and over but then the other toon will see them rarely. Seems like each character has it own set of repeats.
So while there may not be some super secret weighting scheme, i do think that odds on items doesnt really feel random to me over all
I dug up my spread-sheet and here are results of crafting over 300 consoles: ( No one can convince me it's random) the Purple/Blue/Green Quality results were accurate running it at %24-%51-%24
I have done some addtional crafting since OP, I have updated the OP with my results.
Ok, so I'm going to help you out tracking this. I started crafting on 2 separate toons yesterday. I gather 24 Artifacts a week and usually only run the Fabricate Prototype Console assignment over the weekends. I've been running this routine ever since the Children's Toys chain was introduced. I still have nearly all the consoles which I made last weekend in my inventory. It looks as though I have sold a few, however.
Out of the last 21 consoles crafted I have made 3 Emitter Arrays (all blue) and 3 Emergency Force Field Consoles (2x blue, 1x purple).
I've literally made a thousand of these things and I consistently see some of the same ones being made over and over. I have such a tendency for Emergency Force Fields, Biofunction Monitors, Warhead Yield Chambers, Emitter Arrays, Field Emitters, Graviton Generators, Countermeasures, Inertial Dampeners and Sensor Probes you wouldn't believe it! I wish I had been tracking these earlier so I can add more data but I'm very glad you've made a thread where the data can be stockpiled because I know that you're on to something.
I'll make a post after every 50 consoles that I craft to update totals so there should be new totals to add to your list every 2 weeks.
As for theory of sticky RNGs, I call bollocks on this.
Now, how do doff missions resolve? An interesting thing occurs here: A doff mission does not resolve itself and determine its outcome until you look at it!
Well, we don't know how the STO RNG is implemented so anything we're doing is obviously hand-waving. Perhaps for performance reasons there is only one RNG roll per "tick" and that is used game-wide?
If DOFF missions truly do resolve when you look at it, then this becomes an easy issue to test. Everyone wait until they complete their next alien artifact mission, then tranfer their toon to Tribble. Assuming the reward tables, algorithms etc are the same, we could easily get a larger dataset quickly.
Here are my fabrication notes running on Holodeck. I don't have optimal doffs and haven't tracked my rarity chances.
Countermeasure system Mk XII green - worthless
2x Ablative armor Mk XII purple - 4.4 million
Montanium alloy XII green - worthless
Tetryon pulse XII green - 170k
emergency force fields blue - 40k
biofunction monitor mk XII purple - 1.75mil
EPS flow regulater XII green - worthless
warhead yield chamber XII blue
Tactical disruptor coil XII blue - 2.8 mil
I suppose we should take your word for it. Statistically speaking, 300 is a very small sample size, but I gave up on this doff mission after only 30 attempts because I had the same results. Piles of stealth modules and equally worthless garbage. Oh well.
i had same thing ive given up on it same as some in my kdf fleet everyone said there got sensor probes with in there 1st 3 trys. we think its stacked that way maybe its one of them cryptic bugs we all love :-D
Something I have noticed with a few doff related rewards is that the RNG seems off and tend to repeat the same rewards. This seems different per character. Lets use Instigate Defection. On three characters I tend to get the same one about every other time on each character. Sometimes it feels like everytime, So one charater might get Sizzta (energy weapons guy) over and over and another might get Alenia (?..the diplomat) over and over but then the other toon will see them rarely. Seems like each character has it own set of repeats.
So while there may not be some super secret weighting scheme, i do think that odds on items doesnt really feel random to me over all
I can definitely confirm that this seems to occur with my characters. One Gorn character of mine has gotten four of the same purple Rigellian (I think?) Systems Engineers, out of I believe five total attempts at Instigate Defection. A Federation character of mine has gotten at least three of the same purple Nausicaan security/tactical duty officers with relatively few attempts. Without knowing the total number of possible outcomes, this isn't necessarily evidence of something being amiss... but it's still strange and fairly vexing.
I find very similar results with the experimental console upgrade assignment. The characters who undertake these assignments seem to get the same few consoles (different for each character) repeatedly.
I will begin tracking the results of the following assignments:
Instigate Defection
Experimental Console Upgrades
Fabricate Prototype Console from Alien Artifact
I believe the XII console crafting method should be completly reworked, My idea would be: I go to my Fabrication lab on my Starbase or Ship and all 42 consoles are listed on a Menu, I choose what console i want, and like the reputation system is tells me what i need to craft it. This should include certian Doffs with specific traits to get a higher chance at crit success, a set amount of data samples and EC or Dil, a console schematic, and say if you are shooting for a XII purple Antiproton Mag regulator you would need 2 or 3 lower level consoles of the same type. So if I include 3 white antiproton consoles I get a %40 chance at purple, 3 Green %50, 3 Blue %70 chance at purple, and the Doff crit success determines if it is a X, XI or XII. You could do this with your bound consoles so when you level up you dont have to sell or scrap your old stuff you could recycle it toward new higher level or quailty items. You could also do this with Shields, Eng, Weps. It would put to use all the junk drops that we just sell to vendors for EC and it would put to use all the data samples we never use after we hit VA, The effect on the economy would be that lower tier and quality items would increase in price on exchange due to people needing them for crafting just like the common doffs and Romulan ale and contrband did. This system would get players farming data samples and drops to use to craft with and it would alow players as they are leveling up to use their lower tier equipment to get the next tier of equipment they need, Im sure all remember how bad it sucked when you hit commander and got a new ship and went to buy new gear for it and your officers and then 2 or 3 days later you hit capt or RA and it all just got sold to a vendor for scrap. Lets put these items that are already in game use and scrap this alien artifact ramdom method for something better.
Comments
*Retracted*
My statements or test do not suffer from confirmation bias so please read that article Borticus put up again. What I did is test the observed data to the expected pattern and even completed a sample size determination of the power of the data. There is no emotion present whatsoever in the statistical tests I put forth or the probabilities I have provided. The tests I conducted are routine for testing such distributional questions. Now, given the numerical, statistical, and probablistic evidence provided, consoles, regardless of quality, are not dropping uniformly and a sample size of 300 is sufficient to state that.
Whether people want to "believe" it or not in the face of the statistical and probabilistic fact is, well up to them. The only assumption we make in the whole analysis is that the OP's data are accurate, i.e. his sample is representative of the population. That assumption is made in ANY statistical analysis not just this one.
A lot of things that the players have had a gripe with.... STF drops, lock boxes, console fabbing... they all hinge on that RNG. Every-time I have complained about this stuff, I always mentioned the random number generator. I have never thought it was random enough.
You can easily get a 'sense' of it with the lock boxes. I stopped opening them quickly because of my suspicion of the RNG. A lot of RNG's use the current time as a seed and to be sure, I open a lock box every few minutes or open one in between running STFs and the like. I find that I get a better chance of getting a larger variety of stuff vs getting 10 Gamma Quadrant mini-doff packs it less than 3 seconds.
For example, I assume that all the mini-packs have the same drop chance from a lock box. If not that would counter my perception... but anyway, whenever I open like 5 or 6 boxes all at once, I'll get the same mini-doff pack over and over where if I wait a few seconds or a minute I'll get a mixture of doff packs, weapon packs and whatever. I know it could be just chance, but I think it's a crappy RNG.
Just try it on a test shard, give yourself 100 keys and 100 boxes and open them as fast as you can click and then do it again while waiting 10, 15, or 30 seconds or so. You'll notice you get large groups of the same item over and over while taking the fast approach. Taking the slow approach might net you some mid level items and a better mix of low level items.
Now I know you guys use C++ and you probably don't have your RNG in a loop like this example, but see here:
http://stackoverflow.com/questions/7686186/android-random-number-generator-not-random-enough
Even something as silly as where the RNG is initialized made a difference for the unrelated case above. What if STO suffers from such a small, almost unnoticeable oversight like the case above? I am glad you have reached out to your peers on the topic of the RNG. Perhaps you convince them to schedule a 15 min code review of the RNG?
I get these consoles quite often. As I stated before, only frustrated people will tend to post in such threads. The console i get the most? The shield one! Blue, purple, it likes me.
God, lvl 60 CW. 17k.
What are you talking about? This thread is about crafting Powered Alien artifacts from the Protoype Console DOFF mission you get from your Eng officer. If a Dev wants to give my Tribble account the ability to run this mission with no cooldown and iinfinite Powered Alien Objects and 5 purple Eng DOFFS with the required Crit traits for %24-%51-%24. I will spen a week and do a few thousand runs and document the results for everyone to see. Or I know they are busy but can they Just created a test account with a macro that runs this mission over and over with a bank large enough to hold a few thousand items and the review the results. Since my OP several other players have stated that they are seeing the same junk consoles like Bio function/Pre fire chambers/stealth modules/ over and over again. The consoles that drop the most are the generic beam/cannon/ torp for Tac, For SCI it is Bio/Stealth/Power insulator and for Eng,Plasma Dist manafold/Field Emitter/Injector Assembly
I also noticed that if I start another Prototype Console crafting as soon as one is finished (doing them back to back) I will get the same console as list time or another junk console) but when i allow at least a 2-4 hour cooldown I have better results. And the guy who thinks that nothing is wrong beacuse he has gotten a few good consoles obviously has not done this mission a much as myself and some of the other posters that have done this mission hundreds of times. There is a member of my fleet that did it only twice with blue and green doffs and got purple Antiproton Consoles both times. So yes good consoles do drop but after doing the mission several hundred times I see, and many other see a distinct pattern that is showing that it is not random and that the lesser value consoles drop at a much hight rate than the high value consoles do.
Well the easy solution to this if you "feel" (and I mean feel because the stats are there) that the OP's results are somehow overly biased on only represent that very edge case which does not represent the population is to post your suite of data which can then be pooled with the OP's and the analysis can be re-done.
This guy is just trolling.
There were 1 or 2 other people that posted here that have crafted as many if not more than I have. I ask them to please start recording your results and post them here. If a test account of some kind on tribble cannot be created to test this then I would hope the Devs would run a test account on this mission and do 4 or 5 hundred runs of it to see the honest results of it for them selves.
I have also noticed this. Not just opening boxes. I have found doff missions that finish within a few seconds of each other often have the same crit success/falure resault.
What's the most credible possibility? Borticus lying to us for evil purposes or some players being incredibly unlucky? At some point, even something almost impossible happens. It would be abnormal if we didn't have any exceptions, "weird" and unexpected outcomes and so on.
The RNG might be broken. I'm not trolling when i say i get the shield console quite often. I'm a bit curious to see the results of the investigations about it. It would affect more than this assignment if it was flawed though, unless the game has several RNGs for different parts of the game, which would be weird and quite inefficient by design (spreading RNG rolls across players and systems would likely increase the 'randomness'). It would affect damage dealt by ships, other assignments, and so on. If it was true, i'm quite sure that the pvp community would be screaming about it, with threads like "way too much/not enough crits", because those guys are pretty good at monitoring such flaws.
I'm definitely not saying it doesn't require further studies, but only that you should expect - and accept - a "working as intended" answer. There are exceptions all the time. People win at lotteries everyday. You will find dozens of people having 20 or more coins of the same year in their wallets across the world. Not to be mean but you may also very well be the victim of the common belief that if the odds are extremely low it shouldn't happen. But it does. Everywhere. All the time.
Let's quote Sir Conan Doyle for posterity.
"When you have eliminated the impossible, whatever remains, however improbable, must be the truth".
God, lvl 60 CW. 17k.
I'll start recording mine this weekend. I just rounded up 24 Strange Artifacts from NADORC assignments earlier this week and started crafting on 2 different toons this morning.
I would have to agree. This thread has attracted some trolls that aren't really contributing with anything but their own opinion. More people need to be collecting hard data and posting it here. Here's a fact. The Devs have been known to make errors when trying to track down the reasons why certain things behave the way they do. Anyone that's been around this game for awhile knows how true this is. That's why threads like these get started and it's a good thing too.
I wonder if anyone has documented the pre-S7 drop rate of STF gear. How often did you get an AP console? Pretty freaking rare right? Phaser consoles? I can't even remember. So we always felt those were weighted, what if they are using the same mechanic?
Completed Starbase, Embassy, Mine, Spire and No Win Scenario
Nothing to do anymore.
http://dtfleet.com/
Visit our Youtube channel
My character Tsin'xing
"Incredibly unlucky" really does not come close to explaining the OP's results in the light of a uniform drop rate. Right now, the most probable event is something odd in the code chain (not necessarily the RNG). So let us go through the logical decision process so everyone can see where we currently stand. If you want to follow the hyethetico-deductive model, we can do that. We have before us five hypotheses that we have to decide on and unfortunately, they are not mutually exclusive.
1 - The OP has misrepresented his data - This is likely not the case because the OP has expressed a genuine concern and took a great deal of time and effort to collect the data.
2 - Borticus has misunderstood the table or the reward process - Again, we can say with certainty this is not the case. Borticus is familiar with DOFF system and their loot table structure.
3 - The observed results are spurious resulting purely from random chance (i.e. the sample does not reflect the population) - This cannot be denied, however the probability of such an event from the analyses provided is extremely improbable. Remember we have disproven the null hypothesis that the observed distribution and the expected distribution are the same. They are different and the results of those tests provide us the probability of that difference occurring by random chance alone.
4 - The observed results represent a sampling inadequacy (i.e. the sample size is not large enough to detect a difference) - Again, I have gone through some quite basic statistical power analyses that disprove this option. At this point whether people wish to believe it or not, the sample size issue is no longer a valid argument. We have a sufficient sample size to make a determination on the observed differences. Those still arguing a sample size issue are doing so from belief, feelings, or emotions and thus have no logical grounds anymore.
5 - Somewhere in the game code lies an issue that is skewing the observed results from the expected results (the drop rate in the loot table) - Somewhere in the chain of code from the jobs results to the draw from the loot table, the results are getting skewed. This could be anything and it does not matter if we specify it or not. We know it lies in that path because the initial roll on quality is working and we know that the values for the consoles in the loot table are equal (uniform). Something is breaking down in the process from point A----> point B.
We CAN rule out hypotheses 1 and 2 based on the character and knowledge of the two parties in question.
We CANNOT rule out hypothesis 3, but we can say it is highly improbable.
We CAN rule out hypothesis 4 because we have already determined ~300 consoles is enough to make a statement on the observed difference.
We CANNOT rule out hypothesis 5 because none of the parties know the code process.
We have two options here to decide definitively between the two:
1 - Obtain more data of similar sample sizes (replicates) to see if the observed pattern the OP has holds true or is falsified.
2 - Examine the code path for potential factors skewing the results.
If we are to make a determination, now between the last two competing hypotheses without any additional information, we have to invoke Occam's Razor. That invocation would determine hypothesis 5 as the most likely, because it is the simplest and most parsimonious. Given the evidence, your fictional sleuth would also logically deduce hypothesis 5.
Now, again just because you or anyone else is getting consoles you feel are valuable, your statement does not improve the position that the results fit hypothesis 3 or 4. Such arguments are from an emotional standpoint using a qualitative assessment as fact. Such arguments are definitely suffering from a conformational bias, as the focus is on the consoles that are deemed valuable rather than the sample of consoles as a whole. Second, such statements are purely qualitative. Those qualitative metrics are erroneously being used to afford judgment on whether consoles are being obtained in equal proportion and against the quantitative measures (which is the crux of this entire argument). Third, the qualitative statements of, "I regularly...", "I got three of...", etc... also suffer from a bias because they are only providing a partial result and cannot be used to confirm or refute the quantitative data. Our process has moved beyond the qualitative now and we must focus on the quantitative.
The best course of action we can do now is provide more data to confirm or refute hypothesis 3 and pursue an investigation to confirm or refute hypothesis 5. Both are underway.
I recall that thread. I think the way that the game engine "if one crits they all crit" behavior exaggerated player conclusions. And then getting lucky with two or more damage groupings in a row getting crits. That would making a highly suspect lucky streak under the assumption of 1 crit roll per tick of damage.
PS And I'll miss the if one crits they all crit behavior. Seeing a Breen Torpedo Cluster crit with "Attack Pattern Alpha", "Go Down Fighting" and "Fire on my Mark" running is GLORIOUS! :eek:
Just so we're clear, the core cause of this bug (as I understand it - but I'm not a programmer) is that a single RNG roll is being propogated through multiple combat events, not several separate rolls that produce the same figure.
In other words, blaming a "sticky" RNG on this behavior is a false assumption.
Cryptic - Lead Systems Designer
"Play smart!"
Indeed these are the last two options.
If we make such data collections, i'd also like to include the conventional console craft (experimental console upgrade) to see if it's an isolated issue or not. I remember seeing odd results with that one but completely forgot it since i've stopped doing it for this specific reason. By weird i mean a lot of countermeasure systems and an eng console i forgot.
God, lvl 60 CW. 17k.
What I mean is:
From a lore/story standpoint, what's essentially happening is that some alien trading partner stumbles upon some gadget, doesn't know what it is, and sells it to you as a curiosity piece. Your science/engineering teams fiddle with it, make it work, and lo and behold it's some piece of alien technology that provides dramatic to (insert name here) system. Right?
So, what, are we to believe that all these highly advanced mysterious alien ships were loaded up with stealth modules, power insulators, and the like?
No, it's far more realistic that most of them had, you know, useful consoles on their ship. Therefor, there should be a weighted chance of acquiring a useful console from this assignment chain. The most commonly used consoles should be the most commonly generated consoles.
But that's a good point about relevence. The RNG isn't being used the same way. Sticky or not, the rolls are days apart in the data.
My character Tsin'xing
Actually, this is incorrect. There are 42 different consoles that can be crafted using this mission. Assuming they are all equally likely, there is a 1 in 42 chance to get a specific one. So, there is a 1 in 42 chance of getting a Phaser Relay. This means there is a 41 in 42 chance of NOT getting a Phaser Relay.
The math becomes (41/42)^300 = 0.000725. Thus there is a 0.07% chance of not getting a Phaser Relay. This works out to a 1 in 1379 chance.
I stand corrected and will graciously retract my statements pertaining to the probability of obtaining a phaser relay as I have calculated the chances of getting 300 phaser relays. It still does not change the probability that the OP observed distribution differs from the expected distribution. Also, it does not change that the observed distribution happening from "bad luck" is small.
Oh and P.S., the odds then of getting at least one Phaser Relay in 300 tries then becomes 1 - (41/42)^300 =0.999275 or 1 in 1.000725...yet that highly probable result did not happen.
Yes me too and I even considered mentioning that, but the post was getting to long.
Should it be weighted towards tactical consoles? But what if I *want* a biofunction monitor? Who decides what has value- what doesn't- for the weighting? Borticus? He doesn't want that job.
*Edit: I'll add that there may very well be a problem with the RNG in this game. I have noticed that if I open two reward packs together very quickly, they have a seemingly improbable chance of containing the same thing. But, again, that could just as easily be a sample size problem.
Quite simply, I am not convinced that anything shady or interesting is occurring here. It's clear why the expected distribution of rarities matches the results: There are only 3 possible outcomes, with a roughtly 1:2:1 ratio of occurrence. 300 is a fairly sizeable sample size for testing this. However, there are 42 possible consoles...300 is NOT a big sample size to test something like that.
As for theory of sticky RNGs, I call bollocks on this. There are several reasons why RNGs become sticky: Failure to seed the RNG will result in an identical sequence being generated every single time. RNGs are therefore typically seeded with a value, generally some value derived from server time. The other possible way for an RNG to become sticky is if it this identical value is used to seed multiple instances of the same RNG. This would occur only if the time value had a low resolution (say, seconds), and the RNG was reseeded in several different threads that launched simultaneously. This, of course, cannot possibly describe the behavior of "waiting several hours" or even "several minutes" when it comes to opening boxes or queue doff missions. Hours and minutes are pretty much an eternity on a computer's timescale, and even if the resolution was as bad as "seconds", it would matter because of how doff missions resolve.
Now, how do doff missions resolve? An interesting thing occurs here: A doff mission does not resolve itself and determine its outcome until you look at it! How do I know this? Well, awhile back, I queued some doff missions. As is typical after one queues a doff mission, I buggered the hell off and switched to another toon. Hours later, the doff mission had finished, but as I didn't really consider it important, I hadn't bothered to look at it. I needed my toon for something on Tribble. I copied him over, did my test, and noted that my doff missions had ended badly. Curious about how this worked, I deleted the toon off Tribble, and copied myself again: Opening up my doff missions, I had gotten entirely different outcomes...outcomes which were again completely different when I finally opened the real version on Holodeck. In short, doff missions do not even resolve themselves until you look at them. There is no mechanism by which even a sticky RNG could possibly affect doff missions, unless every time you opened a sequence of the same doff mission, you always got the same result. This doesn't happen, so this sticky-RNG theory is bollocks.
Now, the two-reward-packs thing, THAT could very probably be a sticky RNG, IF every query submitted to open a pack was handled by a different thread spawned with the same seed, such as the second...but that would be very noticeable, very fast. Realistically, getting garbage out of a box is such a likely outcome that it should be no surprise at all when it repeats itself multiple times in a row. At least 99% of everything in a box is garbage, and there aren't all that many items to choose from.
Ive had 5 Purple Tellarites since the artifacts started so I here is what I have noticed over the past year.
The Quality ratio seems to be fine. Ive always gotten blues and purples in about the right proportion cant really complain about that.
What i have noticed is that I do seem to get the same few over and over, in my case I get some that others have mentioned they dont see. I get a lot of phaser relays and a lot of bio function monitors.
Something I have noticed with a few doff related rewards is that the RNG seems off and tend to repeat the same rewards. This seems different per character. Lets use Instigate Defection. On three characters I tend to get the same one about every other time on each character. Sometimes it feels like everytime, So one charater might get Sizzta (energy weapons guy) over and over and another might get Alenia (?..the diplomat) over and over but then the other toon will see them rarely. Seems like each character has it own set of repeats.
So while there may not be some super secret weighting scheme, i do think that odds on items doesnt really feel random to me over all
Ok, so I'm going to help you out tracking this. I started crafting on 2 separate toons yesterday. I gather 24 Artifacts a week and usually only run the Fabricate Prototype Console assignment over the weekends. I've been running this routine ever since the Children's Toys chain was introduced. I still have nearly all the consoles which I made last weekend in my inventory. It looks as though I have sold a few, however.
Out of the last 21 consoles crafted I have made 3 Emitter Arrays (all blue) and 3 Emergency Force Field Consoles (2x blue, 1x purple).
I've literally made a thousand of these things and I consistently see some of the same ones being made over and over. I have such a tendency for Emergency Force Fields, Biofunction Monitors, Warhead Yield Chambers, Emitter Arrays, Field Emitters, Graviton Generators, Countermeasures, Inertial Dampeners and Sensor Probes you wouldn't believe it! I wish I had been tracking these earlier so I can add more data but I'm very glad you've made a thread where the data can be stockpiled because I know that you're on to something.
I'll make a post after every 50 consoles that I craft to update totals so there should be new totals to add to your list every 2 weeks.
Well, we don't know how the STO RNG is implemented so anything we're doing is obviously hand-waving. Perhaps for performance reasons there is only one RNG roll per "tick" and that is used game-wide?
If DOFF missions truly do resolve when you look at it, then this becomes an easy issue to test. Everyone wait until they complete their next alien artifact mission, then tranfer their toon to Tribble. Assuming the reward tables, algorithms etc are the same, we could easily get a larger dataset quickly.
Here are my fabrication notes running on Holodeck. I don't have optimal doffs and haven't tracked my rarity chances.
Countermeasure system Mk XII green - worthless
2x Ablative armor Mk XII purple - 4.4 million
Montanium alloy XII green - worthless
Tetryon pulse XII green - 170k
emergency force fields blue - 40k
biofunction monitor mk XII purple - 1.75mil
EPS flow regulater XII green - worthless
warhead yield chamber XII blue
Tactical disruptor coil XII blue - 2.8 mil
i had same thing ive given up on it same as some in my kdf fleet everyone said there got sensor probes with in there 1st 3 trys. we think its stacked that way maybe its one of them cryptic bugs we all love :-D
I can definitely confirm that this seems to occur with my characters. One Gorn character of mine has gotten four of the same purple Rigellian (I think?) Systems Engineers, out of I believe five total attempts at Instigate Defection. A Federation character of mine has gotten at least three of the same purple Nausicaan security/tactical duty officers with relatively few attempts. Without knowing the total number of possible outcomes, this isn't necessarily evidence of something being amiss... but it's still strange and fairly vexing.
I find very similar results with the experimental console upgrade assignment. The characters who undertake these assignments seem to get the same few consoles (different for each character) repeatedly.
I will begin tracking the results of the following assignments:
Instigate Defection
Experimental Console Upgrades
Fabricate Prototype Console from Alien Artifact
I'll post results in this thread.