test content
  • Games
  • Social
  • Arc News
  • Support
  • Zen
What is the Arc Client?
Install Arc
Forums > Home› Neverwinter Discussion › The Foundry
Options

Please explain what is adjusted ratings???

gillrmngillrmn Member Posts: 7,800 Arc User
edited May 2013 in The Foundry
I have a very simple question for devs.

How can this campaign as shown in pic have an adjusted rating of 2.87? What decides to "adjust" the ratings?

Thanks a lot!
Post edited by gillrmn on

Comments

  • Options
    tilt42tilt42 Member Posts: 0 Arc User
    edited April 2013
    The adjusted rating is kept deliberately low until a certain amount of reviews (50 or so, I think). You'll see that this rating increases as the quest is played. They may be doing this to keep quests with just a handful of good reviews from dominating the top of the list, but this has its own downsides. This is actually holding back my From the Shadows campaign right now. Most Founders seem to have already played it in the closed beta, so almost nobody plays it now that the game is out for them. Instead, the missions that are played a lot rise in adjusted rating, pushing mine down the list despite my quests having an actual review average of 4 and 4.5 stars respectively.
  • Options
    gillrmngillrmn Member Posts: 7,800 Arc User
    edited April 2013
    tilt42 wrote: »
    The adjusted rating is kept deliberately low until a certain amount of reviews (50 or so, I think). You'll see that this rating increases as the quest is played. They may be doing this to keep quests with just a handful of good reviews from dominating the top of the list, but this has its own downsides. This is actually holding back my From the Shadows campaign right now. Most Founders seem to have already played it in the closed beta, so almost nobody plays it now that the game is out for them. Instead, the missions that are played a lot rise in adjusted rating, pushing mine down the list despite my quests having an actual review average of 4 and 4.5 stars respectively.

    First of all, if you check it is under 3. No rating goes under 3 even when adjusted for most quests unless it gets a lot of under 3 reviews.

    Secondly, the total number of plays can be counted to be 50+easily [around 70ish] from the screenshot itself.

    Hence your hypothesis does not work on this.

    Hence I want a definitive comment from dev who can actually explain what it is in reality.
  • Options
    tilt42tilt42 Member Posts: 0 Arc User
    edited April 2013
    gillrmn wrote: »
    First of all, if you check it is under 3. No rating goes under 3 even when adjusted for most quests unless it gets a lot of under 3 reviews.

    As I said, it's kept deliberately low. It's not an actual average, so it doesn't make sense from our mathematical formulas.
    Hence your hypothesis does not work on this.

    It's not a hypothesis.
    Hence I want a definitive comment from dev who can actually explain what it is in reality.

    The devs already explained this back in the beta forums, and their explanation is what I based the above on. I don't know all the details, but I know the gist of it.
  • Options
    gillrmngillrmn Member Posts: 7,800 Arc User
    edited April 2013
    tilt42 wrote: »
    As I said, it's kept deliberately low. It's not an actual average, so it doesn't make sense from our mathematical formulas.



    It's not a hypothesis.



    The devs already explained this back in the beta forums, and their explanation is what I based the above on. I don't know all the details, but I know the gist of it.

    You are lying here. The devs *NEVER* explained adjusted ratings.

    Secondly, you are completely making a very vague guess from little information you have. Its worse than hypothesis - its is inaccurate guesswork.

    Thirdly, there exists no rating system which is not based on mathematical formulas. If it does, it is arbitrary, unfair and evil.
  • Options
    tilt42tilt42 Member Posts: 0 Arc User
    edited April 2013
    Wow, uh... Mind if you sit down and draw your breath a few times while counting to 10 before you post? Accusing me of lying, followed by calling the Foundry's rating system "evil" is not only a personal attack on me, but it's also completely out of scale with whatever slight you feel you've been offended by. If Neverwinter truly makes you as angry as you come across in some of your posts, maybe you should consider not playing the game or making content for it? No game is worth this amount of vitriol and hate. I get the impression you can barely sit down with the game without feeling as if the world is collapsing around you, and that's telling.
  • Options
    gillrmngillrmn Member Posts: 7,800 Arc User
    edited April 2013
    I am most offended by misinformation.
    The devs already explained this back in the beta forums, and their explanation is what I based the above on. I don't know all the details, but I know the gist of it.
    You are lying here. I have always read and commented on all of the threads in foundry forums, ever since that section was created back in August or September.
    Adjusted rating have never been explained. Never ever been explained.

    EDIT:
    Also spreading misinformation is much different than sharing opinion. There is a lot of difference in them. While latter is a good thing and always welcomed, misinformation is not.
  • Options
    tilt42tilt42 Member Posts: 0 Arc User
    edited April 2013
    Well, if this is the attitude you're going to have against me when I'm trying to be helpful, then I guess I'm done trying to communicate with you on this forum. I have better things and better people to spend my free time on.
  • Options
    izatarizatar Member, NW M9 Playtest Posts: 1,161 Arc User
    edited April 2013
    I couldn't figure out the actual formula, but it looks like it starts at 50% rating and then goes up an asymptotic curve as the number of ratings go up.
    My numbers are:
    780 runs= 97% actual
    99 runs= 89% actual
    60 runs= 87% actual
    7 runs= 56% actual

    Gillrmr's 75 runs gives 84% which is reasonably, though not exactly, within these parameters. Variance of distribution of ratings and algorithm tweaks in the last couple of days can explain the small discrepancy I think.
  • Options
    gillrmngillrmn Member Posts: 7,800 Arc User
    edited April 2013
    tilt42 wrote: »
    Well, if this is the attitude you're going to have against me when I'm trying to be helpful, then I guess I'm done trying to communicate with you on this forum. I have better things and better people to spend my free time on.

    F u! An what is your attitude after I have answered so many of your questions and told you so many of workarounds?

    And how is it even helpful? At least izatar gave an idea how it works, but you are only lying. How many of total threads in foundry beta or alpha have you read? What is your post count in those threads? How many thanks you have recieved for questions answered? Compare it with mine.

    Dont you teach me about being helpful and not recieving any thanks. I have experienced it a lot already and it does not bothers me. WHat you are trying is getting your misinformation to pass off as a fact which is unacceptable.
  • Options
    tilt42tilt42 Member Posts: 0 Arc User
    edited April 2013
    gillrmn wrote: »
    F u! An what is your attitude after I have answered so many of your questions and told you so many of workarounds?

    Since when does answering questions give anyone the right to act like you're doing?
    And how is it even helpful? At least izatar gave an idea how it works, but you are only lying.

    Again with the personal attacks and accusations of lying. I'm sorry you didn't pay attention back at the beta boards, but this was explained. Not in detail, but in general terms. It takes a special kind of self absorbed to claim that because you didn't catch that explanation means I must be lying. Are you even aware of how that makes you sound?
    How many of total threads in foundry beta or alpha have you read? What is your post count in those threads? How many thanks you have recieved for questions answered? Compare it with mine.

    Is this a post count contest now? I'm not even sure what you're trying to accomplish with this line of thought. Superiority? Moral high ground? Step off your high horse.
  • Options
    bachus1234bachus1234 Member, Neverwinter Beta Users Posts: 79
    edited April 2013
    tilt42 wrote: »
    Since when does answering questions give anyone the right to act like you're doing?



    Again with the personal attacks and accusations of lying. I'm sorry you didn't pay attention back at the beta boards, but this was explained. Not in detail, but in general terms. It takes a special kind of self absorbed to claim that because you didn't catch that explanation means I must be lying. Are you even aware of how that makes you sound?



    Is this a post count contest now? I'm not even sure what you're trying to accomplish with this line of thought. Superiority? Moral high ground? Step off your high horse.

    Seems we need a developer interupt to clear who is right.... Fighting on boards like this gets us nowhere...
  • Options
    zovyazovya Member, Neverwinter Beta Users, Neverwinter Hero Users, Neverwinter Guardian Users, Neverwinter Knight of the Feywild Users Posts: 0 Arc User
    edited April 2013
    That awkward feeling you get when your parents fight in public.
  • Options
    tilt42tilt42 Member Posts: 0 Arc User
    edited April 2013
    zovya wrote: »
    That awkward feeling you get when your parents fight in public.

    Sorry, I don't want to make it a fight, but at this point I have to defend myself. This thread and gillrmn responses are sadly just one of many examples of this kind of behavior. He's attacking me (claiming that I'm a liar), calling Cryptic evil (I mean, seriously?), showing passive-aggressive elitism ("if you'd played my quest, you'd know that..." and "if you'd read my manual, you'd know that..."), expecting special treatment for having been a very active Foundry author, and reacting with outright hostility whenever something isn't exactly the way he wants it to, including other people's opinions.

    One of my first experiences with the Foundry beta was actually to get into an argument with him. After one or two of those, I quickly realized what kind of personality he was, and learned how to defuse these arguments before they truly began. Obviously that didn't work this time. I truly don't want to fight, but I won't have my honor questioned by him in public, nor will I sit on the sidelines and quietly look at as he insults Cryptic.

    During Beta Weekend 3 there was a lot of confusion among the Foundry authors about what this "adjusted rating" was since it was almost always way lower than the actual average rating. After a lot of discussion and speculation, one of the devs chimed in and gave us a few basic details about how it worked. That is the information I repeated, from memory, at the beginning of this thread. I don't recall the exact numbers, but I believe the dev said that the adjusted rating would approach the real average after about 50-100 reviews.

    It turns out gillrmn doesn't remember, or never saw, this conversation. For some reason it doesn't strike him as natural to assume that he might have missed it. No, that seems to be impossible. Instead I must clearly be a liar.

    I won't accept this. Not from anyone, and certainly not from gillrmn. This is the last I'll say on the matter as I don't want this to devolve into a bigger flame war than it has already become. I have said what I needed to say, and if gillrmn wants to take his ball and go home, that's his right. Shouting accusations and insults at other members and devs is not.
  • Options
    gillrmngillrmn Member Posts: 7,800 Arc User
    edited April 2013
    Before I log off, I want to clear off a few things here.

    First of all, we all know that ratings are always near three irrespective of you get good or bad ratings for adjusted ratings. The ratings I get are below 3 while average ratings are above 3. How can it be explained? Where is another quest which gets ratings like that?

    A person with just 1 review of 5 star will not have a 1 rating. So the premise of "dev having told" is wrong.

    Secondly you can already see that I have 50+ plays (nearing 100). This concludes the second argument which is also unsound.

    Thirdly, a dev has never explained adjusted ratings. They have explained average ratings, but not adjusted ratings. The possibility of me skipping that particular info does exist, but it is rare when I am reading each and every post in each and every thread.

    Lastly, regarding character assassination - I did have a clash with him before but you can just check the alpha threads(if they can be made available) to see who refrained from argument war first. The threads must still be in database somewhere and it can be seen who refrained first.
  • Options
    kamaliiciouskamaliicious Member, NW M9 Playtest Posts: 0 Arc User
    edited April 2013
    gillrmn wrote: »
    A person with just 1 review of 5 star will not have a 1 rating. So the premise of "dev having told" is wrong.
    That's what my quest has, and has.

    Actually, I just checked and I have 2 ratings, one is a 4 and one is a 5. My adjusted rating is 1.39.
  • Options
    lanessar13lanessar13 Member, Neverwinter Beta Users, Neverwinter Guardian Users, Silverstars Posts: 8 Arc User
    edited May 2013
    gillrmn wrote: »
    First of all, we all know that ratings are always near three irrespective of you get good or bad ratings for adjusted ratings. The ratings I get are below 3 while average ratings are above 3. How can it be explained? Where is another quest which gets ratings like that?

    Before you flip out on Tilt, I'd reconsider. Just because you don't remember something and Tilt vaguely remembers something isn't a reason to start hurling curses.
    A person with just 1 review of 5 star will not have a 1 rating. So the premise of "dev having told" is wrong.

    Incorrect. Read below.
    Secondly you can already see that I have 50+ plays (nearing 100). This concludes the second argument which is also unsound.

    Thirdly, a dev has never explained adjusted ratings. They have explained average ratings, but not adjusted ratings. The possibility of me skipping that particular info does exist, but it is rare when I am reading each and every post in each and every thread.

    Read below. The number "100" might increase with the number of overall plays - it's not clear from the data below. But in the grand scope of things, "100 plays" might not be sufficient to normalize.
    Lastly, regarding character assassination - I did have a clash with him before but you can just check the alpha threads(if they can be made available) to see who refrained from argument war first. The threads must still be in database somewhere and it can be seen who refrained first.

    wish I had seen this sooner. I could at least give you what little I saved from the beta forums. Here's the only information about ratings that I found, so technically they did explain the rating system in the beta forums, but it's very limited:
    badbotlimit wrote:

    We are looking at the rating system very closely. The stars your quest gets are not an actual average of the reviews. The star rating is a little misleading. The rating of your quest is a "best guess" at the quality of the quest. It take a lot of plays and reviews before your quest can get 3+ stars in the catalog. You can see this by looking at your quests overall star rating and the individual reviews ratings. It is possible to see all your reviews with 4 stars but have a quest with the overall 2 or 3 stars. Although this is the same system that many sites and services use to rate their posts or products it does come off looking like a bug to some. We have considered returning to the flat average.

    As for the reviews for folks that may not have finished a quest... This is a slippery slope. There are cases where we can see that letting reviews before even playing can cause issues but there are also legitimate reasons for someone not playing as well. Internally we have talked about requiring at least one step of a quest be completed but what about cases where that one step cannot be completed. When we start making exceptions on a case by case basis we start chasing an issue that has no clear end and maybe no "right" answers.

    We do share a lot of the same ideas for how we can make it better. TBH, we are going to have to keep working at it.
  • Options
    boydzinjboydzinj Member, Neverwinter Beta Users, Neverwinter Guardian Users Posts: 0 Arc User
    edited May 2013
    gillrmn wrote: »
    F u! An what is your attitude a....

    Hmmm? He may be mistaken ... he might be right... I DO NOT KNOW. However, someone has an attitude and someone is getting angry of pixels? I believe the saying goes, "u mad bro?"
  • Options
    tilt42tilt42 Member Posts: 0 Arc User
    edited May 2013
    Eh, let it go, people! gillrmn hasn't been around since this thread, so we can assume that he's left. Even if he hasn't, it's poor sport to lynch someone who isn't fighting back.
  • Options
    lanessar13lanessar13 Member, Neverwinter Beta Users, Neverwinter Guardian Users, Silverstars Posts: 8 Arc User
    edited May 2013
    tilt42 wrote: »
    Eh, let it go, people! gillrmn hasn't been around since this thread, so we can assume that he's left. Even if he hasn't, it's poor sport to lynch someone who isn't fighting back.

    Hope my post didn't come across as lynching. Just wanted him to calm down a bit. And providing data. :-/

    It's sort of sad Gillrmn left. His open-ended quests were very well-made, and he was quite a guru at foundry tricks. Perhaps after the "beta" period has settled out, he can return and accomplish what he wants to.
  • Options
    jkotchjkotch Member, Neverwinter Beta Users, Neverwinter Guardian Users Posts: 20 Arc User
    edited May 2013
    Per the official Wiki for the foundry, the game is using "Lower Bound of Wilson Score Interval" Otherwise known as the "Binomial proportion confidence interval"
    http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval

    If you can understand that formula exactly, you are a better person than I, and I'm a programmer! However, generally what it means is that as you gain more votes, it will adjust it's average to be closer to 100% of your average score, and potentially (eventually) begin ignoring cases that are far enough outside of the average. (so that 1 star you have won't pull all those 5 stars down as bad as an actual average would.) At least that is what my understanding of the formula is. The downside of what I see with it is that it takes too long to adjust the average based on your popularity (which is a factor)

    This site explains why the use of this formula is better than most other ways of computing the average rating.

    http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
  • Options
    chinspinnerchinspinner Member Posts: 0 Arc User
    edited May 2013
    It is Friday night and I am quite relaxed. Oh no, I was then I read this needless aggro. If you want to be angry please do it somewhere else, haha sorry condescending in the extreme.
    My new quest:

    WIP
  • Options
    vaelicvaelic Member Posts: 176 Bounty Hunter
    edited May 2013
    wow.... just wow.....
    MY FOUNDRY QUEST

    Quest Title: Don't "Count" on it - Ch. 1
    Short Code: NW-DQ3H4MXKG
    Duration: 15-20 minutes
    DAILY FOUNDRY ELIGIBLE? Yes!
  • Options
    chinspinnerchinspinner Member Posts: 0 Arc User
    edited May 2013
    vaelic wrote: »
    wow.... just wow.....

    I know right, it is like sooo strange. That was my attempt at US.
    My new quest:

    WIP
Sign In or Register to comment.