test content
What is the Arc Client?
Install Arc
Options

thinking about matchmaking filter

ursusmorologusursusmorologus Member Posts: 5,328 Arc User
edited May 2014 in PvP Gameplay
I hear a lot of people asking for matchmaking, which I assume means auto-balancing the teams in a queued match. What criteria are you looking for? Seems like the only thing that makes sense is something like kill-death ratio for the toons, possibly weighted by the K/D ratio of the ship, with high-low distribution so that the sum K/D for each team is about even. Maybe you have a guy with 1.6 and another guy with 1.3 then everybody else is 1.0 with a couple of .3 noobs at the bottom, mix it all together so you end up with ~1.0 on each team. Just thinking about this, nothing more.
Post edited by Unknown User on

Comments

  • Options
    brandonflbrandonfl Member Posts: 892
    edited May 2014
    Hilbert's algorithm is a good place to start. It's pretty much a combat effectiveness rating, as I understand it. It tracks your overall damage, damage to other players, heals, heals to others, crowd control (as much as the log records anyway) and gives a style rating based on use of certain abilities that are currently conisdered "cheesy" by many (though I doubt Cryptic would ever use that as a consideration... it's pretty minor in the equation anyway).

    The way it stands, even if they fixed the teams queuing against pugs bug, there's still no mechanism other than your place in line to determine the makeup of the 2 teams. People that want to troll the queue can still easily exploit that fact. As I see it, without some form of matchmaking the chance of balanced games is not any better than it was before. I'm sure that Cryptic could come up with something that functions much better than Hilbert's balancing tool (and it's pretty good). They have access to so much more information than us players do.

    Anyway, yeah. Matchmaking.
    LOLSTO
  • Options
    mancommancom Member Posts: 784 Arc User
    edited May 2014
    Damage, SelfHealing, CrossHealing, Kills and KillAssists (SNB etc.) should be balanced between the teams. The current log file doesn't allow one to get meaningful data for Assists, so my current matchmaking algorithm (which is different from the leaderboard rating system) does not take that into account, but it is a good place to start. There is a write-up of the algorithm at If you want to see it in action, go to http://hilbertguide.com/leaderboard/balance.php and enter a couple of names.
    1042856
  • Options
    ursusmorologusursusmorologus Member Posts: 5,328 Arc User
    edited May 2014
  • Options
    wolverine595959wolverine595959 Member Posts: 726
    edited May 2014
    I am willing to bet the code is so all encompassing that pulling it a part for this on cryptics side would be tedious and error prone.
    Hey I Used to be Captain Data, well I guess I still am in game but the account link really screwed everything up :rolleyes:
  • Options
    gooddaytodie39gooddaytodie39 Member Posts: 0 Arc User
    edited May 2014
    Geko mentioned in the last STO'ked radio interview that a major overhaul to PvP will be taking place "in either season 10 or 11". He did not get too far into specifics but he did mention leader boards being a possibility. Nothing on matchmaking though...
    I'm kinda over ever hoping for any semblance of balance to these matches thought. Geko described MMORPG balance as "the dragon that everyone chases but you know, some imbalance is there by design and needs to be there. Imbalance adds a fun factor."
    The abundance of universal consoles and the imbalance they create in relation to pvp is out of control and there's no stopping them in sight. Universal consoles only being able to be used in device slots I think would help? Who knows.
  • Options
    antoniosalieriantoniosalieri Member Posts: 0 Arc User
    edited May 2014
    Inbalance adds a fun factor. Great student of game theory there.

    It adds a $ factor for them no doubt.

    These devs are pretty clueless so I have no hope for there game anymore either. It is funny to see devs say something like that though... its pretty much an admission that things are way over the line. We all know that yes developers of games that are out to make money factor in a small amount of inbalance.... to come out and say it though means they understand they have went a little to far.

    All well... plenty of other good games around that are closer to an ideal amount of unbalance. Worse case I'll just spend more time playing chess or something. lol
    [SIGPIC][/SIGPIC]
    Dignity and an empty sack is worth the sack.
  • Options
    dontdrunkimshootdontdrunkimshoot Member Posts: 0 Arc User
    edited May 2014
    imbalance is fine as long as it doesn't start excludeing things or directly trumping them. its also fine if skill is enough of a factor in the system. as far as money goes, i think different and fresh will sell as well or better then powercreeped and familiar. shame there's more of the latter in STO, and warbirds are both at once
  • Options
    scurry5scurry5 Member Posts: 1,554 Arc User
    edited May 2014
    Then again, rock-paper-scissors mechanics are sometimes what some may call imbalance.

    Better gear is a certain degree of imbalance too.

    Assuming that's what he's talking about, in context.
  • Options
    riccardo171riccardo171 Member Posts: 1,802 Arc User
    edited May 2014
    mancom wrote: »
    Damage, SelfHealing, CrossHealing, Kills and KillAssists (SNB etc.) should be balanced between the teams. The current log file doesn't allow one to get meaningful data for Assists, so my current matchmaking algorithm (which is different from the leaderboard rating system) does not take that into account, but it is a good place to start. There is a write-up of the algorithm at If you want to see it in action, go to http://hilbertguide.com/leaderboard/balance.php and enter a couple of names.

    Holy... :eek:
    Man that's some really impressive work, congrats!
    Tried it (even if it put three out of three Scimitars, a healer and a vaper together) and it might need a bit of refining, but I would look forward to have something like that implemented.

    still, it gave me quite balanced results in a few attempts.
  • Options
    ursusmorologusursusmorologus Member Posts: 5,328 Arc User
    edited May 2014
    Thinking about it some more, it seems like data needs to be tracked per-ship. Like a player might have a corvette and a chel grett, similar layout but totally different leaderboard stats. I know this data isnt available in combat log, but it would be available to cryptic.
  • Options
    mancommancom Member Posts: 784 Arc User
    edited May 2014
    Thinking about it some more, it seems like data needs to be tracked per-ship. Like a player might have a corvette and a chel grett, similar layout but totally different leaderboard stats. I know this data isnt available in combat log, but it would be available to cryptic.
    Indeed. This is one of the big advantages that Cryptic has if they were to implement a matchmaking system.

    Currently I attempt to account for changing builds by limiting the number of matches that are used to calculate the healing/damage/etc. averages for the balancing algorithm; this way only somewhat current matches are being used instead of the entire player history which might include many different builds.
    1042856
  • Options
    mustrumridcully0mustrumridcully0 Member Posts: 12,963 Arc User
    edited May 2014
    Advanced matchmaking / scoring systems like True Skill tend to differentiate between the strength of a player in the confidence in this strength assessment. They still mix this together with one total score, IIRC, for match-making purposes, but the calculation after the match can change. But in such a system, the confidence could change after a build change.

    [/Random Thought]
    Star Trek Online Advancement: You start with lowbie gear, you end with Lobi gear.
  • Options
    mancommancom Member Posts: 784 Arc User
    edited May 2014
    Advanced matchmaking / scoring systems like True Skill [...]
    I'm not certain how well these systems translate to games that rely a lot on team composition and teamwork (captain types, crosshealing, focus fire). I looked at Elo ratings (used e.g. for chess), but that didn't feel like it would work well in a team setting.

    If you look at the table on the TrueSkill page, it seems that estimating a player's strength gets a lot harder when one has two teams with 5 players than if one had a free-for-all.

    Unfortunately this is not my area of expertise, but if anyone else has experience with such rating systems, I'd be happy to provide the win/loss/healing/damage/etc data that I have collected if it could help create a better matchmaking algorithm.

    (Also note how all these systems seem to be based on Win/Loss whereas my system currently completely ignores this.)
    1042856
  • Options
    wolverine595959wolverine595959 Member Posts: 726
    edited May 2014
    Geko mentioned in the last STO'ked radio interview that a major overhaul to PvP will be taking place "in either season 6 or 7". He did not get too far into specifics but he did mention leader boards being a possibility. Nothing on matchmaking though...
    I'm kinda over ever hoping for any semblance of balance to these matches thought. Geko described MMORPG balance as "the dragon that everyone chases but you know, some imbalance is there by design and needs to be there. Imbalance adds a fun factor."
    The abundance of universal consoles and the imbalance they create in relation to pvp is out of control and there's no stopping them in sight. Universal consoles only being able to be used in device slots I think would help? Who knows.

    Please re-read what I fixed. IF you trot it out enough eventually it may happen, eventually. :D
    Hey I Used to be Captain Data, well I guess I still am in game but the account link really screwed everything up :rolleyes:
  • Options
    mustrumridcully0mustrumridcully0 Member Posts: 12,963 Arc User
    edited May 2014
    mancom wrote: »
    I'm not certain how well these systems translate to games that rely a lot on team composition and teamwork (captain types, crosshealing, focus fire). I looked at Elo ratings (used e.g. for chess), but that didn't feel like it would work well in a team setting.

    If you look at the table on the TrueSkill page, it seems that estimating a player's strength gets a lot harder when one has two teams with 5 players than if one had a free-for-all.

    Unfortunately this is not my area of expertise, but if anyone else has experience with such rating systems, I'd be happy to provide the win/loss/healing/damage/etc data that I have collected if it could help create a better matchmaking algorithm.

    I really haven't gone into depth into this, but my impression is, that these systems work for teams, but it takes longer for the values to stabilize. It may not work if you are always in the same team, then it has no real chance to differentiate your skill levels - but once you leave the team, your score will likely change (probably first the confidence - because it expected you to win or lose, but you didn't, and the first times this happens, it will lower the confidence, as it happens repeatedly, confidence will grow again.)

    I think 5 vs 5 man teams are probably not as bad as something like 12 vs 12 man teams in Mechwarrior Online.

    No system is perfect. But you don't need perfect. (Perfection will automatically fail the moment you don't get the right players in the queue - if there are 3 excellent players, and 5 mediocre, then it can't help but create an imbalanced match, for example, even if the scores have been calculated perfectly.)

    (Also note how all these systems seem to be based on Win/Loss whereas my system currently completely ignores this.)
    I think most systems assume that it's better to abstract away from specifics. Elo rankings were introduced for Chess - they could have tried to include statstics like "How often did he put the enemy into Chess" "how many points worth in figures where still on the board at the end, how often did he use a weak figure to capture a stronger, or what not" - but in the end, the real result that matters is the outcome, not how you got there.

    For teaching you to get better at the game, you need the other statistics. If you realize that your healer heals only 50 % of what the top rated player who usually uses healer builds has, then you know that's a weakness. On the other hand, if you see that you have less DPS then him or that you die more often, you know that's what you need to improve. But the rating systems aren't aimed at improving yourself, they are aimed at creating "fair" matches or predict the likely outcomes of matches.
    Star Trek Online Advancement: You start with lowbie gear, you end with Lobi gear.
  • Options
    brandonflbrandonfl Member Posts: 892
    edited May 2014
    A back and forth between Mustrum and Hilbert... feels like 2011 again :)
    LOLSTO
  • Options
    mancommancom Member Posts: 784 Arc User
    edited May 2014
    If one bases a system purely on win/loss it cannot work for a game where one has teams where players have certain roles. A pure win/loss system couldn't decide that when you have 10 players with equal win/loss ratios it would be a bad idea to put the 5 engineers on one team and the 5 tacs on the other one.

    I can see win/loss working fairly well for 1v1 games and for all games where each player is "interchangeable" (e.g. Unreal Tournament).

    I wonder: FPS have roles (medic/sniper/...) and certainly there is a balancing algorithm for FPS multiplayer matches. But maybe they skip the role part in the balancing process because people can swap roles after every respawn (maybe within certain role limits for the team)? This way players would be interchangeable again (which is not the case for STO). I haven't really played FPS in well over a decade, so I don't know how it works these days or if players are locked into a role for a long time.
    1042856
  • Options
    ursusmorologusursusmorologus Member Posts: 5,328 Arc User
    edited May 2014
    mancom wrote: »
    I wonder: FPS have roles (medic/sniper/...) and certainly there is a balancing algorithm for FPS multiplayer matches. But maybe they skip the role part in the balancing process because people can swap roles after every respawn (maybe within certain role limits for the team)? This way players would be interchangeable again (which is not the case for STO). I haven't really played FPS in well over a decade, so I don't know how it works these days or if players are locked into a role for a long time.

    They probably do it with more modern ones, as they have been aggressively tracking stats since the black ops 1 era, but prior to that there was no auto-balancing that I am aware of. Players in my old COD4 clan used to switch sides manually in the middle of the game to balance it out if it looked too lop-sided. Usually it was 10v10 matches so there was less steamrolling, but it still happened often enough that we had a casual protocol to deal with it.

    The classes were very neutral back then too. You could spawn with a hmg and pickup a lmg on the ground, no difference. Classes didnt have different armor or anything, just different kits or camo if anything.
  • Options
    mustrumridcully0mustrumridcully0 Member Posts: 12,963 Arc User
    edited May 2014
    mancom wrote: »
    If one bases a system purely on win/loss it cannot work for a game where one has teams where players have certain roles. A pure win/loss system couldn't decide that when you have 10 players with equal win/loss ratios it would be a bad idea to put the 5 engineers on one team and the 5 tacs on the other one.
    You are the lord of the data. And on top of that, not as lazy as I am ;).
    So if your own match-maker yields better results in predicting the outcomes of battles and creating "even" battles then an Elo based calculation, then you probably have sufficient evidence for your point.

    But my theory is that roles are not that important. If anything, then the only thing important is that all roles are covered in a team, but beyond that, win/loss based Elo ratings will suffice.


    A back and forth between Mustrum and Hilbert... feels like 2011 again
    Did we have those? I don't remember anymore. I only remember trying to gather as much of the cryptic information about the game mechanics as I could, thankfully having other people doing the dirty work for me. :)
    Star Trek Online Advancement: You start with lowbie gear, you end with Lobi gear.
Sign In or Register to comment.