Why Players Blame Skill-Based Matchmaking for Losing in Call of Duty

Two months ago, esports pro Seth "Scump" Abner logged into the Call of Duty: Black Ops Cold War multiplayer alpha and found himself struggling. Not because of any major gameplay changes developer Treyarch had made, Cold War plays like any other Call of Duty from the past decade, but rather because of the players Abner was being put up against: They were all good.  

This, Abner felt, wasn't normal. He should know: he's a world champion, he spends dozens of hours every week playing against the best in the world, and dozens more streaming his "casual" play on Twitch. Why was he having to suddenly work so hard to win games? A few hours into the alpha test weekend, Abner came up with an answer: it was the skill-based matchmaking (SBMM). 

Skill-based matchmaking, as you can guess, is a type of multiplayer matchmaking system in which players' are pitted against other players of similar skill level. In other words, the Black Ops Cold War alpha was purposefully matching Abner up against players with players who were just as good as him. This, he felt, was not good.

"[Skill-based matchmaking] does not belong in Call of Duty. There should be a ranked playlist for people to sweat in," he tweeted as the alpha weekend was coming to a close. "I’m not trying to play Scuf wielding game fuel chugging demons with szn in their psn on Miami TDM."

Abner wasn't the only esports pro to take issue with this system. With the release of Cold War last week, a number of notable streamers have echoed Abner's criticisms. Skill-based matchmaking, they argue, takes their agency away, forcing them into a purgatory of having to play their "best" every single game. 

"I truly believe it is imperative that Treyarch dials back the difficulty of lobbies. We’re gonna drive so many big creators away, these games have been no joke. I’m dead after playing for 9 hours," 100 Thieves CEO and CoD pro Matthew "Nadeshot" Haag tweeted a day after release. " I don’t mind playing against people that are similar to me in skill, but at least tell me where I stand in comparison to others globally. Give me a rank that reflects my skill."

These critics point to a number of games like Call of Duty: Black Ops 2 and Halo 3 as examples games who have gotten multiplayer "right" by letting players choose between a "ranked" playlist and "unranked" playlist—offering the freedom to decide when they want to sweat and when they want to kick back and own some noobs. Modern multiplayer developers have made a serious misstep in implementing skill-based matchmaking across the board, they argue, and they should go back to the way things used to be. This all sounds reasonable, were it not for the fact that skill-based matchmaking has been in every major multiplayer shooter since Halo 2. 

"From day one in Halo 2, we had skill-based matchmaking," said Max Hoberman, the former Halo 2 multiplayer lead who now runs his own studio Certain Affinity. "You had ranked and unranked playlists, and even the unranked playlist tracked and matched you based on your skill—the sole difference between the two is that one didn't display your skill rating."

According to Hoberman, the reason why Bungie decided to make the distinction between unranked and ranked playlists, despite them running on the same matchmaking systems, was one of perception. 

"I knew that matchmaking and rankings were inherently competitive and a lot of people just did not want that in-your-face competitiveness," Hoberman said. "So I launched with ranked and unranked playlists so all the hyper-competitive people—and a lot of the assholes, to be blunt—end up going to the ranked playlist and the unranked can have a much more casual, friendly environment."

The issue today is not that skill-based matchmaking exists, but that players are now aware of just how prevalent it is. Up until recently, one could assume that joining an "unranked" playlist meant they were being dropped into matches with the entire playerbase, and thus who they played against was purely random. Under this false assumption, it's easy to wave away bad games as flukes, while conveniently believing that any good games were the result of skill.

Now that most know that they're being matched with people with similar skill levels all the time, they can't help but perceive their opponent as equals. In a sense it's become a sort of cursed knowledge—how can anyone have a "casual" play session when you know that the algorithm is pushing you to play against your equals? What was once thought to be a random and "unbiased" system is now working against you, and therefore anything bad is now obviously the fault of skill-based matchmaking. Had a super sweaty game? SBMM. Not performing to your perceived level of skill? SBMM. Getting destroyed? SBMM.

It's similar to a phenomenon in MOBA games like League of Legends or Dota 2 known as "Elo hell." Referencing the chess ranking system developed by Arpad Elo—which modern video derive their own skill-calculations from—players say they're in "Elo hell" or "the Trench" when they feel like they're unfairly matched against players better than them and with teammates who aren't as good. To be stuck in "Elo hell" is to be in a sort of purgatory, in which the matchmaking system is ostensibly conspiring against you to artificially keep your rank low despite you knowing that you're much better than the system thinks. 

"It’s hard to imagine that the Trench exists. More likely, it’s fueled by a combination of Dunning-Kruger and cognitive bias. Players who believe in the Trench likely have a low locus of control, and believe that losses 'happen' to them," wrote Suriel Vasquez in his examination of the phenomenon for Paste magazine. "When you’re at the appropriate ranking and you’re winning around half the time instead of rising up, that’s not a quirk—it’s arguably the intended result."

The thing about skill-based matchmaking is that it's not as perfect as its critics think it is. And that's by design. Hoberman explains that in order to strike a balance between ensuring that skill gaps between players were close while keeping matchmaking wait times low he allowed for a certain amount of "slop in the system." 

"Sometimes there were games where you had the upper hand and you felt like a badass, but you'd also have matches where you'd get your butt whooped" he said. "You kind of get your ego bashed a little bit, but you get exposed to better players and aspire to improve." 

In fact, Hoberman said the Halo 2 matchmaking system worked so well that Microsoft's TrueSkill system—the skill-calculation system implemented with the Xbox 360 revamp of the Live service—was based almost entirely off of it. 

And despite what the skill-based matchmaking detractors might say, the numbers suggest that most players enjoy playing with players around their skill level. A 2013 Microsoft Research study of Halo 3 player skill found that, "almost all participants thought they improved when playing with better players than themselves, and most (but fewer) thought they improved when playing against better players than themselves."

Josh Menke, who has worked on the matchmaking systems for just about every major multiplayer franchise from Call of Duty, World of Warcraft, Starcraft II and now Halo, firmly believes that skill-based matchmaking actually increases player engagement, and keeps them coming back, rather than turn them away.

"During Halo 5 we have tested both extremely loose and extremely tight matchmaking with both systems. I have seen zero evidence that matchmaking tightly on Halo 3’s skill system leads to quitting," Menke said in an email. "In addition, when I look at TrueSkill2 data, I see ample evidence that matchmaking more tightly on skill leads to significantly less quitting (4-6 times less)."

By Menke's own estimation, he sees skill-based matchmaking becoming more prevalent, despite what some of the world's most popular players say.

"From the outside, I’ve noticed more and more very popular games adopting SBMM and I sort of assume it’s because they are seeing the same data I am," he said. "Despite a vocal minority in some games not liking SBMM (and for the most popular games that “minority” can be 100,000+ players), devs are seeing a clear link between higher engagement and tighter matchmaking. Being in development myself, I’ve noticed studios don’t tend to add stuff as involved as SBMM unless the data really supports it."

The unavoidable truth about multiplayer matchmaking is that there will always be winners and losers. Someone's success always comes at the expense of someone else's failure. When players ask to be put into matches in which they can reliably chill and get 20 kills while only dying 10 times, this inevitably requires someone else to die 20 times. What they're asking for is special treatment. And that's just not fair.