The trouble with gauging difficulty – how hard can it be?

hedgemaze2 computerguy_wiki@flickr

Mulling over recent posts*, I noted after the last WoW patch that I had no real idea how difficult the new 5 man instance was because my PUG had kerbstomped it on our first attempt. Was it really that easy? Did we overgear it? Were we lucky with the randomised bosses? Or had I lucked out with the world’s best PUG?  (They’d also got lucky in getting an experienced, Ulduar geared tank – I don’t claim to be the world’s greatest but obviously it makes things easier.)

One thing is for sure, it is hard for us as players to gauge how difficult a challenge will be for someone else, especially when you don’t know them well. We know what will make things easier: –

  • experience with the game
  • good group (if the game is group based)
  • practice
  • better gear (if the game is gear based)
  • luck (if the encounter or game is luck based)
  • knowledge of mechanics
  • hand-eye coordination (if the game is twitch based)
  • good reactions
  • … and any number of other variables that add up to player skill. And even there you can see I’ve listed a lot of ifs.

If you read official bulletin boards, you  will often find hardcore players boasting quickly about how easily they beat new content. This can be demoralising if you and your group are struggling on the same encounters. It’s easy to assume that this is just e-peen waving, the same way people boast that they passed all their A-Levels or got first class degrees without doing any work.

But sometimes, it is because they genuinely just found it easy. If it’s difficult to compare the difficulty of solo content, it’s almost impossible to gauge how hard another raid group will find the same encounter. How can a hardcore raid guild full of handpicked hardcore players in best of slot gear possibly imagine the difficulty of an encounter for a more casual group? They can’t. Their answer will always be ‘get a better guild, noob’ because that’s probably what they did. Not only that, there is no real way in game to recognise the achievements of a more casual group without comparing them to the hardcore. It doesn’t matter in any real sense but it’s a bad idea to feel despondent over these comparisons.

It is definitely worth taking advice from the pros but take their opinions on encounter difficulty with a pinch of salt. I know that I’ve read about people struggling on encounters that we breezed through (Auriaya is a good example) and vice versa. Different groups do face different challenges.

If a recently dinged 80 tank successfully steers a group through the trial of the champion (new 5 man instance)  on normal mode, then my hat’s off to them. I honestly have no idea how hard it is at that level of gear and experience, but I can easily believe that it’s an achievement of which to be proud.

Gevlon did an experiment recently in which his guild cleared Ulduar in blue gear. It proves nothing new. Back in TBC, there were guilds who completed the timed Zul Aman run in Karazhan gear. There weren’t many and the guys who did were supremely skilled. It never proved that any guild could do that, and it never proved that gear didn’t matter. All it showed was that the game was flexible enough that highly skilled players could throw away the safety net offered by better gear. In order to do it, they had to put in a perfect performance.  And still, if a new player showed up to a raid guild, they would probably prefer to see a decent set of gear if only because it shows dedication and proves that they understand their class mechanics.

Even if skill does trump gear, gear is easier to prove. This gets people into all sorts of knots with PUGs at the moment. You hear stories about players demanding to see gear, achievements, etc before issuing invites. And I’m sure this is an issue in any mature game – experienced players want the easy ride that they’ll get by inviting other experienced/geared/skilled players. Part of the problem is that you have to be a skilled player yourself to recognise skill in other people.

For example, I ran a heroic instance on my healer just before the last patch. The tank had 18k health when buffed (he was a paladin). Two of the dps looked at this and left the group before we started, and honestly it IS low for a heroic tank. When I pointed that out to him, he said “I know, but I do have 535 defence so I won’t get crit.” I figured that anyone who actually knew the defence cap for heroics (this is fairly arcane tanking knowledge) probably also knew his stuff and was worth a shot. When I failed miserably on the first boss and he died, the rest of the group immediately blamed his health. I apologised for not realising that there was a silence effect involved (hey, I’d only ever tanked that place before) and after that we had a smooth run through Old Stratholme. I’m not entirely sure what that proves except that it doesn’t matter where skill really does trump gear, what actually matters in game is what players believe.

Devs can look at the hard stats to set appropriate difficulty in encounters. They know what people are doing in game, where they struggle, how much damage they can typically heal or produce, and so on. But as players, we only know how hard it was for us.

* Present participle or gerund? What do you think? (Yes this is a shameless experiment in whether asking direct questions gets more comments – also an interesting grammatical query 😛 )

20 thoughts on “The trouble with gauging difficulty – how hard can it be?

  1. I got crucified a while ago for suggesting that skill wasn’t really a factor in most MMOs and that time was the more important of the two. I’m not going to reignite that old flame but there’s no denying that gear plays a major part in MMOs and that a lot of gear can be achieved through time and perseverance alone.

    Still, what makes a game challenging or not is a subject I’m not qualified to discuss. It falls into the whole realms of game theory design and better people than I have written a lot about it all.

  2. Back on topic I think the issue with WoW is several

    1) it’s a group game. One great player, especially the tank, can carry several hopeless ones.

    2) mudflation is extreme. While Gevlon has a good point about skill it’s a hell of a lot easier to dps with a 160 base damage weapon than with a 90 base damage one. I think WoW gave the players too big increments although I can understand the historical reasons for this which date back to the gear jump between 60 blues and MC loot.

    3) People say “skill” when they really have no idea. I had a character lambasted for lack of skill when I hit 80 and then praised for exceptional skill after I geared up. I pressed exactly the same buttons in exactly the same order. Nothing changed except gear and people’s perceptions.

    Lastly I think people are too obsessed with skill which relates to “I don’t want to spend all night wiping because you suck”. In serious raiding everyone sucks some time. Sure people who are always terrible should be weeded out but people have become so wipe-adverse nowadays. Changing the team is not always the answer but it’s increasingly becoming most players’ preferred solution.

  3. The skill question can really be answered by asking two other questions:

    1) Were you born knowing how to do it? If yes, then no skill is involved. If no, then some degree of skill is required. The only thing left to argue over is how much, exactly, relative to similar activities requiring skill.

    2) If an activity requires skill, is it possible to be bad at it and what are the relative levels of those who are bad versus good versus extraordinary? Tying one’s shoes, for instance, requires a learned skill, but it’s hard to imagine anyone being bad at it so you can conclude it requires very little relative skill. Healing a heroic ToC run (and only healing, DPSing and tanking ToC are easy — I’ve done all three many times now — the difficulty in that one-room excuse for an instance is entirely on the healer) takes a fair bit more skill, and it’s not only easy to imagine someone being bad at it, but you can see it in-game any time you like.

    People hold to this odd notion that most of their fellow players are not very skilled — and seeing the non-hit capped/sub-1K dps, undergeared healers gemming for parry, and non-def capped tanks near every day, it’s hard to disagree — and yet simultaneously the game takes no skill. These are mutually exclusive notions.

    What separates those who are bad at something from those who are good at it is skill. By definition you can’t be bad at something that takes no skill, and if most people are bad at something then then the skill level required is, again by definition and obviously and blatantly, not inconsiderable.

  4. I think that getting comments by asking questions only works if you can assume that the average reader has an opinion about the subject of your question. On the internet even spelling is rare, so I wouldn’t set my hopes too high on questions about grammer.

    Skill and gear on the other hand is very easy. Every MMORPG player knows:
    I succeeded because of my skill.
    I failed because of lack of gear.
    You succeeded because of your gear.
    You failed because of lack of skill.

  5. Well yesterday one of our (new) tanks dinged 80 and we decided to give him a spin – he ran us trough ToC. On heroic. Well he cheated a bit – he chain ran since lvl 78 the ToC normal so he was having all the tanking drops from it + trinket. Also had all the crafted tanking epics – ring + neck + head + shield + cape. Bought himself hodir rep for the shoulders ench and with the argent tabard he was having the head enchant.

    And yes – he was overgearing the heroic from the moment he was 80. Was he skilled or geared – the answer is both – he was skilled enough to know how to obtain the easy hanging gear fruits.

    And he was taking moderate amounts of damage.

  6. You probably were a bit lucky… but your tanking certainly made up for the difference!!

    Bringing in a good tank is three quarters of any fight ‘done and dusted’. You may not necessarily carry a poor group, you still need healed and the DPS still has to shorten the fight, but awesome tankage will make life a lot easier for everyone else. DPS can play to their MAX and good mitigation makes a less taxing shift for the healer.

    For my first visit to TotC Heroic, the PUG group I was in did the normal run 3 times, and had more difficulty on the first normal run than we had in the Heroic. We one shotted the Heroic with my alt Paladin tanking and my guilds ‘MT’ Warrior playing his PVP geared discipline specced Priest healing! The other three were ‘randoms’!!! We did however reset the fight after the mounted section (I’ve done that ever since the normal second run there).

    If I am honest my own Shaman Healer may not have done the heroic with my Paladin, especially after the first normal run!! I think our MT probably wouldn’t have trusted his char with his own healer!!! Since that first day My Paladin has nearly caught up with my Shaman, thanks to the Emblems of Conquest gear. Something in 3.2 that any altist will cheer about!!

    There is a fine line between gear, skill and luck in WOW dungeons, it’s just enough to make it interesting.

    My Shaman is over geared for heroics, and usually DPS’s as much as heals or I’ll just be following the tank casting Earth Shield laying down some totems and refreshing riptide!! However even though my shaman is over geared, I’ve seen some of our over geared Guild runs going pear shaped due to a bit of bad luck (especially ones in Old Kingdom!!).

  7. I acutally like running with people who are the appropriate gear level sometimes in Heroics. A lot of the people need something from there and enjoy the fact that they get a geared tank to lead them through.

  8. all I have to say on the matter is Skill != Knowledge.

    Some people may not know the fights but do well and others may know the fights but cannot move out of the danger or whatnot in time. They’re two separate but related aspects. When someone has both it’s a beautiful thing; when someone has neither that’s where the dreaded failPUG comes from.

    • All I know is that when I know the fights really well, everyone thinks I’m more skilled 🙂 So knowledge can substitute for good reactions (ie. if you already know what’s going to happen, you can prepare for it).

  9. In FFXI gear trumped skill. Didn’t matter how skilled you were, nothing really replaced it. I’m guessing it’s less of a case in WoW because WoW overall is an easier experience, and you have much more margin of freedom. But in FFXI there was nothing you could do skillwise to make up for shoddy gear on virtually every job. Your enfeebles would get resisted, you would hit for pathetically low (badly geared players could hit less than a well-geared player’s pet in total DPS) miss over 50% of the time, and just suck all around.

    Skill I think is mostly how good you are at decoding the game’s stat system, and then just a basic level of competency.

    • Couple of non-sequiturs there. If it didn’t matter how skilled you were in FFXI then WoW (or any other game) could not be easier. (Since FFXI takes zero skill).

      If skill is decoding the game’s stat system why do Lava Walls and Void Zones wipe raids? Is decoding the game’s stat system unnecessary in FFXI? (since it needs no skill).

    • FFXI is hard not because it takes a lot of skill, but because your players have less margin of error without required gear or setups. The gear is first-it doesn’t matter if you are a skilled beastmaster, if you don’t have enough charisma on your charm build, you will miss 25% of your charms and die a lot. The actual skills you build are valuable, but the gear always matters first.

      Or on a bard pulling merits. Before you can even be skilled at stage pulling, you have to have the gear to land sleeps on mobs. If they resist it, you die or slow the party to a crawl, even though you are expert and calculating mob repops and pulling an infinite chain. If you don’t, no matter how skilled you are, an average bard with great gear will probably be better and more consistent because he isn’t getting eaten by three mobs.

      I’m betting lava walls and void zones wipe raids because your WoW player really isn’t taught on the level grind up to follow instructions or your party will wipe. In FFXI even though endgame is as different as it is in WoW from levelling, we learn pretty early if we screw up we wipe the party, and we know stuff will wipe us very easily if we don’t pay attention. That’s not even skill-you don’t need to be skilled to read a faq for an endgame event and follow it, or know to look up a FAQ for your class on a forum.

      We have to in FFXI because we will wipe people and lose a lot of experience if we don’t, early on. Even with a more solo-friendly game, and lower level powerlevelling, there will come a point before endgame where if we don’t have the right gear first, and a very basic amount of skill second, we will screw up horribly in party or mission play. Skill just sweetens what gear makes possible.

      I really think WoW has the illusion of skill because the entire grind is so forgiving up until endgame that most players enter it unprepared, and everyone overestimates the actual difficulty. I mean, compared to high level FPS or fighting game play, for both games it’s just “wear this, stand here, avoid this.”

  10. How many times have you one-shot a boss, bosses or even an entire instance only to assume it is easy.

    The next time through it’s a pure wipefest.

    Sometimes we just get lucky and we are all standing in the right position right from the beginning of the fight… sometimes we aren’t.

    The “skill” is to know when you are in the wrong place and move before it’s too late.

    • This is exactly it. Because everyone tends to assume ‘Oh, it was so easy for us last time, it must be SOMEONE ELSE’s fault that it isn’t easy this time.’

      Sometimes being able to slam through an encounter just means that no one really understood what it was all about, which is why you get taken by surprise later.

  11. Skill has various tiers – the same as gear. First is – dont stand in the fire. By mastering it you avoid around 60-70% of the incoming damage. Second tier is – when you have debuff run to some place. Here you shave another 10% of your healers workload. So even with people that have only this basic skills you are able to clear most of Ulduar and Naxx. And they are almost the only requirement for normal modes. And since the new HC shows that blizzard opinion on gear is – purple ilvl 200 is baseline you have quite a margin for error. The problems begin when blizzard tries to move away the players from the safety zone of the previous two skills – think the pvp encounter in ToC or Yogg-saron/Mimiron – it seems that more of the groups don’t have the precision this fight require, even though they have more than enough the firepower and healing power to finish this encounter.

    • I think the big problem with the faction champions isn’t so much the different skill set but that there is so much going on that it’s very difficult for raid leaders to point to people and tell them what they’re doing wrong.

      It’s a fight where it’s tricky to give people useful feedback. And since you can get a different lineup every week, you can’t just learn the strategy.

      Now some people will enjoy that different type of challenge, others will hate it. But I do think it’s harder to pinpoint what people are doing wrong on that fight.

  12. Doing ToC with no heroic or raid gear in the group, when none of us had seen it before, were a freakin’ challenge. Now, I had me a few crafted/rep epics, so as a tank I were in good shape, but everybodies else were pretty much in rep rewards and crafted pvp blues. We did okay, but that skanky confessor chick were too damn much fer us. We just couldn’t dps that summoned memory nightmare jerkface down before the healer got overwhelmified and we wiped. Again and again and again…

    Now, a few weeks later, we’s got better gear and we knows the fights, and she goes down every time. I imagines if we’d had Ulduar gear that first day, we woulda knocked the place out even if we was watching tv and high on felweed. Everything be relative.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s