Less than 5% of the players go to raid, I wish

From Nick Yee (and from a reasearch on WoW):

3.6% of all observed characters spent more than an hour in raid content over the month of January.

"I want to believe". I REALLY wanted to believe.

I was ready to close an eye and use that piece as a wonderful occasion to rant again against a utterly stupid model of development that just goes nowhere. But what Nick Yee writes is largely bullshit and this time the bullshit is even too easy to detect.

His previous research was also completely off, isolating cases from the context when it was instead the context to be the most important element and the one that was interesting to observe. Extrapolating theories from aseptic environments is just an excellent way to see what you want to see. You can prove everything and nothing and along the years there will be an endless cycles of researches that just contradict each other. Rigged points of view.

This time what that test seems to say is also completely wrong. Of course is not the test itself to be wrong, but its interpretation. What they did was to track unique characters. The problem is that a character doesn't represent in any way a player or an unique account. It's kind of obvious that there is a majority of alts that aren't ALL involved in raid content even if the player is actually a raid player. The equivalence that 3.6% of all observed characters represents a 3.6% of unique accounts is just plain wrong. Of course this isn't what Nick Yee wrote but it is what everyone else would assume by reading what he wrote.

The point is that the test just says nothing useful and nothing that could be used on a concrete discussion (typical bullshit of academic discussions). We already knew that there's a disproportion between raid players and those who don't raid, but we still don't know exactly how significant this disproportion is and the test throws just more smoke in the eyes. I could have five characters, one of them being my main with which I raid most of the time, while the other four are used to dick around, play at the auction house or characters that I logged in once and then forget. This test would still say that 1/5 of the characters raid, we assume that 1/5 of the players raid, but this is wrong because in this example I am still a raid player that raids for most of the time.

Nick Yee can track the characters, but he cannot track the behaviour of a single account. So these tests are bullshit, they don't say anything useful or more reliable than the assumptions we already made without running tests. Things we already know.

The truth is different. The truth is that WoW transitioned many players to the hardcore group. Players that weren't like that and that finished to adapt to the game, dragging their friends in as well. WoW has the "merit" to have made the raid content much, much more accessible and widespread compared to other games. This is why the debate between casuals and hardcore is so strong today. WoW exposed this problem because before the "casual group" didn't even exist. We were ALL hardcore. Only the catasses used to play mmorpg. This genre was closed and specialized.

WoW brough the revolution, it broke the mold. It took the genre and demonstrated how narrow it was, how many limits it had. But at the same time it exposed a bigger problem that before was only latent. To that problem Blizzard wasn't able to answer. But there's a merit there, the merit to have gone past everyone else and having encountered a problem that noone else had to solve.

This cannot be denied and it's part of that "intellectual honesty" that doesn't allow me to just jump on the badwagon and attack Blizzard using bullshit data as a proof. I just cannot do that, despite there are huge flaws in how these games are developed and despite this could have been a perfect occasion for another stab.

We already knew there's a pyramid, we already knew that the current mmorpg development is retarded and narrow sighted, we already knew that Blizzard's development is now clueless. But the data in that test isn't significative and only confuses some more the situation, hiding what really matters.

I thought it was a good idea to point this out before everyone else and their sister start to wave that test as a proof of I don't know what. I had a post open on Q23, with the title written, before I discovered that it was just bullshit. So this is for all you bloggers. Just think a second before going on a crusade on this. I stopped right in time.

Re: Less than 5% of the players go to raid, I wish

On top of that, recall that--unless things have changed since I last checked--Nick Yee gets all his data from volunteers. I enjoy reading what he writes and appreciate how he handles the data he has, but there's only so much one can glean from such a sample.

Re: Less than 5% of the players go to raid, I wish

On top of that, recall that--unless things have changed since I last checked--Nick Yee gets all his data from volunteers.

No, this is incorrect.

On his site he collects data from volunteers.

In the case of Play On, instead, he just uses a better version of the census mod. So he just monitors passively the activities of the characters logged in.

If he used volunteers the study could have been more useful, even if still imprecise.

Yes, he presents things really well. But the presentation is not what actually matters.

Re: Less than 5% of the players go to raid, I wish

Ah, your points make much more sense now, in that case. :P

Re: Less than 5% of the players go to raid, I wish

I've been very careful when debating people on this to use the term "player character." Taking my wife as an example, she has roughly 30 characters above level 15 spread across six servers. She has one level 60, two characters in the late 50s (with no desire to hit 60, naturally), and four more characters above level 40. Of those characters, three have raided, or 10%. Nick Yee proposed a number of 30% of all level 60 characters participating in a raid during his study period: it's reasonable to assume that 50%-75% of all level 60s have been in a raid group at some time, albeit probably a 10 man in Scholo/Strat/?BRS.

Be careful with the statistics. If he doesn't have a lot of catassers participating in his study, then the left side of his graph (those who raid a lot) will be under-represented--I have the gut feeling that it's more accurate than Furor or Tigole would like us to know. A very small percentage of all of WoW's players raid more than five hours a month, although those are the most vocal and engaged players who also have the largest net presence via blogs and forum posts. Blizzard concentrates on relatively quick and inexpensive raid development for this vocal and influential segment of their subscriber base for similar reasons that Ford makes a US$150,000 GT--not everyone can have or afford it, but it will promote interest in the brand.

Re: Less than 5% of the players go to raid, I wish

I think the alt issue is irrelevant. Unless the raid-going player is multi-boxing alts *while* raiding (extremely unlikely ... and if it was happening it would say something really interesting about raiding difficulty), it doesn't matter.

Think of the 3.6% as a measure of content consumption rather than player account activity.

Re: Less than 5% of the players go to raid, I wish

Innsmouth said: "Of course this isn't what Nick Yee wrote ... "

So let me get this straight. You knowingly attribute an argument to me that I didn't make. You attack this argument as "bullshit". Then you go on to proclaim your own unsubstantiated opinion as "the truth".

Are you really so short on material that you have to argue with stuff you made up yourself?

Re: Less than 5% of the players go to raid, I wish

I'm criticizing the way your studies trick people to think in certain ways.

A survey, a poll or a reasearch are never "wrong" on their own, but in the way they are "read". There's ALWAYS a theory at the base, implicit or explicit. The research is a way to test that theory.

I criticized both this survey about the raid activity and the one about the avatars not because of the studies themselves, but the utility of the conclusions. Why take surveys if they have no concrete use? It's obvious that you aimed to demonstrate something and I criticized that implicit affirmation. The idea that everyone has after reading them.

It's "bullshit" because it has no concrete, solid use and because it favors a misunderstanding. What else is there to criticize in a survey if not its practical or theoretic conclusions?

That survey has implicit affirmations, everyone who reads it would arrive at a wrong conclusion. I was starting to write another attack to Blizzard after I read it, then I figured out the inconsistency and decided to write a defence, instead.

The "harsh" tone is because I don't "build consensus" on this site. I'm not removing legitimacy from your studies, you are already much more in high consideration than me. So I'm not worried about making a notch in your reputation.

Instead, for once, I wanted to doubt about what is usually considered as a truth.

Re: Less than 5% of the players go to raid, I wish

My lord, that is an amazing amount of criticism over what is purely an statistical evaluation that, in the end, ONLY offers very minor conclusions based on the raw data (i.e., statistical fact.

Note that he says:
"Thus, 3.6% of all observed characters spent more than an hour in raid content over the month of January."

He isn't saying:
"Therefore, only 3.9% of all WoW players raid."

There is a BIG difference.

He should be commended for not artificially altering his data with assumptions as you're suggesting, such as your paragraphs which begin with "The truth is different" and "WoW brought the revolution." These paragraphs are formed on pure opinion, and should be suspect by any reader.

The only part of your arguement that I do agree with is the value of the research. Instead of bashing Mr. Yee, why not suggest how to better farm data for a better defined (I didn't say 'accurate' since his article is accurate based on the data sample) model.

Re: Less than 5% of the players go to raid, I wish

Not sure if I agree with your take on this one or rather I believe your inference is incorrect.

1. the data represents accounts and taking a sample of 100 accounts, the data suggests that 3.6 people spent more than 1 hour accessing high end content in World of Warcraft across a specified time period.

2. the data represents avatars, so 3.6 characters accessed that same content for longer than an hour over that time period.

Multi-boxing could be considered a statistical anomoly over the large user base so there is no point in bothering about it however you could provide a +/- percentage to be completist.

Interpretation 1 or 2 doesn't really affect the underlying result because the potential for alts to impact the data exists equally for all users and one unique account is only playing one unique character at any given time.

16.5% of the characters were level 60 and given the requirements of the raid instances, it would probably be fairer to state that of those capable, 30% of World of Warcraft players accessed high end content.

What this data shows me is that Blizzard are creating content of 16.5% of the population of whom only 30% access it at all. And only 21.6% of that targeted population accessed it for any period of time.

-Illaraphaniel

p.s. And actually, you've made the inference that it's unique accounts in your heading, not the data as such as it specifies "characters". But in saying that, one character online per account means the terms are almost interchangable.

Essentially, 95% of all people when accessing World of Warcraft aren't accessing raid content during time spent in game.

p.p.s So you're heading actually skews the results. If you'd have said, "less that 5% of time spent in World of Warcraft is used accessing Raid content" you'd have been much closer to the data. Or even "More than 3/4's of World of Warcraft players can't be bothered using the raid content".

p.p.p.s The differential between total raid users versus raid users who stayed in longer than an hour can easily be attributed to alt swapping in raid for specific usages or for loot in my opinion.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.