Community Collaboration and Dangerous Data

I believe very much in getting the players feedback, most is not valuable in the slightest and the usual forums are not a good place for ascertaining the true feeling of a population but there is a certain element there that is needed. The masses generally don’t have an understanding of what goes on in the developing and running of an mmo and so they’re feedback is centred on experience rather than the restraints. Developers also have their goals, ideas, and data that it is all founded on and we are mostly not privy to that, well not in a way that matters anyway. It is a difficult balance to achieve, sticking with a vision or getting feedback from the uninformed but I just wanted to elaborate on my comments in the previous post about why I believe getting a direct and increased amount of player collaboration into development and planning can be a good thing for a game.. more so than developing in isolation.


I am just another player gorging themselves on games and the way I look at things is entirely different, and from my view I see three undeniable facts. One is that they are often wrong with the direction of development or how they have developed something either due to lack of knowledge, understanding or skill; a bias in certain interests; or due to pressure from many factors. With the pace of developing these large projects it is very hard to remain aware and develop for the current standards, more than that you need to be a step ahead, therefore it is not opertune to develop within a vacum. Some of these projects can and do create specific in-groups whose ideas will often dominate without regards to other information yet every idea needs a critical opinion to further the dialogue .. this is kind of what happens when you don’t.. hmm, maybe not that ugly.

In quite a few of the GW2 state of the game videocasts I witnessed that the developers were insistent on their ideas and visions to the exclusion of any outside sources. For weeks to months the playerbase was telling them about certain issues to have them give the “working as intended” response only to later recant to this. I remeber one situation about the damage and survivability of certain Ele builds that was constantly expressed to be insanely strong, then months on they only just started realising this based on their own play…it really is the kind of the that makes you lose trust and respect for the people designing your game

A lot of times it just comes down to the fact of being unable to test all the mechanics, and even if they are aware of issues it is getting the resources to fix them

Two is that they often hold their own opinions higher then that of players. When this happens it can result in a group that is resistant to change yet games, and mmo’s in particular should always be iterative and adaptive. Also, believe it or not but certain players do and will have insights into a game about mechanics that might not have explored or issues that might not have been thought of. Through many games, over and over I’ve also witnessed what I would call savants from within the playerbase that manipulate algebra, the systems, statistics and optimise play in ways the developers would never have dreamed.

Which brings me to the third point. What they perceive will happen is not necessarily what the players do. Yes there might be a tonne of metrics showing what players usually do. QA testers might have spent countless hours, days, and months perfecting a certain levelling process or content patch only to watch as it is decimated under the pressure of thousands upon thousands of eager players. Wasn’t Tequatl meant to take weeks to down; who knew that players would get to end game so quick in Swtor; how were they to know the insidious effects the AH would have on Diablo…. the players that’s who and well beforehand.

Developers can be and are very wrong at times and based on certain press statements and appearances many need a lesson in humility. You can be wrong and will be wrong from time to time, you are human so no one expects you or the game you create to be perfect but what gamers do ask is for you to keep an open mind. They might act like you should be perfect, or hold you to a higher standard but it is really nothing that can’t be solved by elaborating on your thinking in the open.

Data Mining

Developers are increasingly using more complex tools for gathering information regarding the patterns of play. It is undeniably a great power that has been granted developers so that they can create content better suited to their players habits, likes and such but, you know that saying, with great power yadda yadda. I have no doubt that many mechanics and such in current and past mmo’s have been solely developed and employed because the data said they would reach a larger target audience in order to sell more boxes and that has mostly worked. The genre has seen increased growth.

However, I think the data and statistics right now are often misused and cause a myriad of issues just as much as they benefit the players and the games they are used in. Having more data while being a worthwhile factor in some areas for ironing out imbalances and such, unfortunately it seems many have become a bit complacent because of it, relying on this more than the word of players and even more so than any sort of vision for the project.

One example about metrics I hear pop up regularly is developers making changes to an area where people are regularly dying but if we remove this we make a world devoid of differences and subtleties. Exploration is not just about finding different areas and poking your head into every nook and cranny but also learning about the world. The powerful monsters, the better loot, the areas to avoid are all part of the experiences you have and anything that removes that is just making another bland space.

Sometimes data won’t be able to tell you any confounding variables that may be affecting it, for instance why a type of content is popular. Fractals in GW2 were a huge thing when they initially launched and the number of players enjoying it was immense for some time but even if you know they like it the why can still be elusive. Is it to gain progress, to feel powerful, to complete a challenge, greater customisation, or the competitive nature. Is the reward better, the novelty of new content,  or did people just enjoy this style of content more than the others.

How do you distinguish between true popularity of content and people just doing it for reward? When is it beneficial to continue with a certain type of content over others? And what are the right reasons to to be promoting these types of content when many aren’t doing them solely for the benefits of fun?

Another example from GW2 is the SPvP side of the game. It has been rather unpopular to date, that is quite clear and if I were a developer and going through the data I would probably just write it off. Was it the lack of matchmaking as numbers have gone up, the smaller tournament grouping, is it the game mode, or classes, balance, maps, other features, or just the combat. There are so many other factors that are a complete unknown. These are times when you should come down from the ivory towers and be actively questioning and polling the playerbase. I’m not talking about the “pro’s” I see in the state of the games or the vocal minority on the forums, it needs to be a proper cross section of the playerbase.

Even if you do have something that is widely popular people can universally hate something but still continually consume it, the data would attest to this with wow and the dailies. Is it ok to make content that people do yet hate? Is it ok to have a certain amount of content that people don’t enjoy but is somehow required? Even then how do you trust yourself in determining what to do or add? These are questions no amount of data can solve but that can be helped by gaining insight from the playerbase.

These days though I wonder if the patterns of play, popular features, and enjoyed content are even good ways to make an mmo as popularity does not equal retention… or even a fun game.

#ArmchairDeveloper #data

5 thoughts on “Community Collaboration and Dangerous Data

  1. I think the data mining and the more sciencey (not at all interacting with their own community) aspects of running a MMO do hurt a lot. They feed directly into that sterile feeling I get when playing newer MMOs, that seem perfectly designed to not stand out at all to any one particular group.

  2. I love analytics. The problem with analytics is that it lacks context. Sometimes the concept lacks analytics. Wow dailies are a GREAT example of poor use of analytics. Everyone did dailies in Cata for some rep grinds. The rep grinds hid basic gear and some cosmetic things at tail end of Cata. MoP design said “so many people do dailies, let’s make more and put more rewards behind them”. Without context, that is logical. Sadly the context said that gear availability was achieved through many means and they cut that to one.

    I get it, the picture is big and complex. People who see it all are rare. Professional guilds don’t even see it all. But the developers, they are paid to do that.

    • exactly. The see the popularity of one style of content and want to replicate that but that lack the context of why players do it in the first place. When you do that you’re also designing against the payers that don’t like the particular thing.

      Understanding the why and how should be more important than just knowing it does

  3. Pingback: This week in Guild Wars 2, 26 October-1 November | GuildMag

Comments are closed.