Our Metrics Don't Matter

CCP Falcon

CCP Falcon

Today, CCP Falcon announced in a dev post on the EVE Online forums that CCP Games has declared the use of ISBoxer or any other automated input for multiple client sessions to be henceforth verboten. This news has generated both anguish and delight in the EVE Online player community, depending on whether those players were fans of automated multi-boxing or not.

What this all really means

For those unfamiliar with the above terminology, or what it means, the only important thing to know is that this is a wholly unexpected move by CCP Games, if one assumes that their key metric for measuring success is PCU - peak concurrent users. Simply put, this is the number of players logged in to the EVE Online server at the same time, which has become the most commonly monitored indicator of player activity in the game.

Players can start multiple simultaneous client sessions, one for each of their accounts - this is known as "multiboxing", and can mean either multiple instances of the EVE Online client running on one computer, or on separate computers. I multibox frequently, usually running two or three character sessions on a single computer and using Alt-Tab to switch between sessions - or I'll sometimes run a couple different clients at the same time using both a laptop and a desktop PC. Multiboxing is very handy when running coordinated operations with multiple characters in the game at the same time. I often do this for mining fleets, with two characters flying exhumers and a third providing fleet support in an Orca.

CCP Games has always allowed multiboxing, and it is a common practice among EVE Online devotees. I find it makes the game more interesting, as it requires me to juggle my attention between different characters - this can make even routine PvE activities, like high-sec asteroid mining, a little more challenging. And it is certainly handy for Level IV missions or small fleet hauling runs.

Some people get really crazy about multiboxing in EVE Online. Like, really, really crazy.

Some people get really crazy about multiboxing in EVE Online. Like, really, really crazy.

While CCP Games allows multiboxing, it has always banned the use of automated macros or "bots" - programmed input of actions in the game without human involvement. I have personally observed more than a few obviously automated mining and mission-running bots. In one case, I saw someone gank a mining barge, and the pod continued to shuttle between an asteroid belt and a nearby station, over and over, for hours - clearly, a poorly programmed automated routine. Several of us who witnessed this behavior reported it to a GM, and we never saw that character in game again.

However, CCP Games has long ignored the use of automated input of simultaneous commands to multiple client instances, known as input broadcasting. This form of automation was very helpful to players flying fleets of ships piloted by their own characters, each running in a separate multiboxing instance. Input broadcasting software, such as ISBoxer, made this much easier to manage, and enabled individual players to perform complex combat maneuvers, such as massed stealth bomber attacks.

While many players decried the unfairness of automated input broadcasting, those who used ISBoxer and similar tools defended it by pointing out that such practices actually benefited CCP Games, since it increased the PCU metric of success. As a result, many ISBoxer users felt safe and secure that they would continue to enjoy the advantages that automated multiboxing provided them.

When you assume...

With CCP Falcon's announcement, CCP Games proved the input broadcasters wrong. Starting January 1st, 2015, the use of ISBoxer and similar automated tools are no longer welcome, and will receive the same harsh treatment as the use of macros and bots.

Soon after CCP Falcon's post, I saw numerous #tweetfleet posts on Twitter from ISBoxer users, who were shocked and upset that CCP Games had suddenly turned on their previously allowed practice. "It's dumb," one Tweet said. "Expect to see PCU drop fast."

Personally, I have never used ISBoxer, as I find automated broadcast input to be a disproportionately unfair advantage to those who use it. Yes, it does increase PCU, but it does so while providing unnatural and artificial abilities to individual players, by using tools to affect their real-time commands into the client.

I recently debated with one ISBoxer user, who tried to equate automated broadcast input tools to the use of other third-party applications, such as the EVE Central market database. His argument had little merit - EVE Central is not designed to input orders into the EVE Online client without human involvement, as ISBoxer clearly does.

ISBoxer users have a right to be somewhat confused, however, since it has taken so long for CCP Games to declare their practices as now bannable. However, their confusion is based on a false assumption - that CCP Games' chief metric of success is PCU. CCP Falcon's announcement, along with other CCP Games actions over the last year, indicates that this is no longer true - and this bodes extremely well for the health of EVE Online, and for the future size of its player community.

Be careful about how you measure success

In Real LIfe, I provide business consulting services to my clients. Recently, a manager called me with a problem. He had wanted to increase the number of successfully resolved customer service calls to his business. He had established a bonus system for employees who received such calls, providing a small incentive payment for each call logged as addressed and closed. But a month later, his customers' satisfaction ratings and repeat business had plummeted. Why? Because it didn't take long for employees to figure out that to maximize their earnings, they should finish each customer service call as quickly as possible. While the number of successfully resolved calls did increase quite dramatically, it did so at the expense of relationship-building, rapport and friendliness, and customers didn't like it.

Why do I tell you this story? It illustrates that over-emphasizing any one quantitative metric of success can lead to aberrant results. For many years, CCP Games focused too much attention on PCU. In fact, during the tenth anniversary of EVE Online, it promoted an event to drive PCU to record levels.

And when you first start the EVE Online launcher, what do you see first? The number of concurrent pilots on the server.

PCU is a useful metric of general player activity level, but it provides no insight into the quality of the experience of those players. Under CCP Seagull's leadership, the EVE Online development and community teams have begun to realize that positive changes in the player experience will produce the best results eventually, even if that means short-term setbacks in certain activity metrics, such as PCU.

At EVE Vegas last month, CCP Seagull said, "It's time to get over Incarna and start taking risks again." That expansion, released in the summer of 2011, was so poorly received that it resulted in dramatic reductions in player subscriptions and PCU counts. This eventually resulted in layoffs at CCP Games, and rattled the confidence of the entire company. After Incarna, devs focused almost solely on improving the flying of Internet spaceships, providing a series of solid, but relatively safe incremental expansions.

qualityvsquantity11.jpg

It is because of CCP Seagull's vision, more than any other factor, that EVE Online developers now feel emboldened enough to take risks, even if that means changing long-held assumptions about many game mechanics, such as the recent shake-up of force projection ranges.

As a result, the most recent and now also more frequently released expansions have focused primarily on improving the quality of the player experience above all other considerations. And this emphasis on the quality of player experience first, over any quantitative success metrics, is now paying off. EVE Online is certainly a better game today than it was a year ago - it has more of a sense of momentum, direction, excitement and fun - more heart - than ever before.

We in the player community may continue to watch the daily PCU counts and wonder if "EVE is dying", but our metrics don't matter. The only thing that really counts is whether CCP Games can remain courageous and dedicated enough to make the tough choices and implement the changes that help EVE Online to improve, prosper and grow. CCP Falcon's announcement is yet another recent indicator that this is indeed the case - and I couldn't be more impressed.

Fly safe! o7


How do you win EVE?

scoreboard-me-1-you-0_design.png

How do you win EVE Online? How do you keep score?

  • Is it the number of kills on your killboard? The ISK value ratio of your victories versus losses? The number of battles you have fought and survived? The number of corpses of your enemies that you have accumulated?
  • Is it the total value of your assets? The amount of ISK you have in your wallet? The size and number of production lines you have operating on all your manufacturing alts?
  • Is it the number of skill points you have accrued? The number of skills you have at level V? The average point value of each skill category?
  • Is it the number of transactions you've closed in Jita? The total profits you have amassed?
  • Is it the number of naive pilots you have scammed? The value of all the stuff you stole from the trusting corps that you infiltrated?
  • Is it the number of ships you have in your hangar? The number of exotic dancers you have collected? The amount of weird flotsam and jetsam you've taken from all the missions you've flown?
  • Is it the number of noobships, industrials and miners you have ganked in high sec? Is the number of tears you have elicited from their anguished cries in Local? Is it your negative sec status value? The size of your bounty?

What is the metric of success in EVE Online? How do you win this game?

For me, I have enjoyed pursuing some obvious metrics of success in EVE:

  • The number of types of flyable ships in my collection
  • The amount of ISK in my wallet
  • The total value of all my assets

As you can see, I'm naturally a hoarder and a builder and a trader, mostly. EVE allows me to play the game that way with some success, as I defined by these measures.

But when I really think about what makes EVE most enjoyable for me - the kinds of things from which I get most satisfaction in game - I come up with less obvious but more significant measurements:

  • The number of 1 ISK donations I get after I teach a class in EVE University - I always ask students to send me a 1 ISK tip with a comment if they liked a class, and I get some very nice replies, and sometimes a bit of a bonus. I must admit, knowing that people got some value out of a lecture and some practice always makes me feel pretty good.
  • The number and variety of classes and events on the UNI calendar - me and my staff strive to schedule at least one activity or lecture every day, and we usually do far better than that. I always get a sense of satisfaction to see a calendar packed with fun stuff for people to do - it's what our corp is all about. (That reminds me, I need to get some more stuff in there this month...)
  • The number of students attending our classes and events - we track how many UNIs come to the classes and events in our calendar, and that number has been trending up for quite a while now. It means we must be doing some things right.
  • The number of EVE players with whom I converse with in chat, on Mumble or at Fanfest, who I consider friends.

I think it's interesting that none of these metrics have anything to do with game mechanics, yet they are very much a part of how I play EVE.

So, how do you win EVE Online? I think you win when the time you invest produces results you find supremely fulfilling or satisfying - whatever that may be for you. I think knowing what those metrics really are for you is one of the principal reasons that players stick with the game, or conversely, why they drop out.  As for me, I'm still enjoying "winning EVE" in my own way, even after nearly four years of play.

I've read some EVE bloggers who say things like: "High sec players need to go play in null sec." What they are really saying is: "Your metrics of success are invalid - you need to play by my metrics." They assume that their measurements are better, and that everyone should play by those standards.

To which I say only this: "Dude, welcome to EVE. Nothing here works exactly like you think it does."

I hope you find your own victory conditions, whatever they may be - in EVE Online, and in Real Life.

Fly safe! o7