Our Metrics Don't Matter

CCP Falcon

CCP Falcon

Today, CCP Falcon announced in a dev post on the EVE Online forums that CCP Games has declared the use of ISBoxer or any other automated input for multiple client sessions to be henceforth verboten. This news has generated both anguish and delight in the EVE Online player community, depending on whether those players were fans of automated multi-boxing or not.

What this all really means

For those unfamiliar with the above terminology, or what it means, the only important thing to know is that this is a wholly unexpected move by CCP Games, if one assumes that their key metric for measuring success is PCU - peak concurrent users. Simply put, this is the number of players logged in to the EVE Online server at the same time, which has become the most commonly monitored indicator of player activity in the game.

Players can start multiple simultaneous client sessions, one for each of their accounts - this is known as "multiboxing", and can mean either multiple instances of the EVE Online client running on one computer, or on separate computers. I multibox frequently, usually running two or three character sessions on a single computer and using Alt-Tab to switch between sessions - or I'll sometimes run a couple different clients at the same time using both a laptop and a desktop PC. Multiboxing is very handy when running coordinated operations with multiple characters in the game at the same time. I often do this for mining fleets, with two characters flying exhumers and a third providing fleet support in an Orca.

CCP Games has always allowed multiboxing, and it is a common practice among EVE Online devotees. I find it makes the game more interesting, as it requires me to juggle my attention between different characters - this can make even routine PvE activities, like high-sec asteroid mining, a little more challenging. And it is certainly handy for Level IV missions or small fleet hauling runs.

Some people get really crazy about multiboxing in EVE Online. Like, really, really crazy.

Some people get really crazy about multiboxing in EVE Online. Like, really, really crazy.

While CCP Games allows multiboxing, it has always banned the use of automated macros or "bots" - programmed input of actions in the game without human involvement. I have personally observed more than a few obviously automated mining and mission-running bots. In one case, I saw someone gank a mining barge, and the pod continued to shuttle between an asteroid belt and a nearby station, over and over, for hours - clearly, a poorly programmed automated routine. Several of us who witnessed this behavior reported it to a GM, and we never saw that character in game again.

However, CCP Games has long ignored the use of automated input of simultaneous commands to multiple client instances, known as input broadcasting. This form of automation was very helpful to players flying fleets of ships piloted by their own characters, each running in a separate multiboxing instance. Input broadcasting software, such as ISBoxer, made this much easier to manage, and enabled individual players to perform complex combat maneuvers, such as massed stealth bomber attacks.

While many players decried the unfairness of automated input broadcasting, those who used ISBoxer and similar tools defended it by pointing out that such practices actually benefited CCP Games, since it increased the PCU metric of success. As a result, many ISBoxer users felt safe and secure that they would continue to enjoy the advantages that automated multiboxing provided them.

When you assume...

With CCP Falcon's announcement, CCP Games proved the input broadcasters wrong. Starting January 1st, 2015, the use of ISBoxer and similar automated tools are no longer welcome, and will receive the same harsh treatment as the use of macros and bots.

Soon after CCP Falcon's post, I saw numerous #tweetfleet posts on Twitter from ISBoxer users, who were shocked and upset that CCP Games had suddenly turned on their previously allowed practice. "It's dumb," one Tweet said. "Expect to see PCU drop fast."

Personally, I have never used ISBoxer, as I find automated broadcast input to be a disproportionately unfair advantage to those who use it. Yes, it does increase PCU, but it does so while providing unnatural and artificial abilities to individual players, by using tools to affect their real-time commands into the client.

I recently debated with one ISBoxer user, who tried to equate automated broadcast input tools to the use of other third-party applications, such as the EVE Central market database. His argument had little merit - EVE Central is not designed to input orders into the EVE Online client without human involvement, as ISBoxer clearly does.

ISBoxer users have a right to be somewhat confused, however, since it has taken so long for CCP Games to declare their practices as now bannable. However, their confusion is based on a false assumption - that CCP Games' chief metric of success is PCU. CCP Falcon's announcement, along with other CCP Games actions over the last year, indicates that this is no longer true - and this bodes extremely well for the health of EVE Online, and for the future size of its player community.

Be careful about how you measure success

In Real LIfe, I provide business consulting services to my clients. Recently, a manager called me with a problem. He had wanted to increase the number of successfully resolved customer service calls to his business. He had established a bonus system for employees who received such calls, providing a small incentive payment for each call logged as addressed and closed. But a month later, his customers' satisfaction ratings and repeat business had plummeted. Why? Because it didn't take long for employees to figure out that to maximize their earnings, they should finish each customer service call as quickly as possible. While the number of successfully resolved calls did increase quite dramatically, it did so at the expense of relationship-building, rapport and friendliness, and customers didn't like it.

Why do I tell you this story? It illustrates that over-emphasizing any one quantitative metric of success can lead to aberrant results. For many years, CCP Games focused too much attention on PCU. In fact, during the tenth anniversary of EVE Online, it promoted an event to drive PCU to record levels.

And when you first start the EVE Online launcher, what do you see first? The number of concurrent pilots on the server.

PCU is a useful metric of general player activity level, but it provides no insight into the quality of the experience of those players. Under CCP Seagull's leadership, the EVE Online development and community teams have begun to realize that positive changes in the player experience will produce the best results eventually, even if that means short-term setbacks in certain activity metrics, such as PCU.

At EVE Vegas last month, CCP Seagull said, "It's time to get over Incarna and start taking risks again." That expansion, released in the summer of 2011, was so poorly received that it resulted in dramatic reductions in player subscriptions and PCU counts. This eventually resulted in layoffs at CCP Games, and rattled the confidence of the entire company. After Incarna, devs focused almost solely on improving the flying of Internet spaceships, providing a series of solid, but relatively safe incremental expansions.

qualityvsquantity11.jpg

It is because of CCP Seagull's vision, more than any other factor, that EVE Online developers now feel emboldened enough to take risks, even if that means changing long-held assumptions about many game mechanics, such as the recent shake-up of force projection ranges.

As a result, the most recent and now also more frequently released expansions have focused primarily on improving the quality of the player experience above all other considerations. And this emphasis on the quality of player experience first, over any quantitative success metrics, is now paying off. EVE Online is certainly a better game today than it was a year ago - it has more of a sense of momentum, direction, excitement and fun - more heart - than ever before.

We in the player community may continue to watch the daily PCU counts and wonder if "EVE is dying", but our metrics don't matter. The only thing that really counts is whether CCP Games can remain courageous and dedicated enough to make the tough choices and implement the changes that help EVE Online to improve, prosper and grow. CCP Falcon's announcement is yet another recent indicator that this is indeed the case - and I couldn't be more impressed.

Fly safe! o7