Bungie knew about the Loot Cave, it just didn’t think you’d care about it

Destiny‘s famous loot cave was no secret to its creators when the game was released last year. Developer Bungie just didn’t think it was a big deal, user research lead John Hopson revealed during a Game Developers Conference 2015 session this week. Players proved otherwise almost immediately. The problem was math. Or at least a calculation that the game’s inventors ran differently than its players.

Hopson runs a unit within Bungie that studies its own products. The user research team directs a large part of its effort to figuring out how players spend their time in Bungie’s games. And part of their goal is to figure out how Bungie can get its players to do what Bungie wants.

But as the Loot Cave proved, even years of planning can’t mitigate unintended consequences.

OMNIVORES WELCOME

Using the knowledge the studio gained by making five Halo games in 10 years, Bungie understood that players fall into several categories. Some spend most of their time in the campaign and dabble in multiplayer. Others spend hundreds of hours in multiplayer and barely touch the campaign. A third section — and one with many subtypes — captures those who spend something like 90 percent of their time playing a very specific part of the game, like capture the flag or custom games in multiplayer.

In its Destiny research, Bungie was particularly interested in a specific type of player. Hopson calls them omnivores. They don’t gravitate toward any one corner of a game. Instead, they play a variety of ways, from campaign to multiplayer, single player to co-op.

“An activity for every mood.”

Bungie’s interest in its old games was more than academic. It had a purpose: Destiny would have many ways to play, and Bungie wanted to foster nimble players who wanted to experience each. To do so, the games maker would provide incentives for hopping between everything. Destiny, if Bungie succeeded, would be a game where players chose to do many things with many characters. And the user research group would study the game as it was being created to figure out ways to make the desire a reality.

“There’s always been an idea for Destiny that there would be a variety of activities,” Hopson said. “One of the essential design pillars was, ‘An activity for every mood.’ That we would have intense competitive activities and very calm activities, story missions that you were guaranteed to win if you spent enough time and raids that were intended to be deliberately very hard.”

Bungie, in short, wanted to create omnivores. And the user research group was there to help them craft the best ways to play. But predicting the future is a difficult proposition, as the Loot Cave would prove.

SLOPING FOR THE FLOOR

The theory was that diverse players are robust players. If someone gets bored with a game that doesn’t offer much, they can leave. But in Destiny, if a player gets bored with multiplayer, they can do a raid or a story mission or old-school multiplayer. If there are many things to do, they’ll keep playing.

“There have been occasions where players wandered off the path.”

So, in Hopson’s words, Bungie decided to “slope the floor,” to give incentives to play more than one way. Bungie didn’t force it, he was quick to point out, but it strongly encouraged it. That’s why, for example, Destiny‘s bad guys drop items that aren’t for your class. If a Warlock gets a Titan drop, that could encourage him to experiment with playing a Titan.

There’s only so much a developer can do, short of forcing players’ hands. Bungie could create a path, and they could incentivize a diversity of play styles, but no matter how extensive the user research team’s testing was, releasing a game to millions of players is a process that brings surprises.

And that’s where the Loot Cave comes in, a shining example of unintended consequences and Bungie’s inability to predict player behavior perfectly.

“There have been occasions where players wandered off the path that we thought they were going to take, probably the most famous being the Loot Cave,” Hopson said.

UNINTENDED CONSEQUENCES

In Destiny, enemy appearance is typically seamless. Players trudge into an area and meet gun-toting alien hordes who feel like they’ve alway been there. Of course, that’s an illusion, but it’s well-realized and helps the game feel alive and its worlds lived-in.

But a few weeks after Destiny‘s release, someone discovered a cave out of which, every few seconds, an enemy would wander. Players realized that, if they stood at a distance, they could pick off an endless succession of bad guys. And so they did, over and over. They could do it for hours, and they did. The calculation was simple: In Destiny, bad guys drop loot, and this endless conga line of enemies requires almost no effort.

After shooting dozens upon dozens in the Loot Cave, players ran in and pillaged piles of booty.

The user research team was aware of the Loot Cave’s spawning properties from its pre-release studies. To Bungie, it wasn’t a big deal. The developers believed that, even if players discovered the Loot Cave, they wouldn’t waste their time there because the enemies were low-level and unlikely to drop anything of significant value.

Bungie was wrong — not about the loot drops, but about how players would value them.

“The funny thing is, we knew about this before launch,” he said. “We knew this was a potentially exploitable activity. But we didn’t care, because we said the actual drop rate per minute spent is not any different than anything else. So you actually will get less loot doing this per hour than just playing the game.

“They weren’t doing the loot per hour, they were looking at loot per effort.”

“But the players weren’t actually doing the math that way. They were doing the math of — they weren’t doing the loot per hour, they were looking at loot per effort. And this was so low effort that it felt like free loot to them.”

The Loot Cave’s discovery and exploitation spread like the Hive across the galaxy and fractured Destiny‘s player base. It coincided with a surge in reports for cheating. “The time of the Loot Cave was the highest peek for players of Destiny,” Hopson said.

Some reported players because they were using the Loot Cave to get drops. Others tattled because they were preventing the loot cave from being used.

While players squabbled and debated the validity of exploiting the Loot Cave, the question now, for Bungie, was what to do about it.

COURSE CORRECTING

Bungie’s misjudgment about loot’s inherent magnetism coupled with the ease with which players can earn it allowed the Loot Cave to ship with the game. Even if the developer was logical in thinking that players wouldn’t exploit it, Bungie learned otherwise. As the Loot Cave became a bona fide phenomenon, the user research team was there to figure out what happened and fix it. Bungie reacted.

According to one of Hopson’s slides, the user research team started working on Destiny 47 months — almost four years — before the game shipped. That team helped shape the way people played the game, based on its creator’s desires. And and the team wasn’t stopping just because Destiny had been released.

Loot wasn’t something to be mined with ease, in Bungie’s view. It was something to be earned and rewarded with in battle. So, not long after the Loot Cave’s discovery, Bungie patched it out of the game. Not long after, the developer also discovered a few other “minor loot caves” with algorithms and snuffed those out, too.

Unintended consequences aside, the Loot Cave is precisely the kind of thing that the user research team is there to understand and react to.

“We’re constantly doing this pretty much throughout the game,” Hopson said, “looking for moments where players are beating things very quickly, taking too long, taking too short, looking for places where the experience isn’t what we expected. So we can then investigate and find out what’s going on.”