The number of Pokémon Go players who downloaded the app upon its launch was 50 times higher than expected by Niantic Lab, its developer.
Players suffered numerous crashes when the popular Pokémon-hunting game came out in Australia and New Zealand back in July, followed by similar problems in the UK and US shortly after.
But this is easily explained by the sheer popularity of Pokémon Go, on a scale that Niantic just wasn’t expecting, according to its hosting provider, Google Cloud.
People downloading its app quickly outpaced Niantic’s expectations by 50 times, and was 10 times higher than its worst case scenario.
“Within 15 minutes of launching in Australia and New Zealand, player traffic surged well past Niantic’s expectations. This was the first indication to Niantic’s product and engineering teams that they had something truly special on their hands,” said Luke Stone, Director of Customer Reliability Engineering (CRE), in a blog post yesterday.
Niantic was the first customer of CRE, which is a new Google initiative that floods customer teams with Google’s technical staff to help ramp up capacity for critical cloud applications.
Pokémon Go’s developers duly called in CRE to help prepare for the US launch due the following day, as it went on to become the most downloaded mobile game ever within its first month, and took just 20 days to make $100 million.
The app used a dozen Google Cloud services, but the infrastructure provider’s NoSQL database, Cloud Datastore, was front and centre as the game’s primary database.
Stone wrote: “The teams targeted 1x player traffic, with a worst-case estimate of roughly 5x this target. Pokémon Go’s popularity quickly surged player traffic to 50x the initial target, ten times the worst-case estimate. In response, Google CRE seamlessly provisioned extra capacity on behalf of Niantic to stay well ahead of their record-setting growth.”
That may not be most early players’ memory of the game, with plenty greeted with crash notifications or the game simply refusing to open.
Stone admitted: “Not everything was smooth sailing at launch! When issues emerged around the game’s stability, Niantic and Google engineers braved each problem in sequence, working quickly to create and deploy solutions.”
Niantic and the CRE team worked closely together to review the game’s architecture in light of the scaling problems.
Pokémon Go’s application logic (which dictates how domain and infrastructure are configured to meet the app’s specific demands) is hosted in Google Container Engine (GKE), powered by the container orchestration software Kubernetes.
This gave Niantic’s game “planetary-scale” for its millions of users, allowing developers to focus on deploying live changes for players.
But ahead of the game’s launch in Japan at the end of July, Niantic and Google’s CRE team had learned their lesson from the game’s earlier launches. They decided to upgrade to a newer version of GKE to add a thousand new nodes to prepare for the anticipated demand.
“Akin to swapping out the plane’s engine in-flight, careful measures were taken to avoid disrupting existing players, cutting over to the new version while millions of new players signed up and joined the Pokémon game world,” said Stone.
At the same time, the teams upgraded to an HTTP/S Load Balancer to give players faster connections to the game.
It all paid off when Pokémon Go launched without incident in Japan despite three times the number of users downloading it compared to the US launch a fortnight earlier.
“Niantic’s Pokémon GO was an all-hands-on-deck launch that required quick and highly informed decisions across more than a half-dozen teams,” said Stone. “On behalf of the Google CRE team, I can say it was a rare pleasure to be part of such a memorable product launch that created joy for so many people around the world.”
