Replacing your gearbox at 100 mph: How live games monitor and change with millions playing

https://learning.oreilly.com/videos/oreilly-velocity-conference/9781492050742/9781492050742-video328386

• Overview of analytics/telemetry in the gaming world; how it’s collected, processed, and used.
• A lot of game studios use the Chrome profilerto analyze telemetry from games. This is possible because the profiler works off JSON input, and is not coupled to in-browser analysis. This is really cool!
• Call of Duty asks if the user had fun at the end of every match; a ton of inference/analysis is probably not sufficient to answer this question, but just asking the user is a surprisingly reliable way to get this information. They receive 60-80% “non-skip” answers, and the options are randomized to prevent selection bias.
• “ms per frame” or “frame budget” is a more robust contemporary metric than FPS.
• If telemetry is collected processed fast enough, the system can respond to it in real time.
• As of this talk, Steam peaks at 14M concurrent users.
• MAU (monthly active users) is a common metric to judge usage/engagement. Numbers from Activison:
  • 300M MAU: Mobile games are in a different league to almost a ridiculous degree.
  • 70M MAU: Non-mobile games.
• In addition to session length and session interval (time between sessions), another common metric is session depth (number of events in a session).
• The switch can be used in offline situations; what happens if people want to play together but don’t have the same version of a game? The switch allows for P2P patching, so a leader (with the newest version of the game) is determined, and the leader pushes out updates to all other players.
• Trajectory/heatmap analysis is very useful for tuning the gameplay itself, things like levelling, rewards, level design, etc.
• Fortnite used the “a black hole appears and sucks up everything” strategy to work in a 2-day maintenance window.