Replacing your gearbox at 100 mph: How live games monitor and change with millions playing

  • Overview of analytics/telemetry in the gaming world; how it’s collected, processed, and used.
  • A lot of game studios use the Chrome profilerto analyze telemetry from games. This is possible because the profiler works off JSON input, and is not coupled to in-browser analysis. This is really cool!
  • Call of Duty asks if the user had fun at the end of every match; a ton of inference/analysis is probably not sufficient to answer this question, but just asking the user is a surprisingly reliable way to get this information. They receive 60-80% “non-skip” answers, and the options are randomized to prevent selection bias.
  • “ms per frame” or “frame budget” is a more robust contemporary metric than FPS.
  • If telemetry is collected processed fast enough, the system can respond to it in real time.
  • As of this talk, Steam peaks at 14M concurrent users.
  • MAU (monthly active users) is a common metric to judge usage/engagement. Numbers from Activison:
    • 300M MAU: Mobile games are in a different league to almost a ridiculous degree.
    • 70M MAU: Non-mobile games.
  • In addition to session length and session interval (time between sessions), another common metric is session depth (number of events in a session).
  • The switch can be used in offline situations; what happens if people want to play together but don’t have the same version of a game? The switch allows for P2P patching, so a leader (with the newest version of the game) is determined, and the leader pushes out updates to all other players.
  • Trajectory/heatmap analysis is very useful for tuning the gameplay itself, things like levelling, rewards, level design, etc.
  • Fortnite used the “a black hole appears and sucks up everything” strategy to work in a 2-day maintenance window.