Replacing your gearbox at 100 mph: How live games monitor and change with millions playing
https://learning.oreilly.com/videos/oreilly-velocity-conference/9781492050742/9781492050742-video328386
- Overview of analytics/telemetry in the gaming world; how it’s collected, processed, and used.
- A lot of game studios use the Chrome profilerto analyze telemetry from games. This is possible because the profiler works off JSON input, and is not coupled to in-browser analysis. This is really cool!
- Call of Duty asks if the user had fun at the end of every match; a ton of inference/analysis is probably not sufficient to answer this question, but just asking the user is a surprisingly reliable way to get this information. They receive 60-80% “non-skip” answers, and the options are randomized to prevent selection bias.
- “ms per frame” or “frame budget” is a more robust contemporary metric than FPS.
- If telemetry is collected processed fast enough, the system can respond to it in real time.
- As of this talk, Steam peaks at 14M concurrent users.
- MAU (monthly active users) is a common metric to judge usage/engagement. Numbers from Activison:
- 300M MAU: Mobile games are in a different league to almost a ridiculous degree.
- 70M MAU: Non-mobile games.
- In addition to session length and session interval (time between sessions), another common metric is session depth (number of events in a session).
- The switch can be used in offline situations; what happens if people want to play together but don’t have the same version of a game? The switch allows for P2P patching, so a leader (with the newest version of the game) is determined, and the leader pushes out updates to all other players.
- Trajectory/heatmap analysis is very useful for tuning the gameplay itself, things like levelling, rewards, level design, etc.
- Fortnite used the “a black hole appears and sucks up everything” strategy to work in a 2-day maintenance window.