Supporting distributed real-time debugging in online games

In these last few years we are witnessing a tremendous change in the way video games are developed. On the one hand, large development teams with a multi-layered organisation are employed. On the other hand, we see an increasing request for online services and functionalities. Combining these two trends together usually results in large projects involving parallel and distributed systems. Despite the adoption of team-oriented source code and asset repository managers, code testing and debugging is still left to human direct management. In particular, distributed debugging is a complex problem due to the synchronisation required between network nodes to correctly reconstruct the sequence of events leading to a malfunctioning feature. To solve the aforementioned problem, we designed and implemented DREAD: an architecture to support distributed debugging in real-time games. When adopting our architecture, developers will record events on edge systems, collect them on a centralised sync, and then consolidate everything in a synchronous way. Following this approach, it will be easily possible to detect unexpected software behaviours and trace back to their causes.