Catalyst is a game about running a FIRST Robotics Competition team. You design a robot for the season’s game, build it, test it, take it to competitions, and come back to the workshop to make it better. It’s built in Unreal Engine 5 with high-end, realistic visuals, on top of a physics simulation I’ve spent a lot of time making trustworthy.
I started this Unreal Engine version in early 2024, when the Crescendo season was already well underway. Over the next year and a half, I built much of the framework you’ll see here, then stopped last September and only recently picked it back up. The original plan was to develop it in secret and make a big splash with a surprise announcement. But game development is hard, and I don’t know if this will ever ship as a full game. I was lucky enough to receive a license to publish Catalyst 2012 as an official FIRST video game on PC and Xbox 360, but whether that could happen again is unknown. I’d hate to keep it to myself forever, so from here on I’m sharing my progress publicly. Some stretches will move fast and some slow, but I’m excited to show what I have.
This post is meant to be a look at the project’s history, its honest (and very much in-development) current status, and where I’d like to take it should the time and space allow. Each of the technical pieces will get its own deep-dive post, as much a development blog as a portfolio piece. I’ve never been good at showing my in-development work. It’s a habit I’ve carried throughout my entire adult life, and by definition, I don’t have a lot to show for it. I’ve kept this project quiet for two and a half years (with a year of no development)! At this rate, nobody will ever see it and it will die with me! The only way that can be avoided is if I break this habit. So, please, bear with me.
Why build this#
I’ve never considered Catalyst a driving sim. Those are useful and they have their place, and simulation is at Catalyst’s core, for sure. But for years, I’ve been captivated by the impact possible from a high-quality, fun, and approachable FRC video game. And when I say “captivated,” I mean it.
It’s been 15 and a half years since I spent every day of my freshman spring break barricaded in my dorm room, feverishly exploring the BEPU physics engine and how I could build rigid bodies and connect them with joints and then drive those joints with motors and… oh my God, I have a robot! A few months later my friends and I were emailing with FIRST leadership, signing NDAs, learning the next year’s game in the middle of June, and tasked with building software for the Kit of parts that could help every team in FRC strategize about how they would approach competing in that year’s game.
A few years later, my friends and I were preparing to graduate college and felt we needed to focus on graduating well and finding jobs. So, we ended our agreement with FIRST and moved on. Moved on… Well… I tried. I really did. But, with every Unity and Unreal update, I’d think, “Wow, I could use that new feature to build a new robotics game,” or I’d see tutorials and announcements from new technologies like Substance and Houdini and think, “That’s it! That’s how I could have solved such and such problems with the Catalyst game all those years ago!” During the Catalyst 2012 and 2013 games, I had such a clear vision for what I wanted to build, and never could shake it.
This project is me giving in. I’m compelled to make it. Whether it ever gets released is anyone’s guess. I didn’t spend my career building skills in software engineering, machine learning, design, content pipelines, and technical art just to make this one game, but, somehow, they all point right at it.
Origins#
Catalyst started in 2012 at Diamond Bullet Studios, a studio I co-founded with my friends in high school. Catalyst 2012 was an XNA game for PC and Xbox 360 built on BEPU physics. It was even a part of the 2012 FRC Kit of Parts and available to all teams for free. That version and its history will get their own post. The project you’re reading about here is a ground-up rebuild in Unreal, not a port.
Catalyst 2012
Catalyst (UE5)The vision#
You’d create a team and start its rookie season. Each season brings a new game, and you design a robot for it from mix-and-match parts: drivetrain, intake, shooter or manipulator, and more. During this build season, you’d be moving between a career-center-style classroom and a practice area cobbled together from overturned tables.
Then you take it to a fully simulated regional:
- Ten qualification matches
- A pit for small edits and repairs between matches
- Scouting other teams’ robots, which are generated from the same configurable parts you build your robot from
- Off-screen matches, rankings, a simulated alliance selection, playoffs, and rewards
Between events, you go back to the workshop and change what you can (within reason) before the next regional and eventually a world championship. Other teams’ drivers get more experience and their robots become more resilient as the season goes on. When your season ends, another one starts with a new FRC game and your team’s history keeps growing.
I have a lot of exciting ideas for how progression works. Right now, though, the priority is building a technical foundation, so that gameplay tuning has something solid to sit on.
Where we stand today#
Robots built from configurable parts#
The goal is for you to build your robot from individual parts, not pick from a lineup of prebuilt robots. That’s what lets you think strategically about how to play a given season’s game, and it’s what makes the robot feel like yours.
The test parts that exist today are at very different levels of completion and quality. They’re there just to prove the mechanics of swapping parts.
AI and art#
I’m not an artist. I can barely draw a smiley face or model a table. I am a technical artist. I can use math and data to make some things, but not all. For example, the mechanical content is created with procedural modeling tools I build in Houdini. The project doesn’t, and never will, contain AI generated images or meshes. I may, eventually, use AI automation to drive the tools I build that create content… and, yes, that does get confusing, I know. I’ll do a post specifically on that fine line and my philosophy. The bottom line is that no artist will lose a sale or an opportunity to AI because of this project. What strange times we live in…
The arena and the workshop are currently modified versions of two Dekogon Studios environments from Fab: the High School Basketball Gym and the 90’s High School Science Lab Classroom. They’re not where I want them long term, and the goal isn’t to flip store assets. They’re enough to reach a visual bar for this stage of development and to communicate where the game is going. More arenas, more workshops, and more unique assets are all part of the plan, just not yet.
Arena
WorkshopPhysics that Unreal doesn’t own#
The robots don’t use Unreal’s built-in physics. A networked physics game needs to control its own tick rate, save and restore the entire world, and rewind and resimulate exactly. Chaos didn’t give me that control, so physics runs as its own engine, with Unreal drawing the result.
Three layers, and a game you can run without the game#
The code is being split into three tiers. Anything that changes the outcome of a match belongs in the bottom one.
Unreal Blueprints hold meshes, splines, and numbers.
The look and the tuning values live where a designer can reach them.
Unreal C++ is glue.
Actors connect the engine to the simulation and do very little else.
The simulation core has no Unreal in it at all.
Physics, controls, intake, and shooter logic compile and run on their own.
That core runs in a standalone program that builds in seconds and replays recorded game sessions byte for byte. That’s true on Bullet and on Jolt, across two compilers and two standard libraries. Getting there meant tracking down every place two “identical” simulations differed, down to a single bit in one wheel’s target speed.
Rollback networking#
Option A: Server-only simulation
One of the simplest ways to network a physics game is to run the simulation only on the server. Each player sends their inputs, the server simulates, and everyone gets the result back. It’s easy to build, the server is always right, and nobody can disagree about what happened. The catch is that every input waits a full round trip before you see your robot respond. At 100 ms of latency your robot reacts a tenth of a second late, every time, and players with low latency get a real advantage over everyone else. It only works well when everyone has very low latency.
Option B: Prediction and rollback
Catalyst uses client-side prediction with server rollback instead. Your machine simulates immediately, so your robot responds the moment you move the stick. The server stays authoritative, and when your machine disagrees with it, the client rewinds to the last state they agreed on and resimulates forward with the corrected information. It’s the same approach Rocket League uses for its cars and ball.
For option B to work, the physics has to be able to save and restore the entire world, resimulating has to give exactly the same answer every time, and rewinds cost CPU time. You’re also always guessing what other players are doing, so when a guess is wrong, their robots visibly correct. Robot-on-robot pushing is the hardest case, because both machines are predicting the same collision from different information.
Jolt’s exact world snapshots make those rewinds bit-identical. The rebuilt netcode adds a frame clock, inputs tracked by frame, and prediction of other players’ robots.
Trained robot drivers#
Robots can now be driven by trained neural network policies. They drive to targets, collect game pieces, avoid obstacles and each other, and play six-robot matches. The policies feed their output into the same input path a human player uses, through a hand-written forward pass with no ML runtime in the game.
Training on Isaac Lab’s PhysX and then dropping the policy into the game exposed a real sim-to-sim gap. It scored 22 destinations a minute in Isaac Lab and 13 in the game, with a third of the robots stuck. About 200 more training iterations on the game’s own physics brought it to 20, with almost nothing stuck.
A full run currently takes about 20 minutes. The policies are not very complicated right now and there’s no environment or dynamics randomization yet, so while 20 minutes doesn’t sound too bad, it’s actually a warning shot for me. I have some ideas for improving iterative training runs that I’ll share and explore in a future post.
The physics worlds in OpenUSD#
The field, game pieces, and robot are also exported to OpenUSD with physics. They read back into the exact same simulation, bit for bit, and can be replayed in PhysX and Newton to compare engines. Parity there is partial, and that comparison will get an honest post of its own.
Current state#
What (mostly) works:
- One robot config, one field, one workshop, one game
- Driving, collecting, and shooting
- Exact replay across the game and the standalone runner
- The networked physics framework
- Jolt integration in UE5
- The ML training framework and in-game inference
What doesn’t yet:
- Match flow: scoring, timers, and match states
- The netcode has only been tested on one machine
- Robots aren’t trained to score yet
- Intake and shooter state aren’t part of network rollback yet
Tech stack
- Unreal Engine 5
- C++
- Jolt Physics
- Bullet
- Houdini
- Isaac Lab
- OpenUSD
- PyTorch
Where this blog is headed#
I plan on writing deep dives into almost all of the systems I have and am still building. Some will be technical pieces explaining a problem and my solutions for them, and other posts will be my active development work, mistakes and all!
- Two physics engines?
- Syncing physics across a network
- Merging real machine learning with gameplay
- Procedurally building assets in Houdini
- And many more shenanigans.