25,000+ resources across deployments, 65,000+ lines of Python, a team of two.
You've built a platform to deploy things, offer services, and power your business. But how do you manage the platform itself? Can you make the boring parts truly set-and-forget: patch management, progressive rollout, compliance? At some point you think you need a platform for your platform, and then you realise you already have one.
This talk is the story of developing x-RD's infrastructure automation codebase (codenamed skywater) that runs the platform for our flagship product (secd3v), internal projects, and internal infrastructure, including the development and CI environments that skywater itself is built in. It's also the story of the hair-pulling complexities we built ourselves into and had to build ourselves out of.
x-RD’s secd3v product hosts isolated services for customer use, such as GitLab, GitLab Runners, and AI services including Claude Code Service and MCP tooling. Deployed on AWS using the skywater automation codebase, which is built with Pulumi and, of course, Python.
Pulumi was chosen specifically because it meant writing real Python, not HCL or YAML. Early prototyping showed that problems like orchestrating cross-account AWS resources with complex dependencies were more naturally solved in a general-purpose language, and the dependency management would only grow more complex from there.
From the start, infrastructure was treated as software with a commitment to an automation-first approach. The scope goes well beyond production infrastructure: compliance guardrails, observability, networking, and the CI and development environments used to build skywater are all part of what it manages. This talk explains what that philosophy looks like in practice, and why 25,000+ managed resources isn’t a sign of sprawl, but the natural result.
That investment paid off: the team handling skywater development and operations across all environments is a permanent team of two. The platform is used to maintain itself, with other engineers brought on as needed to support bursts in development related to the platform, products built on top of it, and customer support.
But early assumptions didn’t survive growth. The monolithic stack that worked at first eventually hit a wall, and had to be migrated incrementally to a modular architecture. This talk digs into the specific problems that had to be solved: how do you split a live Pulumi stack without reprovisioning thousands of resources? How do you refactor a class hierarchy that production depends on? The solutions involved moving resources between stacks at the state level while refactoring in backward-compatible ways, and using cooperative inheritance (Python’s method resolution order used to bridge old and new class hierarchies) to refactor incrementally while production kept running.
If starting from scratch today, the team would do things differently: architecting for modular stacks from the start rather than unwinding tight coupling later, and adopting well-known software patterns like dependency injection sooner, before the codebase accumulated complex internal machinery that is now being replaced with cleaner approaches. Moving fast enabled delivery, but it also left behind decisions that are still being unwound.
Through real code examples and architecture decisions traced across a five-year timeline, the audience will walk away with patterns and lessons applicable to any team managing infrastructure as code, whether at this scale or just starting out.
The core message: automation-first compounds over time, for better and for worse.
Alisdair likes fixing problems, optimising systems, and automating the boring stuff. Working for x-RD gives him plenty of opportunity to make problems for himself. He maintains that writing code was never the hard part — the friction stopped you building the wrong thing twice.
For the last five years Alisdair’s life has (mostly) revolved around this talk’s subject matter. He has a B.Eng in Network & Software Engineering (First Class Honours) and over a decade of immersion in the world of DevOps and platform engineering.
Despite being bitten by many a Python (language versions and species), he still enjoys working with them. He automates everything he can, which is why his electric blankets are voice controlled. Canberra based, currently planning a motorcycle trip around Tasmania.
