DT: Building Overhaul

Hey everyone, A22 is finally on Steam Stable, so let’s switch our focus this week to one of the team’s long-term projects: fixing building, from the fundamentals. At Pax East last March, while watching you guys struggle through the building process in personal, Chris (finally? it was a long time coming) flipped a metaphorical table over building’s general fragility and horrible UX. He said, and I quote: “I will not go one more PAX having to apologize constantly for this (insert colorful word) system.” So there! In the background, on a separate branch, he nuked the original building infrastructure and started over with everything we’ve learned over the last few years. This was incredibly dangerous AND ambitious, given the years of fixes poured into the existing building system, but in the last three months he’s made incredible progress. Let’s check in on how he’s doing!

Warning: this is a long-term project that affects all of building. We will not put this in the game until it’s much, much more stable than it currently is. In a completely unrelated side-note, we will not be attending PAX west this year.

Building Overhaul

Also, if you haven’t yet taken the Desktop Tuesday survey, you can find it here:

Recap from the video:

We’ve known since Alpha 1 had no roofs that creating a system that allows AI agents to build any 3-D structure you can design would be a monumental challenge–a challenge that I believe no video game has ever solved. Over the last three years, Tony and Chris have stretched and patched and strengthened Tony’s original system to the very farthest edge of it’s capabilities, but we know from your many, many steam posts that you’re all still discovering buildings that won’t build.

For example, we’ve created a template we call “the hearthling trap.” Due to the way that the dependencies between the walls and floors of this template are constructed, hearthlings simply cannot build it in our current system.

A truth in software design is that if a problem has persisted for this long, it means that though it could be addressed with more patches and fixes, it’s no longer actually a bug–ie, an error, or omissions in our code. It means it’s probably a systemic issue with the base design–one that needs dramatic structural or conceptual changes to address. At PAX East last March, while watching the thousandth person struggle through the building process, Chris finally asked: if we took everything we’ve learned about building and started over from scratch, what would a better, cleaner, less fragile version of building be? After four solid months of lonesome toil, he’s finally got a solution that we think is pretty promising.

The reason we’ve waited so long to pull the trigger on this is that rebuilding an existing system is horribly expensive, in terms of time and effort. Even if you have a great design, you have to account for hundreds of bugs that were fixed in the old system, that may reappear in the new. Why was the old building infrastructure so fundamentally broken that we need to create a new infrastructure to replace it?

First of all, we had the wrong underlying model for how buildings should exist in the world. When you think of a real building, you think of walls, and floors, and roofs, because those are concepts that make intuitive sense to us as humans. We can use our squishy brains to easily navigate the ambiguity of, say, a roof-top patio. Is that a roof, or a floor? Well, it’s both, depending on how we want to look at it. Easy! This model also allowed a 1:1 mapping with our UI, where walls and floors are treated differently by the authoring tools. But for a software model of a building, that kind of messy ambiguity creates major problems. This is one of the reasons why, for example, you can place windows in a wall, but not in some slab that looks _exactly_ like a wall. They can be identical-looking, but the representation in Stonehearth is with completely different kinds of things, and so this lead to a difficult player experience where buildings don’t work the way you want. We want a new system does away with all that–one in which everything, conceptually, is remembered and treated as voxels, so that if you want to place a window in the roof, you can absolutely do something like that.

But don’t we lose the benefits of a 1:1 mapping between the UI and the building model? Yes, but actually, it turned out that the close mapping between the UI and the building structure became more of a liability than a benefit because any change to the UI (for example, to fix bugs with undo) risks the entire building system. (Let me say that again, for emphasis. In the existing system, fixing a bug related to undo may cause completely unrelated buildings to not build. This is about as awful as an engineering problem can get, and why our building UI is so buggy, and why nobody but Chris was brave enough to even touch it.) The new building system cleanly divides the editor from the underlying building model, and stores the underlying model as voxels, meaning that building pieces are all treated and built the same by hearthlings.

The second reason the old model was fundamentally broken was because to make sure we can build things, we pre-calculate the plan to construct the building, but our calculations are hit-and-miss: sometimes the algorithm works and sometimes it doesn’t–even for the same building! To properly understand how to construct a building, you need topological information for the building: what parts of the building are connected to what other parts, which parts can be accessed by hearthlings, stuff like that. Without this crucial information, we basically have to fake some parts of building; if you watch closely, some building parts will “pop” into existence while hearthlings are working. This is because we couldn’t be absolutely sure that hearthlings would be able to build that structure; the system just isn’t smart enough to figure that out. Chris’s new system does not do any of this cheating: we use our nifty topology/reachability service, which I talked about earlier this year, to ensure a correct build ordering, every time.

With all these issues, and all the patches we’ve put in to try to fix them, including adding the dependency analysis model in 2016 on top of the original naive “build everything with as much scaffolding as you can”, the building system as it exists in Stonehearth right now is incredibly intricate and fragile. It’s very hard for the algorithm to do important things, like tearing down scaffolding that’s in the way of future structures. The functionality may be in there, and working, but it was brutal getting it right, andsmall changes to the code can break things in subtle ways.) Because of these subtle errors, trying to fix any bugs in the current system is terrifying.

For example, in the video above, you can see the hearthlings are building Kythandra’s basilica. This building worked in Fall of 2016, after our first big revisions to building. Months later, while fixing an unrelated bug, it broke again, showing how fragile the system was, and how the original success was in fact, the result of an error. In the new system, the basilica builds again, hopefully permanently.

The new system is built with all of our improvements baked into the fundamental model. As a result, it’s less than half the lines of code of the current system, but far simpler and more maintainable. It supports many more kinds of buildings that _cannot_ be built in Stonehearth right now, computes building plans ten times faster than the current system, and uses way less scaffolding. We can actually create nice heuristics for figuring out where to place scaffolding, so as to minimize usage, in a way that is impossible to do in the current system. Lastly, the building plan is deterministic, so if a building doesn’t build for you, it’s 100% guaranteed to not build for us as well, helping us fix what problems remain.

So with all these improvements, what else is needed before we can see this new building system in game? Well, there are a few obvious things, like getting fixtures doors and windows working again, which Justin is currently tackling, but the real blocker is the existing building editor. It’s super crufty, difficult to use, buggy, and unpolished, even in it’s current third iteration. Hooking it up to the new building UI would be a ton of work. What our team really wants to do is design a brand new editor–something that’s fun to use, that rewards experimentation, that is forgiving of mistakes, and allows the flexibility to do all kinds of things. Nikki and Chris are in the middle of prototyping a brand new editor, and though we’re very excited about their progress, it it’s going to be a while before we have something we’re ready to commit to creating. We know building is really important to all of you, though, so we’ll keep you up to date as soon as we have things we’re ready to show and get feedback about. Also, if there’s a building template you’d like to test our new system against, stick it on and give Chris a holler via @not_own_wilson.

Other Announcements

Many thanks to all of you who have filled out or Desktop Tuesday survey. If you haven’t yet had a chance, you can find a link here.

Schedule updates: Our twitch stream this thursday will be moved up one hour to 5:00pm PST on Streams after that will continue as usual.

We will miss Desktop Tuesday next week, due to a team trip to EVO. Destkop Tuesday will be back on Tuesday, July 25th.