For the past several weeks I've been working on a sort of embedded domain-specific language project. It's designed to help create simple event-trigger based missions and general game content. The details of the language itself are more or less irrelevant; they were already very well hammered out by another guy on the team before things got handed over to me.
What I've been doing lately is working on the back-end that actually executes the DSL code. Originally, the DSL was implemented on top of another DSL, which ran on an interpreter implemented in C. It's a testament to the skills of our senior programmers that the entire system performed as well as it did, both in terms of CPU time and stability.
However, for a number of reasons, the mid-level DSL is going away in the future, in favor of a purely C++ implementation of game logic. This means that the top-level DSL - the one that we use to create game content - has to be reimplemented in C++. A large part of this was already done by a couple of other guys, one of whom was the original author of the system.
The port
For the most part, the port was a success; it lacked some features, but that was primarily because the corresponding game logic hasn't been implemented yet, so the DSL had nothing to hook into. The overall principal mechanics of the system were in place, though, and working great.
Unfortunately, the port was more or less a straight copy of the code as it had been built in the mid-level DSL. That language had a number of critical differences: no floating point numerics, full garbage collection support, easy string manipulation, and weak object/class support. Compared to C++, it's a radically different beast.
The upshot of this is that the system basically relied entirely on string parsing. Behind the scenes, the type system was basically "everything is a string." This worked fine in the original implementation (largely because there wasn't really a better option), but in C++, it meant we were doing a huge amount of string munging for minimal benefit.
And now things get ugly
We decided to support floats in the DSL, as well as a few other types that could help reduce/eliminate errors for the content authors. This meant that any time we wanted to evaluate a string to see what it meant in the DSL, we had to know what type it was: integer, float, ID of a game object, and so on.
There were two basic options here. The easy out was to force the content authors to always specify what type of data they were working with. If the string "45.6" was supposed to be a seconds value, then they had to say so. If it was meant to be a number of kilometres, they had to indicate that instead.
This meant that suddenly a very friendly and accessible DSL was polluted by a lot of explicit type declarations. This cruft was really annoying, and after taking a hard look at the implications, we decided to try and eliminate it.
Making the ugly into something beautiful
We already had a good variant class implemented in the DSL's back end, so the obvious solution was to make the DSL essentially typeless from the author's point of view, and just do some intelligent type coercion behind the scenes. Any time an invalid result occurred, we could also flag it as an error and report it to the user.
This required the construction of an entirely new parser for the DSL, because the old one only understood strings. The new parser produces a syntax tree which can then be evaluated very cheaply, versus the old strings which cost a fair amount of CPU time to evaluate. Considering the fact that we may be dealing with hundreds if not thousands of evaluations per frame, this is a considerable bonus.
The downside is that a lot of logic was built to execute the DSL using... you guessed it... the old strings-based method. So scattered all through the DSL engine are countless string munging calls. A lot of the system was built rather ad hoc, so it's not always easy to locate these calls.
As a matter of fact, I'm in the middle of converting from the old to new parser in the DSL interpreter; it's tedious and mind-numbing work, which is why I'm procrastinating and writing about it instead of doing it.
Takeaway lessons
So here's what we've learned for the future:
- Stay consistent to your DSL's purpose. If you want to make life easy and simple for the DSL authors, carefully consider any decision that increases the language's complexity. In my case, I didn't carefully consider the implications of explicit typing, and thus did a lot of duplicate work that is now wasted.
- Plan twice, implement once. Any DSL implementation - and especially an ad hoc one - is going to take a lot of effort to rewrite when you're working with a substrate like C++. In this scenario, try not to change horses in mid-stream. Plan out exactly what the implementation should look like, and save yourself a lot of time rewriting and tweaking things.
- Ideally, don't code a DSL on a substrate like C++. Use something like Scheme or OCaml instead, because building languages on top of C++ is like having to build your own log cabin with nothing but a dull piece of flint.
So that's what I've been doing. And now, we return you to your regularly scheduled boredom - because life after The Bag of Holding just isn't worth living.
