Say Ha-Lua To My Little Friend

I’ve always had a soft spot for the scripting language Lua, for a number of reasons:

1. It’s lightweight
2. It’s fast
3. It’s super-easy to integrate into C/C++
4. It was used in Grim Fandango

So when I started tinkering with a new engine recently it was inevitable that I’d want to integrate Lua for entities in the game world. There’s a lot of info on the web about object-orientation in Lua, using metatables etc., and plenty of stuff about calling C functions from Lua, but I couldn’t find anything that covered what seemed to me to be quite a simple use case:

1. Entities in script can inspect and modify properties defined in C++
2. Properties should be accessible as properties, e.g. ‘self.position’, rather than through function calls
3. Properties shouldn’t be copied into Lua and back out every update (there might be thousands of entities and not all of them will want access to all properties)
4. Defining an entity script should be simple and as foolproof as possible
5. The integration shouldn’t be auto-generated, by Swig or similar

The method I’ve settled on (so far, the tinkering continues) involves creating a global table in Lua for an entity type (the class), with its __index and __newindex metamethods set to redirect to C++ for properties that aren’t found. Since this is boilerplate for every entity type it shouldn’t be included in an entity’s script itself. Instead I perform this setup in C++:

lua_newtable(L);
lua_setglobal(L, classNameString);
lua_getglobal(L, classNameString);
lua_getglobal(L, "lookup");
lua_setfield(L, -2, "__index");
lua_getglobal(L, "insert");
lua_setfield(L, -2, "__newindex");
lua_pop(L, 1);

Lua object-orientation tutorials tend to define a class:new(o) function that creates instances of a class by creating a new table and setting its metatable to be the class table. I didn’t want to have to copy and paste this boilerplate code for every entity type, but since it would only be called from C++ anyway, and the colon syntax is actually syntactic sugar for class.new(class, o) i.e. we are passing the class as the first parameter this all means we only need to define a single function to create all instances of all classes:

function newObject(self, o)
    o = o or {}
    setmetatable(o, self)
    return o
end

So now when we try to read a property on the entity that doesn’t exist, Lua will go via the metatable and call lookup. Similarly when we try to write a property, insert will be called. Both functions are defined in Lua:

function lookup(table, key)
    return rawget(getmetatable(table), key) or getEntityProperty(table, key)
end

function insert(table, key, value)
    if not setEntityProperty(table, key, value) then
        rawset(table, key, value)
    end
end

Lookup will try to find the property on the parent/class table (to allow for pseudo-‘static’ properties) and if it’s still not found there it’ll call getEntityProperty, a C function. Conversely, insert will first ask C if it’s responsible for the property via setEntityProperty and if that returns false it’ll set the property on the entity table in Lua.

The getEntityProperty and setEntityProperty functions are static, they know which entity the Lua script is part of because the entity table in Lua has an ‘addr’ property which is a light user data pointer to the entity in C++. An instance is created like this:

// push the entity pointer onto the stack (we'll use it later)
lua_pushlightuserdata(L, entity);
// call newObject(class)
lua_getglobal(L, "newObject");
lua_getglobal(L, classNameString);
lua_pcall(L, 1, 1, 0);
// set newly-created entity table's 'addr' member to the entity pointer
lua_pushstring(L, "addr");
lua_pushlightuserdata(L, entity);
lua_settable(L, -3);
// finally, set registry[entity pointer] = entity table
lua_settable(L, LUA_REGISTRYINDEX);

We store the Lua entity table in the registry, keyed on the entity pointer, so when the C++ entity wants to update it’s easy to find it’s Lua counterpart.

What all this means is in Lua I can define an entity script like this:

function cube:tick(dt)
    pos = self.position
    pos.x = ...
    pos.y = ...
    pos.z = ...
    self.position = position
end

And the position in C++ is updated transparently, without having to copy it into and back out of Lua.

That’s nice and simple, but it comes with a drawback that you have to cache C++ properties in Lua to avoid requesting them multiple times (which is an issue if the property is non-trivial). Initially I solved this by caching the property in Lua from the C++ side, so when a property was requested from C++ just before it was returned, I’d store it in a property on the entity table. This way any future reads would read from the Lua property and not redirect to C++, but of course any writes wouldn’t redirect to C++ either. I suspect I can probably solve it with an intermediate cache table between the class and entity tables; use the cache table to hold the properties so that the entity table will redirect to the cache table to read properties and to C++ to write them. The write would have to update the value in the cache table also. But that starts to feel a little overengineered… we’ll see!