Mac Unreal 5.2 Quixel Bridge Issues

Won’t run, and crashes the editor

If trying to run Bridge from UE5 on Mac results in a) Bridge not starting and b) UE5 crashing, it’s the same issue Unreal 5 has always had: the node-bifrost.app included in UE5 doesn’t run in contemporary versions of OSX. The Mac used to popup an error dialog saying “node-bifrost is damaged and can’t be opened” but now (possibly since the recent OSX update) it no longer even does that, and either way Unreal copes with it not running by crashing to the desktop.

Previously you could just google the error message to find the solution, but if you haven’t seen it before (or like me, you just couldn’t remember it) you don’t have a lot to go on. I found the name of the .app in the debugger, which was enough to remind me.

Anyway – the solution is simple: Quixel have uploaded a newer version of the .app which you can download from this thread.

Update: Quixel have closed their forums so the thread is no longer accessible. You can see it via google cache, but in case the .app is also deleted later I’ve posted a copy here – replace the one in UnrealEngine/Engine/Plugins/Bridge/ThirdParty/Mac with it, then right-click and select ‘Open’ to let OSX know it’s safe.

Add button doesn’t do anything

Short version: if you download an asset, hit ‘add’, and nothing happens… just drag and drop the asset into the editor instead.

I have no idea what the issue is here. There is a lot of talk about very similar issues on the web, mostly around the fact that the Bridge you can download from Quixel is capable of exporting to multiple other apps, where the one built into UE5 can (not unreasonably) only export to UE5, and if you have left over config files from a full install trying to direct to a non-UE5 app that will cause problems… but that wasn’t the case for me. I’ve never used Bridge outside UE5. I deleted any files I could find on the hard drive relating to Bridge, and it didn’t solve the problem. But given drag and drop works, none of that seems relevant anyway. *shrug*

Say Ha-Lua To My Little Friend

It’s fine, he’s already dead

I’ve always had a soft spot for the scripting language Lua, for a number of reasons:

  1. It’s lightweight
  2. It’s fast
  3. It’s super-easy to integrate into C/C++
  4. It was used in Grim Fandango

So when I started tinkering with a new engine recently it was inevitable that I’d want to integrate Lua for entities in the game world. There’s a lot of info on the web about object-orientation in Lua, using metatables etc., and plenty of stuff about calling C functions from Lua, but I couldn’t find anything that covered what seemed to me to be quite a simple use case:

  1. Entities in script can inspect and modify properties defined in C++
  2. Properties should be accessible as properties, e.g. ‘self.position’, rather than through function calls
  3. Properties shouldn’t be copied into Lua and back out every update (there might be thousands of entities and not all of them will want access to all properties)
  4. Defining an entity script should be simple and as foolproof as possible
  5. The integration shouldn’t be auto-generated, by Swig or similar

The method I’ve settled on (so far, the tinkering continues) involves creating a global table in Lua for an entity type (the class), with its __index and __newindex metamethods set to redirect to C++ for properties that aren’t found. Since this is boilerplate for every entity type it shouldn’t be included in an entity’s script itself. Instead I perform this setup in C++:

lua_newtable(L);
lua_setglobal(L, classNameString);
lua_getglobal(L, classNameString);
lua_getglobal(L, "lookup");
lua_setfield(L, -2, "__index");
lua_getglobal(L, "insert");
lua_setfield(L, -2, "__newindex");
lua_pop(L, 1);

Lua object-orientation tutorials tend to define a class:new(o) function that creates instances of a class by creating a new table and setting its metatable to be the class table. I didn’t want to have to copy and paste this boilerplate code for every entity type, but since it would only be called from C++ anyway, and the colon syntax is actually syntactic sugar for class.new(class, o) i.e. we are passing the class as the first parameter this all means we only need to define a single function to create all instances of all classes:

function newObject(self, o)
    o = o or {}
    setmetatable(o, self)
    return o
end

So now when we try to read a property on the entity that doesn’t exist, Lua will go via the metatable and call lookup. Similarly when we try to write a property, insert will be called. Both functions are defined in Lua:

function lookup(table, key)
    return rawget(getmetatable(table), key) or getEntityProperty(table, key)
end

function insert(table, key, value)
    if not setEntityProperty(table, key, value) then
        rawset(table, key, value)
    end
end

Lookup will try to find the property on the parent/class table (to allow for pseudo-‘static’ properties) and if it’s still not found there it’ll call getEntityProperty, a C function. Conversely, insert will first ask C if it’s responsible for the property via setEntityProperty and if that returns false it’ll set the property on the entity table in Lua.

The getEntityProperty and setEntityProperty functions are static, they know which entity the Lua script is part of because the entity table in Lua has an ‘addr’ property which is a light user data pointer to the entity in C++. An instance is created like this:

// push the entity pointer onto the stack (we'll use it later)
lua_pushlightuserdata(L, entity);
// call newObject(class)
lua_getglobal(L, "newObject");
lua_getglobal(L, classNameString);
lua_pcall(L, 1, 1, 0);
// set newly-created entity table's 'addr' member to the entity pointer
lua_pushstring(L, "addr");
lua_pushlightuserdata(L, entity);
lua_settable(L, -3);
// finally, set registry[entity pointer] = entity table
lua_settable(L, LUA_REGISTRYINDEX);

We store the Lua entity table in the registry, keyed on the entity pointer, so when the C++ entity wants to update it’s easy to find it’s Lua counterpart.

What all this means is in Lua I can define an entity script like this:

function cube:tick(dt)
    pos = self.position
    pos.x = ...
    pos.y = ...
    pos.z = ...
    self.position = position
end

And the position in C++ is updated transparently, without having to copy it into and back out of Lua.

That’s nice and simple, but it comes with a drawback that you have to cache C++ properties in Lua to avoid requesting them multiple times (which is an issue if the property is non-trivial). Initially I solved this by caching the property in Lua from the C++ side, so when a property was requested from C++ just before it was returned, I’d store it in a property on the entity table. This way any future reads would read from the Lua property and not redirect to C++, but of course any writes wouldn’t redirect to C++ either. I suspect I can probably solve it with an intermediate cache table between the class and entity tables; use the cache table to hold the properties so that the entity table will redirect to the cache table to read properties and to C++ to write them. The write would have to update the value in the cache table also. But that starts to feel a little overengineered… we’ll see!

Builtin A* Pathfinding in Unreal Engine 4.25

A few years back I wrote Neatly replacing NavMesh with A* in UE4 and ever since I’ve had a vague notion that it’s probably gone wildly out-of-date. As it happens I was recently working on another project that would benefit from A* and I noticed UE4 already has an A* implementation called FGraphAStar, so I thought I’d write a little updated post talking about both. (Fun fact: FGraphAStar was added about two months before I wrote the earlier post… the lesson here is: always search the codebase…)

I’ve put a simple demo project up at https://bitbucket.org/crussel/astar425. Left-click the ground and watch ConeMan A* pathfind like a winner.

ANavigationData Replacement

The process of replacing NavMesh hasn’t changed much so I won’t go over it again, though it is a little simpler to plug-in: you don’t need to edit DefaultEngine.ini manually, you can just add a default Agent to the supported agents list in Project Settings/Navigation System and set its NavigationClass to your NavMesh replacement.

FGraphAStar

There’s a comment above the definition in GraphAStar.h that tells us what we need to implement. FGraphAStar is a template struct where the template parameter class TGraph has to define a type and a couple of functions. The type, FNodeRef is just a value that uniquely-identifies a node in the graph – in my simple grid demo, it’s just an FIntPoint since a grid cell can be uniquely-identified by it’s coordinates. The functions needed are:

int32 GetNeighbourCount(FNodeRef NodeRef) const

This returns the maximum number of neighbours the given node could have. In my grid-based demo, I return 8 since I allow diagonal movement. Notice I return 8 even for an edge cell – trying to filter at this point just creates unnecessary clutter.

bool IsValidRef(FNodeRef NodeRef) const

Does the given NodeRef refer to a valid node? In my demo I just return whether the coordinates are on the grid.

FNodeRef GetNeighbour(const FNodeRef NodeRef, const int32 NeighbourIndex) const

Now things get interesting – return the Xth neighbour of the given node. In my demo I count the neighbours clockwise from postive Y (this is why trying to cater for edge cells in GetNeighbourCount would get messy: if I’m a corner cell and return 3 in that case, I need to check if I’m a corner cell here again to reinterpret NeighbourIndex correctly – but if I always have 8 neighbours, the values of NeighbourIndex always have the same meaning).

So that gets the basic structure of our node graph into A*. The rest is in the query filter, which defines the following:

float GetHeuristicCost(const FNodeRef StartNodeRef, const FNodeRef EndNodeRef) const

This is the cheaply-calculated estimated cost of pathing from StartNodeRef to EndNodeRef. For my grid demo I’m just returning the manhattan distance.

bool IsTraversalAllowed(const FNodeRef NodeA, const FNodeRef NodeB) const

Can you go from NodeA to NodeB? For the grid demo, if I only allowed 4-directional movement this would be trivial: yes, if node A and node B are both open. Since I allow diagonal movement I also make sure e.g. north and east are both clear before allowing a move north-east.

float GetTraversalCost(const FNodeRef StartNodeRef, const FNodeRef EndNodeRef) const

This is the real (i.e. not estimated) cost of travelling from StartNodeRef to the adjacent node EndNodeRef. A* finds the cheapest path, so this can be used to mark preferred routes e.g. if the current node is ‘road’, and left is ‘road’ but right is ‘swamp’ we might return 1 to go left and 2 to go right.

float GetHeuristicScale() const

The comment says “used as GetHeuristicCost’s multiplier”; I haven’t used it in the demo (I just return 1) since the environment is trivial enough that heuristics don’t really matter. In a more complex environment (and a more complex heuristic) the scale can be used to adjust how much A* relies on the heuristic – if you return zero, the heuristic is ignored completely, making A* completely accurate but much slower. As you increase the value A* will search fewer paths and possibly settle on one that’s suboptimal.

And that’s about it. Nice and simple, but then A* is.

Just Cause-style wingsuit/grapple/gliding

I’ve clocked up over 60hrs on Just Cause 3, and most of it is just gliding about with the wingsuit. There’s something very compelling about that calm beauty intersersed by moments of complete panic as you very nearly smack into a tree/car/tower block.

I wanted to have a go at recreating that same feel, and I knew I’d need a nice terrain to fly over so I turned to the UE4 open world ‘Kite’ demo. I’ll upload the code to a repo just as soon as I have time to extricate it neatly from the (multiple gigabyte) project. In the meantime, here’s a soothing video:

https://www.youtube.com/watch?v=3zowfGoYFjs

VR Hub / Apartment Trashing Simulator

As part of the International Festival for Business last week I helped out at Realspace’s VR hub in the Liverpool Science Centre. Realspace have recently set up, amongst other things providing a space for local businesses to experiment with VR. They knew I was working on a VR game and asked if I could demo it at the event – but the game is such an early prototype it’s still full of coder art, which I’m not keen to show to people! Instead I showed the very first thing I made after getting my DK2: I took Epic’s ‘Realistic Rendering’ environment and added some physics-based gaze mechanics. So you could walk around Epic’s beautiful apartment lounge, and turn the lamps on and off and pick up and smash objects ‘with your mind’. Ludicrous but oddly compelling – it was fun to see Serious Business People light up with joy when they realised they could smash the place up.

https://www.youtube.com/watch?v=MSDCi16vx7s

For The Loot!

Match details: join us tonight 20:00 BST (GMT+1), Steam has to be running, and change your download region to UK/Manchester. I’ll be hosting as ‘chrismcr’.

Download the post-jam game here!

https://www.youtube.com/watch?v=-9KD6ll9l1k

Last weekend I took part in my first 4-day #ue4jam, as part of the Sleepless Beanbags team. The theme was ‘fire in the hole’ so obviously that implies a multiplayer brawler about miners chucking dynamite at each other.

Because we could only test 3-person multiplayer it was nervewracking to watch Allar set up a 10-person game on his livestream… but it just about held up!

Anyway – this post is to say we’ve done a post-competition release which fixes many of the bugs, most importantly the names are now visible everywhere so you can actually see which character you are!

But beyond that, because it’s a multiplayer game it’s not much fun on your own! So I’ll be hosting a game at 20:00 (UK time!) tonight, and we’d love it if you’d join us. You’ll need to run Steam first, set your download region in settings to UK/Manchester, then run the game and join the match called ‘chrismcr’.

So grab the post-jam game here and we’ll see you at 20:00!

Escape in UE4

Just over four years ago, for Ludum Dare 23, my friend Pete and I made a game called Escape in C++/OpenGL. The theme was ‘Tiny World’, which we chose to interpret as ‘the level shrinks as you play’. Inspired by the red weed from H.G. Wells’ War of The Worlds we decided the player is stranded on a planet, surrounded by alien goop which is slowly spreading towards them. Their task is to harvest resources strewn about the environment to fuel/repair their rocket and escape. Different types of resource are needed, and each type requires a different length of time to harvest, during which the player can’t move. Adding to the difficulty, the player can only carry three resources at once, after which they’ll need to drop off at the rocket before going out to harvest again. The player is also in a vehicle, controlled with the mouse – so it feels ever so slightly out of control. Pete wrote a great theme tune which lent the whole thing an unnerving, anxious feeling. The graphics would politely be called ‘lo-fi’, but overall I was rather pleased with it.

Fast-forward to last weekend, and I was investigating networking in UE4. It occurred to me Escape would be a good project to try out in Unreal, because I’d networked the original shortly after Ludum Dare and I remembered how long it had taken to find the balance between prediction and sending lots of data – it would be interesting to see how smooth that part would be in UE4.

Since then I’ve been recreating Escape in UE4, and in my network research I noticed a few people on the forums asking for complete networked demos so I thought it’d be useful to release the source. It’s not finished by any means, there’s only one type of resource right now and the logic detecting win and lose is decidedly crummy, but the networking is all there and you can play a complete game from start to finish in single and multiplayer. You can grab the project here. I wanted to do the whole thing in blueprints, but I also wanted to play online via Steam – and right now the blueprint session interface just isn’t up to the task, so I’m using Mordentral’s Advanced Sessions plugin. Which means it does need compiling, though all of the game code is still in blueprints.

I’ll be writing a bunch of posts about the project next week hopefully – the networking aspects, but also some of the effects were quite fun to reimagine so I’ll probably write about those too. And maybe a word or two about why the UE4 vehicle looks like Postman Pat’s van. For now though, it’s the #ue4jam!

Unity Visibility/Fog of War Effect

Here’s a method for ‘fog of war’ from an old Unity project. This is the kind used in games like XCOM where the world geometry outside the visible range of your units is darkened, but still at least partially visible.

I added a circle mesh (an extremely short cylinder will also work) to the player unit prefab with a radius the same as the units view radius. The circle is on its own layer (UnitViewDistance), and is textured with a radial gradient going from white to transparent black.

Screen Shot 2016-03-24 at 17.21.40

I added a camera (UnitViewCamera) with a depth of 0 (all other cameras are depth 1, so UnitViewCamera renders first). UnitViewCamera’s culling mask is set so that it only renders the UnitViewDistance layer, and it renders into a RenderTexture created in code. The result of all this is a transparent image with various white blobs, something like this:

mask

The RenderTexture is then plugged into a shader and used as a mask. The intensity of the mask texel is used to lerp the colour of the final texel between full colour (at 1) and desaturated (at 0):

fixed4 frag (v2f i) : SV_Target
{
    fixed4 col = tex2D(_MainTex, i.uv);
    float bwCol = col.r*.3 + col.g*.59 + col.b*.11;

    fixed4 maskCol = tex2D(_viewTex, i.uv);
    float maskIntensity = maskCol.r;

    col = lerp(bwCol, col, maskIntensity);
    return col;
}

Here’s a screenshot of a programmer-art world demonstrating the (very subtle) effect – the unit is off-screen to the bottom-right, so you can see the cubes desaturating towards the top-left:

Screen Shot 2016-03-24 at 17.39.21

Neatly replacing NavMesh with A* in UE4

tilebasedworld

[ Update: Builtin A* Pathfinding in Unreal Engine 4.25 ]

I’ve been working on a tile-based game recently, and I wanted to use A* for pathfinding (NavMesh is overkill, and not a great fit). I could’ve just written an A* pathfinder and custom AI code that uses it, but I wondered if there might be a better way – ideally I’d like to just replace NavMesh with my pathfinder, and have the standard AIController/PathFollowingComponent code work with it seamlessly.

I came across MieszkoZ’ answer to this question, which got me started – turns out not only is it possible, it’s pretty simple (with a couple of caveats!)

A New Pathfinding Class

NavMesh is contained in a class called ARecastNavMesh, a subclass of ANavigationData. In theory all you need to do is create your own subclass and plug it into the engine. It’s worth looking at the source for ARecastNavMesh to see how it does things – the key function is FindPath:

static FPathFindingResult FindPath(const FNavAgentProperties& AgentProperties, const FPathFindingQuery& Query);

What’s a little odd is that FindPath isn’t virtual, it’s static. Comments in ANavigationData and ARecastNavMesh explain it’s for performance reasons: Epic are concerned that if a lot of agents call the pathfinder in the same frame the vtable lookups will be too slow, so instead the function is declared static and stored in a function pointer, ANavigationData::FindPathImplementation.

Another effect of FindPath being static is that it has no this pointer. Thankfully there is a weak pointer to this in Query.NavData, so you can Get() that and use it instead.

FindPath is expected to return an FPathFindingResult struct, which contains a success/failure enum and an FNavigationPath. FindPath‘s Query parameter may contain an FNavigationPath for you to use (if Query.PathInstanceToFill is valid) or you’ll have to create a new one using ANavigationData::CreatePathInstance.

All of which is much easier to say in code! So here’s a template FindPath based on ARecastNavMesh::FindPath [updated for 4.12]:

FPathFindingResult AAStarNavigationData::FindPath(const FNavAgentProperties& AgentProperties, const FPathFindingQuery& Query)
{
	const ANavigationData* Self = Query.NavData.Get();
	AAStarNavigationData* AStar = const_cast<AAStarNavigationData*>(dynamic_cast(Self));
	check(AStar != nullptr);

	if (AStar == nullptr)
	{
		return ENavigationQueryResult::Error;
	}

	FPathFindingResult Result(ENavigationQueryResult::Error);
	Result.Path = Query.PathInstanceToFill.IsValid() ? Query.PathInstanceToFill : Self->CreatePathInstance(Query);

	FNavigationPath* NavPath = Result.Path.Get();

	if (NavPath != nullptr)
	{
		if ((Query.StartLocation - Query.EndLocation).IsNearlyZero())
		{
			Result.Path->GetPathPoints().Reset();
			Result.Path->GetPathPoints().Add(FNavPathPoint(Query.EndLocation));
			Result.Result = ENavigationQueryResult::Success;
		}
		else if(Query.QueryFilter.IsValid())
		{
			// **** run your pathfinding algorithm from Query.StartLocation to Query.EndLocation here
			// add each point on the path with:
			// NavPath->GetPathPoints().Add(FNavPathPoint(WORLD_POSITION));
			// NOTE: the path must contain at least 2 non-start path points

			// if your algorithm can only find a partial path call NavPath->SetIsPartial(true),
			// but remember to check if partial paths are acceptable to the caller (Query.bAllowPartialPaths)
			// - if they aren't you should return ENavigationQueryResult::Fail

			NavPath->MarkReady();
			Result.Result = ENavigationQueryResult::Success;
		}
	}

	return Result;
}

Connect your pathfinding algorithm, assign FindPath to FindPathImplementation in the constructor, and you’re done!

Plugging In

To plug your new pathfinder into the engine, you need to edit Config/DefaultEngine.ini. Find the section called:

[/Script/Engine.NavigationSystem]

And add the line:

RequiredNavigationDataClassNames=/Script/ProjectName.NavigationClass

In your map, create a NavMeshBounds as normal, and an instance of your pathfinder will be automatically added (instead of ARecastNavMesh-Default).

Now any AIController::MoveToLocation calls will automatically use your new pathfinder.

Other Changes for Tile-Based Games

There are three things about the default behaviour of an AI controller that don’t feel right to me in a tile-based game:

  1. Characters accelerate and brake as they run around, so they overshoot corners and bump into things
  2. Characters turn immediately in the direction of movement
  3. Characters get ‘close enough’ to their destination, and stop

So I also do the following:

  1. In the CharacterMovement component, set Requested Move Use Acceleration to false (or just lower the character’s max walk speed)
  2. In CharacterMovement, set Orient Rotation to Movement to true and Rotation Rate (Yaw) to 1440
  3. In calls to MoveToLocation, set the Acceptance Radius to 0 and Stop on Overlap to false

Which gives me characters neatly running around a tile-based world, not bumping into things, and stopping exactly where I want them. Marv!