UE4 Networking and Tower Unite

While playing Tower Unite, I captured some packets, wondering why my connection intermittently dropped when playing a minigame with a lot of mouse movement. I saw a lot of similarly formatted UDP packets and after a while I became curious about how Unreal replicates network state.

I Googled for someone else’s summary, but then I discovered UE4 is partially open source and the docs are available. You are forced to create an Epic Games account and agree to a license agreement to access their GitHub organisation, but then you may download the engine’s source code.

Eventually I found their fairly comprehensive documentation, which is obviously written by a mishmash of developers and technical writers. The first page, Networking Overview is a good start for understanding how Unreal Engine views networking, but doesn’t give you much on how Actors are actually replicated under the hood. This detail isn’t useful reading for developers - they select an engine to avoid thinking about engine level considerations, like differential state replication. The second useful page was Detailed Actor Replication Flow. While rummaging through ReplicateActor related source, I opened ActorChannel.h which pointed out that DataReplication.h/cpp contained the low-level replication logic in the form of FObjectReplicator.

Those links look vulnerable to bit rot, so here’s a simple overview

  • Actors may or may not replicate based on complex criteria. Some Actors may replicate faster than others, some Actors may never replicate if not relevant to the connection/player. The longer an actor doesn’t replicate, the more important it gets.
  • Underlying network channels are created and destroyed automatically depending on actor activity, with a channel per actor.
  • Differential state is stored in changesets, which are blasted across the network in property bundles, which are finally serialised into bytes. Since the server and the client use the same underlying objects, there’s no need to share excessive type information.

How does Tower Unite find game servers?

  • An HTTPS connection is made to A file is retrieved.
  • A hundred milliseconds later, Source Engine server queries are emitted to IP addresses that have no other source than the webserver above. It seems like SourceWorks offers this mechanism to non-source games, and the server queries and responses are well formed according to this format. There are a few libraries, but the format is so simple that anyone could decode it.

I couldn’t see where the server list was being looked up beyond an HTTPS server looked up soon after the game launches. I think it’s somewhere in the PAK file. I loaded the executable up with Ghidra, but the lack of symbols made the engine’s source rather cumbersome. I found a URL I didn’t expect to see, but I think it’s time to look at the PAK file. While on the way, someone mentioned a nice way to get a PDB file for the Unreal Engine and combine it with the local source to get legible reversed assembly for all the nuts and bolts of the engine, which might make everything else more legible. Essentially, download the engine version, download the utterly massive PDB files. The user mentions they ran a build, but I figure the PDB files will just be in a path somewhere.

Since this has gotten me interested in reverse engineering again, here’s another link to tutorials. Since I want to extract PAK files, here’s a tip on getting the PAK index decryption key. Seems like there’s a few half assed utilities that only run on Windows, could probably reimplement it in a nice UI, thread one thread two

I found a Python unpacker, which I felt like updating. I don’t really like being restricted to a C library, and it was a good excuse to refactor someone’s work. I should probably get my hands on some min-size PAK files from each version for simple testing, but I’ll just update with my v5 file in a backwards compatible way. Here’s what I was going to put in a GitHub comment.

UE4.20.3 PAK files with version v5 adds support for HasRelativeCompressedChunkOffset. From what I can tell, from UE4’s source, this allows for entries to be offset within a compressed chunk. Read requests must add compressed chunk offset to the start of their compression block reads, seemingly. The code is a little abstracted, so I need to read it more.

The v4 support that exists doesn’t seem to be aware of the byte added to indicate if index encryption is being used. Index encryption hampers the usability of these tools by requiring a PAK decryption key which is stored in the original application’s binary. Since this isn’t being checked, there’s no way for the tool to know if it’s about to fail to read an encrypted index.

I imagine applications whose files were unpacked with v3 were lying in their version numbers, made harmless PAK format changes, or disabled the features listed above, allowing older versions to decrypt their contents.

Required v4 features

  1. read_index isn’t aware of a byte introduced to FPakIndex in v4 indicating if the index is encrypted or not
  2. read_index can’t error requesting a hex password or decrypt encrypted indexes

Required v5 features

  1. PAK Records, known to the engine as FPakEntry, require an offset to be applied to their compressed block reads.

…anyway, I got bored. I discovered quite a lot about the network protocol, but it’s a binary serialisation format, so it’s annoying to reason about what event means what outcome. I also got bored of Tower Unite itself, though I did still want to fix the bug that made me crash when I had a high refresh rate.