How to read Grim Dawn binary files

Before we begin

00 · What you will be able to do

When you see four bytes in hexadecimal notation — 44 00 00 00 — you will not have to guess what they mean. You will ask the right questions: where does the value begin, how many bytes does it occupy, and which rule tells us how to read them?

1. distinguish the bytes themselves from the notation used to show them;
2. read multi-byte integers in little-endian order;
3. locate a region of a file from its offset and length;
4. understand why bytes do not have one “correct” meaning without a format rule.

One cell

01 · A file is a numbered sequence of bytes

On disk, a file is not divided into “headers,” “records,” or “names.” To a computer, it is a sequence of small numbers. Each number occupies one byte at a particular position.

A bit is one binary digit: 0 or 1. A byte consists of eight bits. Interpreted as an unsigned integer, one byte can hold a number from 0 to 255. The same number can be written in different ways. For example, decimal 68 and hexadecimal 0x44 describe the same byte.

A shorter notation

02 · Hex changes the notation, not the data

Binary files are often shown in hexadecimal notation (usually shortened to hex). It uses sixteen digits: the familiar 0–9 plus the letters A–F.

Two hex digits cover exactly one byte, from 0x00 to 0xFF. This makes byte boundaries immediately visible, unlike a long sequence of zeroes and ones.

This series follows one simple rule for individual numbers: a number with 0x is hexadecimal; a number without it is decimal. Therefore 0x18 and 24 are the same value.

A sequence of bytes is preceded by the label HEX. The label applies to the whole sequence: in HEX 44 00 00 00, every pair of characters is one hexadecimal byte.

An unsigned integer from 0 through 255 fits in one byte. The value 256 does not, so one number must occupy several bytes. That raises a new question: in which order should we combine them?

The order problem

03 · The same two bytes can form different numbers

Start with ordinary decimal notation. In 21, the digit 1 means one unit and 2 means two tens. So 2 × 10 + 1 = 21: a digit's position determines its multiplier.

Bytes follow the same principle, except that the next position uses a multiplier of 256 rather than 10. One byte has 256 possible values, from 0 to 255. In a two-byte integer, one position is therefore multiplied by 1 and the other by 256.

In the HEX sequence 01 02, byte 0x01 comes first and 0x02 comes second. The sequence alone does not determine which one should be multiplied by 1. There are two possible results:

first byte × 1

The second byte gets the multiplier 256: 1 × 1 + 2 × 256 = 513.

first byte × 256

The second byte gets the multiplier 1: 1 × 256 + 2 × 1 = 258.

Both calculations are valid. The format must specify which multiplier belongs to the first byte. This rule is called byte order.

ARZ uses little-endian order: the byte with multiplier 1 is stored first, followed by the byte with multiplier 256. In ARZ, the HEX sequence 01 02 therefore means 513: multiply the first byte by 1 and the second by 256.

u32

An unsigned 32-bit integer, which occupies four bytes.

LE

Little-endian: the least significant byte — the one with multiplier 1 — is stored first.

A map of the file

04 · An offset answers “where?”

An offset is a distance in bytes from a defined reference point. In file+0x18, that point is the beginning of the file: skip 0x18, or 24 bytes.

The synthetic ARZ header occupies the first 24 bytes. One of the later sections is the record table; the next article explains its purpose. The record_table_offset field contains 0x9D, so the table starts at position 157.

0x00..0x18: The start is included; the end is not. This range therefore covers positions 0x00 through 0x17: exactly 24 bytes, not 25.

An offset does not name a section. It only leads to a position. The rules of the particular format determine what starts there and how to read it.

From numbers to strings

05 · A string is bytes plus a rule

A string is a sequence of characters. In a file, it is stored as bytes with no built-in “I am a string” label. The format must define where the string starts, how many bytes it occupies, and which encoding interprets those bytes.

ARZ collects these values in a string table: one numbered list shared by the whole file. Some formats place a zero byte called NUL after each string. ARZ does not. Instead, every string is preceded by a four-byte length that counts its bytes.

In this example, ASCII turns bytes into characters. It is a mapping in which, for example, byte 0x43 represents the letter C.

In the four verified ARZ archives, every string consists only of ASCII bytes. The format stores a length, however, not the name of an encoding. This is an important boundary: an observation about the current profile should not become an eternal rule for every possible ARZ file. If bytes above 0x7F appear, a reader should preserve them and report the encoding as unresolved instead of silently guessing.

Reading without guessing

06 · Safe reading starts with bounds checks

A header may say that a section starts here and has a certain length, but that does not guarantee those bytes actually exist. The file may be truncated, damaged, or from another version.

1. Confirm the expected structure

   Apply only a format profile whose layout you actually support.

2. Calculate the start and end

   Calculate the section end as offset + length, using checked arithmetic. If the addition overflows or the calculated end falls before the section start, the result is invalid and reading must stop.

3. Compare against the container bounds

   A section cannot extend beyond the end of the file.

4. Read exactly the declared region

   Extra or missing bytes are a signal, not a minor detail.

Applying the rules

07 · Reading the first four ARZ bytes

We can now move from general byte-encoding rules to the specific current ARZ profile. Its first four bytes are two little-endian integers, each two bytes wide. The specification gives them neutral structural names because their original private names are unknown:

The current official writer emits (2, 3). Native readers reject header_word_0 only when it is below 2, so this is not an exact-match magic value. We describe its observed role as a format compatibility revision.

The second word is not a scalar version. It is a set of header feature bits: 0x0001 participates in compatibility checks between archive layers, and 0x0002 announces the 16-byte regional-checksum footer. The meanings of the two states of the first bit and of the remaining bits are not established.

The same four bytes form 0x00030002, decimal 196610, if displayed as one u32 LE. That is only a packed representation of the two words, not one semantic version value.

After reading

Five ideas to keep

1. A file is an addressed sequence of bytes.
2. Hex is a compact notation for those numbers.
3. A multi-byte integer needs a width and a byte order.
4. An offset and length define a region of a file.
5. A value appears only after applying a format rule.