A high-throughput model for recovering global objects in monolithic firmware.

TL;DR: A 13M-parameter model trained from scratch recovers global objects in stripped firmware more accurately than GPT-6.1-Sol (81.8 vs 69.3 Bounds+Kind F1 on 53 real-world images), over 100x faster and runnable locally.

Figure 1: Pipeline overview.

Context

To detect an out-of-bounds access in a program, you must know the bounds of each object. This information isn’t available when working with a stripped binary. So how do you tell?

With gray-box fuzzing, we rely on the program crashing to tell us something has gone wrong. The issue with that is that 1) not every corruption manifests as a crash, and 2) of the ones that do, they tend to produce many duplicates. So in theory, we could have a lot better SNR if we could catch out-of-bounds accesses the moment they occur.

This process is called sanitization. Most sanitizers today work at the compiler level, meaning they won’t work if all you’ve got is a binary. But how would a sanitizer even know an OOB access has occurred without knowing the objects in the program first?

Objects (the kind DWARF metadata gives you) typically involve many man-hours to recover.

Modern decompilers (Ghidra, IDA, etc.) will generate labels in the RAM segment for each access they are able to make out in the disassembled code. Not each label is an object start, though, as many labels are intra-buffer accesses made by the code (i.e., a parser accessing the header of a buffer). Additionally, not every object start will be a label, since decompilers cannot recover memory accesses where the address is not known until runtime.

Sure, you could throw the binary at GPT-6 and give it access to tools, but it’s very slow and expensive.

Initial approach

My initial instinct was to use constant literals to tell the start of each object and then estimate their sizes based on their relative positioning. This works well half the time, but for certain compilation modes (i.e.,-fsection-anchors in GCC), you can end up missing a lot of objects when the objects share the same constant literal. Here’s an example:

ldr r1, [pc, #0x104] ; r1= 0x20_003_958
str r5, [r1] ; F_CPU_ACTUAL = ..
ldr r1, [pc, #0x104] ; r1 = 0x20_003_970
str r3, [r1] ; F_BUS_ACTUAL = ..

In the above example, each object maps to a constant literal (the object’s RAM address is stored in ROM and retrieved via a PC-relative load). However, with -fsection-anchors the above becomes:

ldr r1, [pc, #0x88] ; r1 = 0x20_003_700
str.w ip, [r1] ; F_CPU_ACTUAL = blah
str r2, [r1, #4] ; F_BUS_ACTUAL = blah

This problem is further aggravated when you have LTO (-flto) enabled because it’ll relocate globals from across source files and you’ll end up with even more anchor sections.

The second challenge is determining the object kind (arrays, structs, primitive etc.). The primary goal of object identification is finding all global arrays so we can sanitize them at runtime. Therefore, this distinction is critical.

What makes this tricky is that you can have all sorts of C types that compile to the same disassembly, meaning some level of type erasure is inevitable:

For example, each pair below compiles to identical machine code:

struct { uint8_t data[N]; } x;     ≡   uint8_t x[N];
struct { float a, b, c, d; } q;    ≡   float q[4];
enum mode_e m;                     ≡   uint8_t m;      // 1-byte enum
char name[N];                      ≡   uint8_t name[N];

Pipeline

There’s a paper from 2021 (OSPREY) that argues compilation throws away too much information for deterministic rules to recover types reliably. The paper instead combines many different features with probabilistic inference. After toying around with a few heuristic methods like the literal-based approach above, I wanted to see how far that idea could go if the features were weighted by a neural net instead. The result is a pipeline that takes in a raw binary, computes features for potential object starts, and runs them through a neural net to predict C-like types.

Static analysis

The first stage of our pipeline (Figure 1) uses static analysis to come up with a list of object candidates and a corresponding vector feature set. It assumes that the base address and RAM memory regions are already known (things our platform already provides).

An object candidate is an address that could start an object, a field, or an element inside one. Most candidates are not objects, which is the encoder’s job (described below) to figure out.

The .data and .bss sections are first reconstructed in RAM by symbolically executing from the firmware’s reset handlers (yes, firmware can have multiple vector tables). This provides an initial RAM layout and initial values for globals.

Functions are recovered by following call targets starting with the reset handler and scanning for function prologues across the firmware.

Once function recovery is complete, each function is analyzed using inter-procedural VSA to determine memory accesses and their contexts. Context includes the base address and offset, width (how many bytes are stored/loaded), whether it is a load or a store, the stride of the access, and the bound of the loop it appears in. The context also includes instruction information, which can often reveal the type (i.e., a VFP instruction would suggest a float). This context information is also used to identify memset/memcpy functions, which are then inspected at their call sites to identify constant-size args, which help provide size hints.

Each object candidate gets about ~200 features that encode information about its position, size, and type. For example, we encode how the candidate is referenced (i.e., code literal or at an offset from a code literal), its distance to adjacent candidates (a useful predictor of size), its initial value type (pointer/entropy/zero), how it’s accessed (i.e., in strides or in a loop).

For the section anchor problem I described earlier (which didn’t become apparent until I picked apart the scores by compilation flags), we look at how the anchor is used across functions. More specifically, to tell struct fields apart from object starts accessed through the same anchor, we track which functions touch which offsets from the same anchor. Accesses to adjacent offsets from the anchor in largely the same set of functions are typically indicative of structs. Conversely, when these accesses happen in mostly distinct functions, they typically relate to disparate objects.

ldr  r3, =0x20012570   ; the only literal aka the anchor
ldr  r0, [r3, #0]         ; used by dispatch code
strb r1, [r3, #off1]    ; another global used in a different context
ldrh r2, [r3, #off2]   ; another global used in a different context

Using the initial naive approach, the above example would have yielded a single object start, when in reality the same literal is used to access three distinct globals in separate contexts. Furthermore, without the context features, the above accesses would be confused for struct fields or interior accesses of an array object.

One obvious limitation with this approach is that if two globals are always accessed together through the same anchor, they can be confused for struct fields.

Encoder

The encoder is a small transformer model (~13M params) that we trained from scratch. It reads object candidates in address order in windows of 256. Feeding multiple object candidates allows the model to reason about candidates in the context they occur. For example, if an address is accessed with a stride of 12 and the next candidate is 96 bytes later, and the code clears 96 bytes here, this is probably an array of 8 12-byte structs.

Figure 2: Reasoning across neighbouring candidates. Accesses every 12 bytes from candidate_i and the next candidate 96 bytes later (candidate_j) point to an array of eight 12-byte structs.

We trained the encoder from scratch on 5.1 million objects from 59k firmware builds with various RTOSes, compilers, and compiler optimization flags. The DWARF metadata was used as the ground truth.

The encoder’s output is class probabilities for each candidate, including:

• its role: object start, struct field, array element, or interior access.
• its kind: scalar/pointer/function pointer/enum/struct/string/array.
• end: which of the next 255 candidates is the next object start
• element size

Decoder

The decoder turns these probabilities into objects using deterministic rules. Candidates with an “object start” class probability above a certain threshold are turned into objects. Each object is ended at the most likely end candidate and adopts the highest-probability kind. For structs, candidates’ gaps between members are filled using alignment and initial values from .data and .bss.

Here are some snippets comparing the decoder’s predictions to the original ground truth to give you an idea of what the output looks like:

servoFilter : biquadFilter_t[8] (320 bytes, Betaflight flight controller)

typedef struct {
  float b0;      /* +0x00 */
  float b1;      /* +0x04 */
  float b2;      /* +0x08 */
  float a1;      /* +0x0c */
  float a2;      /* +0x10 */
  float x1;      /* +0x14 */
  float x2;      /* +0x18 */
  float y1;      /* +0x1c */
  float y2;      /* +0x20 */
  float weight;  /* +0x24 */
} biquadFilter_t;
biquadFilter_t servoFilter[8];

struct obj_2001365c {
  float f_0x00;  /* +0x00 */
  float f_0x04;  /* +0x04 */
  float f_0x08;  /* +0x08 */
  float f_0x0c;  /* +0x0c */
  float f_0x10;  /* +0x10 */
  float f_0x14;  /* +0x14 */
  float f_0x18;  /* +0x18 */
  float f_0x1c;  /* +0x1c */
  float f_0x20;  /* +0x20 */
  float f_0x24;  /* +0x24 */
};
struct obj_2001365c g_2001365c[8];

g_usb_pstd_driver : usb_pcdreg_t (52 bytes, Edge Impulse firmware)

typedef struct {
  uint8_t *p_devicetbl;   /* +0x00 */
  uint8_t *p_qualitbl;    /* +0x04 */
  uint8_t *p_configtbl;   /* +0x08 */
  uint8_t *p_othertbl;    /* +0x0c */
  void **p_stringtbl;     /* +0x10 */
  void (*devdefault)(…);  /* +0x14 */
  void (*devconfig)(…);   /* +0x18 */
  void (*devdetach)(…);   /* +0x1c */
  void (*devsuspend)(…);  /* +0x20 */
  void (*devresume)(…);   /* +0x24 */
  void (*interface)(…);   /* +0x28 */
  void (*ctrltrans)(…);   /* +0x2c */
  uint8_t num_string;     /* +0x30 */
  /* padding */           /* +0x31 */
  /* padding */           /* +0x32 */
  /* padding */           /* +0x33 */
} usb_pcdreg_t;
usb_pcdreg_t g_usb_pstd_driver;

struct obj_20008a9c {
  void *f_0x00;       /* +0x00 */
  void *f_0x04;       /* +0x04 */
  void *f_0x08;       /* +0x08 */
  void *f_0x0c;       /* +0x0c */
  void *f_0x10;       /* +0x10 */
  void (*f_0x14)(…);  /* +0x14 */
  void (*f_0x18)(…);  /* +0x18 */
  void (*f_0x1c)(…);  /* +0x1c */
  void (*f_0x20)(…);  /* +0x20 */
  void (*f_0x24)(…);  /* +0x24 */
  void (*f_0x28)(…);  /* +0x28 */
  void (*f_0x2c)(…);  /* +0x2c */
  uint8_t f_0x30;     /* +0x30 */
  uint8_t f_0x31;     /* +0x31 actually padding */
  uint8_t f_0x32;     /* +0x32 actually padding */
  uint8_t f_0x33;     /* +0x33 actually padding */
};
struct obj_20008a9c g_20008a9c;

Addressing type erasure

As I mentioned earlier, some type erasure is inevitable, and we don’t want to penalize the model for types it cannot recover. To that end, we “canonicalize” the types and consider semantically equivalent types the same in our scoring. For example, this means that structs containing elements of the same type and width are treated the same as arrays of that type.

Conversion rules, applied until no rule matches (T is a primitive type, S a struct, n and m counts):

(1)  struct { T a[n]; }               ->  T[n]
(2)  struct { T x; }                  ->  T
(3)  T[1]                             ->  T
(4)  struct { T f1; T f2; … T fn; }   ->  T[n]      // no padding
(5)  S[m],  where S ⇒ T[n]            ->  T[m x n]
(6)  enum E                           ->  uintW_t   // W = 8 · sizeof(E)
(7)  char[n]                          ->  uint8_t[n]

Integers of the same size are treated as the same type (uint8_t, int8_t, char, bool are all just 1-byte integers). Floats and pointers stay distinct. A prediction counts as correct if it matches the declared type, or if both canonicalize to the same class.

Results

We compare our pipeline with GPT-6.1-Sol on 53 real-world firmware images in our test set. The images range from small bootloaders to large flight controllers totaling 51k objects. None of their project families appear in the data we used to train the encoder. We gave GPT-6.1-sol access to disassemblers (radare2, Capstone, and objdump) and Python with Capstone installed. It also had the same memory map information as our pipeline. You can find the complete prompt here.

We measure two things:

• Bounds: an object counts as found if its start address is exact and its end is at or after the true end without running into the next object (trailing padding is allowed).
• Kind: scalar, pointer, function pointer, enum, struct, union, string, array of primitives or array of structs. Declarations that compile to the same bytes are treated as equal (see Addressing type erasure).

Recall is the share of true objects found, whereas precision is the share of reported objects that are correct.

Since one of our big motivators was finding buffers to sanitize at runtime, we also looked at arrays separately:

Most of the arrays we miss sit inside structs, and this is where GPT-6.1-Sol clearly beats us (39.4 vs 19.5 Untyped Array F1). These arrays are usually reached only through a pointer obtained at runtime. Our static analysis only follows addresses it can compute from the code, so it sees no accesses inside these structs, and they never reach the encoder. My guess is that GPT-6.1-Sol recognizes many of these structs from its training data.

One idea we will explore in the future is leveraging the runtime traces to infer type information to fill in these gaps. Our platform already rehosts the binary in an emulator for fuzzing, so that should be an easy lift.

Applications

The main application I had in mind when working up this experiment was sanitization. There’s quite a bit of work on using object bounds for runtime memory-safety checks (SoftBound, Annelid and BinArmor). This usually involves associating each pointer with an object and flagging when the pointer is used to access an unrelated object. This comes with its own set of challenges, as pointers can be nominally reused to point to different objects (i.e., a memset that clears the bss).