libdestruct Skills

# libdestruct Skills

libdestruct is a Python library for destructuring binary data into typed objects. It maps raw bytes to C-like types (integers, floats, strings, structs, pointers, arrays, enums, bitfields) with read/write support.

## Installation

```bash
pip install git+https://github.com/mrindeciso/libdestruct.git
```

## Core Concepts

All types inherit from `obj`. Every `obj` has:
- `.value` property to read/write the underlying data
- `.address` property for the memory offset
- `.to_bytes()` to serialize back to bytes
- `.freeze()` / `.diff()` / `.reset()` for snapshotting
- `.hexdump()` for a hex dump of the object's bytes
- `.from_bytes(data)` class method to create a read-only instance from raw bytes

Memory is accessed through an `inflater`, which wraps a `bytes`, `bytearray`, or `mmap.mmap` buffer. Use `bytearray` or writable mmap for read/write access. For file-backed memory, use `inflater_from_file()`.

## Quick Reference

### Imports

```python
from typing import Annotated
from libdestruct import (
    inflater,              # memory wrapper (bytearray / mmap)
    inflater_from_file,    # file-backed inflater (convenience)
    FileInflater,          # file-backed inflater class
    struct,                # struct base class
    c_int, c_uint,         # 32-bit integers (signed/unsigned)
    c_long, c_ulong,       # 64-bit integers (signed/unsigned)
    c_short, c_ushort,     # 16-bit integers (signed/unsigned)
    c_char, c_uchar,       # 8-bit integers (signed/unsigned)
    c_float, c_double,     # IEEE 754 floats (32/64-bit)
    c_str,                 # null-terminated C string
    ptr,                   # 8-byte pointer
    ptr_to,                # typed pointer field descriptor (legacy)
    ptr_to_self,           # self-referential pointer field descriptor (legacy)
    array, array_of,       # array type + field descriptor
    vla_of,                # variable-length array field descriptor
    enum, enum_of,         # enum type + field descriptor
    flags, flags_of,       # bit flags type + field descriptor
    bitfield_of,           # bitfield descriptor
    union,                 # union annotation type
    union_of,              # plain union field descriptor
    tagged_union,          # tagged union field descriptor
    offset,                # explicit field offset
    size_of,               # get size in bytes of any type/instance/field
    alignment_of,          # get natural alignment of any type/instance
)
```

### Type Sizes

| Type | Size (bytes) |
|---|---|
| `c_int` / `c_uint` | 4 |
| `c_long` / `c_ulong` | 8 |
| `c_float` | 4 |
| `c_double` | 8 |
| `ptr` | 8 |
| `c_str` | variable (reads until null) |

### Reading Primitives from a Buffer

```python
memory = bytearray(b"\x2a\x00\x00\x00\x00\x00\x00\x00")
lib = inflater(memory)

x = lib.inflate(c_int, 0)    # inflate c_int at offset 0
print(x.value)                # 42

y = lib.inflate(c_long, 0)   # inflate c_long at offset 0
print(y.value)
```

### Reading Primitives from Raw Bytes

```python
x = c_int.from_bytes(b"\x2a\x00\x00\x00")
print(x.value)  # 42
# Note: from_bytes returns a frozen (read-only) object
```

### Writing Primitives

```python
memory = bytearray(4)
lib = inflater(memory)
x = lib.inflate(c_int, 0)
x.value = -1
print(memory)  # bytearray(b'\xff\xff\xff\xff')
```

### Defining Structs

```python
class player_t(struct):
    health: c_int
    score: c_uint
    position_x: c_float
    position_y: c_float
```

Struct fields are laid out sequentially. Access members as attributes; each returns a typed `obj` (use `.value` to get the Python value).

### Inflating Structs

```python
import struct as pystruct

memory = bytearray(16)
memory[0:4] = pystruct.pack("<i", 100)
memory[4:8] = pystruct.pack("<I", 5000)
memory[8:12] = pystruct.pack("<f", 1.5)
memory[12:16] = pystruct.pack("<f", -3.0)

lib = inflater(memory)
player = lib.inflate(player_t, 0)

print(player.health.value)      # 100
print(player.score.value)       # 5000
print(player.position_x.value)  # 1.5
```

Or from raw bytes (read-only):

```python
player = player_t.from_bytes(memory)
```

### Pointers

```python
class node_t(struct):
    value: c_int
    next: ptr["node_t"]       # pointer to own type (forward ref)

# Typed pointer to another type:
class container_t(struct):
    data: c_int
    ref: ptr[c_long]          # subscript syntax (preferred)
```

Legacy syntax with `ptr_to()` and `ptr_to_self()` is still supported:

```python
class node_t(struct):
    value: c_int
    next: ptr = ptr_to_self()

class container_t(struct):
    data: c_int
    ref: ptr = ptr_to(c_long)
```

Dereference with `.unwrap()` or safe `.try_unwrap()` (returns `None` if invalid):

```python
node = lib.inflate(node_t, 0)
print(node.value.value)
next_node = node.next.unwrap()       # follow pointer
maybe_node = node.next.try_unwrap()  # None if invalid
```

Pointer arithmetic (C-style, scaled by element size):

```python
p = lib.inflate(ptr, 0)
p.wrapper = c_int
print(p[0].value)           # element at index 0
print(p[1].value)           # element at index 1
print((p + 2).unwrap().value)  # element at index 2
```

Pointer results are cached; call `.invalidate()` after memory changes.

### Forward References

For mutually referential structs, use `ptr["TypeName"]`:

```python
class tree_t(struct):
    value: c_int
    left: ptr["tree_t"]
    right: ptr["tree_t"]
```

### Arrays

```python
class packet_t(struct):
    length: c_int
    data: array[c_int, 8]     # subscript syntax (preferred)
```

Legacy syntax with `array_of()` is still supported:

```python
class packet_t(struct):
    length: c_int
    data: array = array_of(c_int, 8)
```

Access array elements:

```python
pkt = lib.inflate(packet_t, 0)
print(pkt.data[0].value)       # first element
print(pkt.data.count())        # 8
for element in pkt.data:
    print(element.value)
```

### Variable-Length Arrays

VLAs model C flexible array members: the count is read from a sibling field at inflation time.

```python
class packet_t(struct):
    length: c_int
    data: array[c_int, "length"]     # subscript syntax (string = VLA)
```

Or with the descriptor:

```python
class packet_t(struct):
    length: c_int
    data: array = vla_of(c_int, "length")
```

```python
pkt = lib.inflate(packet_t, 0)
print(len(pkt.data))              # reads from pkt.length.value
print(pkt.data[0].value)          # first element
```

Size semantics: `size_of(packet_t)` returns the fixed part only (excludes VLA). `size_of(instance)` includes VLA data. VLA must be the last field in the struct. VLA elements can be structs.

### Enums

```python
from enum import IntEnum

class Color(IntEnum):
    RED = 0
    GREEN = 1
    BLUE = 2

class pixel_t(struct):
    color: enum[Color]        # subscript syntax (preferred, defaults to c_int backing)
    alpha: c_int

# With a custom backing type:
class pixel_t(struct):
    color: enum[Color, c_short]  # 2-byte backing type
    alpha: c_int
```

Legacy syntax with `enum_of()` is still supported:

```python
class pixel_t(struct):
    color: enum = enum_of(Color)
    alpha: c_int
```

```python
pixel = lib.inflate(pixel_t, 0)
print(pixel.color.value)  # Color.RED
```

### Bit Flags

Use Python's `IntFlag` for bitmask fields:

```python
from enum import IntFlag

class Perms(IntFlag):
    READ = 1
    WRITE = 2
    EXEC = 4

class file_t(struct):
    mode: flags[Perms]            # subscript syntax (defaults to c_int backing)
    size: c_int

# With a custom backing type:
class file_t(struct):
    mode: flags[Perms, c_short]   # 2-byte backing
    size: c_int
```

Legacy syntax with `flags_of()`:

```python
class file_t(struct):
    mode: flags = flags_of(Perms)
    size: c_int
```

```python
f = lib.inflate(file_t, 0)
print(f.mode.value)           # Perms.READ|Perms.WRITE
print(Perms.READ in f.mode.value)  # True
```

By default flags are lenient (unknown bits produce raw int). Use `flags_of(Perms, lenient=False)` for strict mode that raises `ValueError` on unknown bits.

### Bitfields

```python
class flags_t(struct):
    read: c_int = bitfield_of(c_int, 1)
    write: c_int = bitfield_of(c_int, 1)
    execute: c_int = bitfield_of(c_int, 1)
    reserved: c_int = bitfield_of(c_int, 29)
```

Consecutive bitfields with the same backing type are packed together. The struct above is 4 bytes total, not 16.

### Unions

```python
from libdestruct.common.union import union, union_of, tagged_union

# Plain union — all variants overlaid at the same offset
class packet_t(struct):
    data: union = union_of({"i": c_int, "f": c_float, "l": c_long})

pkt = lib.inflate(packet_t, 0)
pkt.data.i.value  # interpret as int
pkt.data.f.value  # interpret as float (same bytes)

# Tagged union — discriminator selects the active variant
class message_t(struct):
    type: c_int
    payload: union = tagged_union("type", {
        0: c_int,
        1: c_float,
        2: point_t,  # struct variants work too
    })
```

The discriminator field must appear before the union. The union size is the max of all variant sizes. Struct variant fields are accessible directly: `msg.payload.x.value`. Use `.variant` to get the raw variant object. Unknown discriminator values raise `ValueError`.

### Struct Alignment

```python
# Default: packed (no padding)
class packed_t(struct):
    a: c_char
    b: c_int
# size: 5

# Aligned: natural C alignment with padding
class aligned_t(struct):
    _aligned_ = True
    a: c_char
    b: c_int
# size: 8 (1 + 3 padding + 4)

alignment_of(c_int)       # 4
alignment_of(aligned_t)   # 4 (max member alignment)

# Custom alignment width
class wide_t(struct):
    _aligned_ = 16
    a: c_int
# size: 16, alignment: 16
```

### Explicit Field Offsets

```python
from typing import Annotated

class sparse_t(struct):
    a: c_int
    b: Annotated[c_int, offset(0x10)]  # Annotated syntax (preferred)
```

This works with any type, including subscript types:

```python
class example_t(struct):
    a: c_int
    data: Annotated[array[c_int, 4], offset(0x10)]
    ref: Annotated[ptr[c_int], offset(0x20)]
```

Legacy syntax with default values is still supported:

```python
class sparse_t(struct):
    a: c_int
    b: c_int = offset(0x10)
```

### Nested Structs

```python
class vec2(struct):
    x: c_float
    y: c_float

class entity_t(struct):
    id: c_int
    pos: vec2
```

```python
e = lib.inflate(entity_t, 0)
print(e.pos.x.value)
```

### Struct Inheritance

Structs support Python class inheritance. Derived structs include all parent fields first, then their own.

```python
class base_t(struct):
    a: c_int

class derived_t(base_t):
    b: c_int
```

```python
d = derived_t.from_bytes(pystruct.pack("<ii", 10, 20))
print(d.a.value)  # 10
print(d.b.value)  # 20
size_of(derived_t)  # 8
```

Multi-level inheritance (A -> B -> C) and alignment inheritance both work. Parent fields always appear first in layout and `to_dict()`.

### size_of

```python
size_of(c_int)             # 4
size_of(c_long)            # 8
size_of(player_t)          # computed from fields
size_of(array_of(c_int, 10))  # 40
size_of(some_instance)     # works on instances too
```

### Hex Dump

```python
player = lib.inflate(player_t, 0)
print(player.hexdump())
# 00000000  64 00 00 00 88 13 00 00 00 00 c0 3f 00 00 40 c0  |d..........?..@.|  health, score, position_x, position_y
```

Struct hexdumps annotate lines with field names. Primitive hexdumps show raw bytes.

### Dict / JSON Export

```python
point = point_t.from_bytes(memory)
point.to_dict()  # {"x": 10, "y": 20}

import json
json.dumps(entity.to_dict())  # nested structs produce nested dicts
```

`to_dict()` works on all types: primitives return their value, structs return `{name: value}` dicts, arrays return lists, unions return variant values, enums return their int value.

### Freeze / Diff / Reset

```python
x = lib.inflate(c_int, 0)
x.freeze()           # snapshot current value
x.value = 99         # raises ValueError (frozen)

# For non-frozen objects:
x.freeze()           # save state
# ... memory changes externally ...
print(x.diff())      # (old_value, new_value)
x.reset()            # restore to frozen value
x.update()           # update frozen value to current
```

### C Struct Parser

Parse C struct definitions directly (requires `pycparser`):

```python
from libdestruct.c.struct_parser import definition_to_type

player_t = definition_to_type("""
    struct player_t {
        int health;
        unsigned int score;
        float x;
        double y;
    };
""")

player = player_t.from_bytes(data)
```

Supports: nested structs, pointers (including self-referential), arrays, bitfields, typedefs, `#include` directives (requires a C preprocessor), and `__attribute__` stripping.

## Common Patterns

### Parsing a binary format

```python
class header_t(struct):
    magic: c_uint
    version: c_int
    num_entries: c_int
    entries_ptr: ptr[entry_t]

with open("file.bin", "rb") as f:
    data = bytearray(f.read())

lib = inflater(data)
header = lib.inflate(header_t, 0)

for i in range(header.num_entries.value):
    entry = header.entries_ptr[i]
    # process entry...
```

### Modifying binary data in-place

```python
data = bytearray(open("save.bin", "rb").read())
lib = inflater(data)
player = lib.inflate(player_t, 0x100)
player.health.value = 999
open("save.bin", "wb").write(data)
```

### File-backed inflater

Read (and optionally write) binary files directly via mmap, without loading the entire file into memory:

```python
# Read-only
with inflater_from_file("firmware.bin") as lib:
    header = lib.inflate(header_t, 0)
    print(header.magic.value)

# Writable — changes are persisted to the file
with inflater_from_file("save.bin", writable=True) as lib:
    player = lib.inflate(player_t, 0x100)
    player.health.value = 999
```

You can also pass an `mmap.mmap` object directly to `inflater()`.

### Working with libdebug

libdestruct integrates with [libdebug](https://github.com/libdebug/libdebug) for live process memory inspection. The debugger's memory view can be passed directly to `inflater`.