SPUD (Structured Payload of Unintelligible Data)

SPUD is a custom binary file format for efficient storage and transmission of structured data. It uses type tags and field name interning for compactness. This implementation is written in Rust.

Features

• Compact Binary Representation: Binary encoding for all supported data types.
• Supported Data Types: null, bool, signed/unsigned integers (i8–i64, u8–u64), f32, f64, String, and raw binary blobs (Vec<u8>).
• Field Name Interning: Field names are mapped to unique IDs to reduce redundancy.
• Versioning: Files start with a version string for compatibility.
• Simple Structure: Version header, field name map, data payload, and EOF marker ([0xDE, 0xAD, 0xBE, 0xEF]).
• Serde Integration: Serialize/deserialize Rust structs via serde.

File Structure

A .spud file consists of:

1. Version String: UTF-8 bytes.
2. Field Name Map: Sequence of (length, field_name_bytes, id) entries, ending with 0x01.
3. Data Payload: Sequence of (field_id, type_tag, value_bytes) entries. Strings/blobs include length. Arrays/objects are delimited by start/end tags.
4. EOF Marker: 0xDE, 0xAD, 0xBE, 0xEF.

Usage

Encoding (Writing a SPUD file)

You can build SPUD files manually or (in the future) by serializing Rust structs with serde.

Manual Usage

use spud::spud_builder::SpudBuilder;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut builder: SpudBuilderSync = SpudBuilderSync::new();

    builder
        .object(|obj: &SpudObjectSync| {
            obj.add_value("email", SpudString::from("alice@example.com"))?;

            obj.object("userdata", |f: &SpudObjectSync| {
                f.add_value("name", SpudString::from("Alice"))?;
                f.add_value("surname", SpudString::from("Smith"))?;

                Ok(())
            })?;

            obj.add_value("balance", Decimal::from_str_exact("1234.56").unwrap())?;

            Ok(())
        })
        .unwrap();

    Ok(())
}

Decoding (Reading a SPUD file)

You can decode SPUD files manually or (in the future) deserialize them into Rust structs with serde.

Manual Usage

use spud::spud_decoder::SpudDecoder;

let mut decoder = SpudDecoder::new_from_path("output_dir/my_spud_data.spud").unwrap();
let data = decoder.decode().unwrap();

println!("{:?}", data);

Roadmap / TODO

• Parallelism
• serde integration for SpudBuilder and SpudDecoder
• spud!{} macro

Known Issues

Some minor bugs may exist; please report any issues.

Contributing

Contributions are welcome! Open an issue or submit a pull request.

README.md partially generated by AI