MyraCodec Guide

MyraCodec is a schema-driven binary serialization library that generates zero-copy flyweight accessors from YAML schema definitions. It is designed for high-frequency trading and other latency-critical applications where every allocation matters.

Quick Start

Dependencies

Gradle (Kotlin DSL):

plugins {
    id("express.mvp.myra-codegen") version "0.1.0-SNAPSHOT" // Optional: Gradle plugin
}

dependencies {
    implementation("express.mvp.myra:myra-codec-runtime:0.1.0-SNAPSHOT")
    implementation("express.mvp.roray:roray-ffm-utils:0.1.0-SNAPSHOT")
}

Maven:

<dependency>
    <groupId>express.mvp.myra</groupId>
    <artifactId>myra-codec-runtime</artifactId>
    <version>0.1.0-SNAPSHOT</version>
</dependency>
<dependency>
    <groupId>express.mvp.roray</groupId>
    <artifactId>roray-ffm-utils</artifactId>
    <version>0.1.0-SNAPSHOT</version>
</dependency>

JVM Arguments

MyraCodec uses Java’s Foreign Function & Memory (FFM) API:

java --enable-preview --enable-native-access=ALL-UNNAMED -jar myapp.jar

Code Generation

Using the CLI

java -jar myra-codec-codegen.jar \
    --schema src/main/resources/schemas/order.myra.yml \
    --output build/generated/myra \
    --lockfile src/main/resources/schemas/order.myra.lock

Generated Artifacts

For each message, the codegen produces:

1. {MessageName}Flyweight - Zero-copy reader with getters
2. {MessageName}Builder - Single-pass encoder with setters

For each enum:

1. {EnumName} - Java enum with id() method

Lock Files

The .myra.lock file tracks:

• Stable field IDs for wire compatibility
• Schema version history
• Evolution metadata

Never delete the lock file - it ensures backward compatibility.

Binary Format

Message Layout

┌─────────────────────────────────────────────────────────────────┐
│                         Message Header                          │
│  ┌──────────────┬──────────────┬──────────────┬───────────────┐ │
│  │ Frame Length │ Template ID  │ Schema Ver   │  Reserved     │ │
│  │   (4 bytes)  │  (2 bytes)   │  (2 bytes)   │  (8 bytes)    │ │
│  └──────────────┴──────────────┴──────────────┴───────────────┘ │
├─────────────────────────────────────────────────────────────────┤
│                       Presence Bits                              │
│         (N bytes, where N = ceil(optional_fields / 8))          │
├─────────────────────────────────────────────────────────────────┤
│                       Fixed Fields Block                         │
│  ┌──────────────┬──────────────┬──────────────┬───────────────┐ │
│  │   Field 1    │   Field 2    │   Field 3    │     ...       │ │
│  └──────────────┴──────────────┴──────────────┴───────────────┘ │
├─────────────────────────────────────────────────────────────────┤
│                   Variable Fields Headers                        │
│  ┌──────────────────────┬──────────────────────┐                │
│  │  Offset (4 bytes)    │  Length (4 bytes)    │  × N fields   │
│  └──────────────────────┴──────────────────────┘                │
├─────────────────────────────────────────────────────────────────┤
│                   Variable Fields Data                           │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │  Field N data ... Field N+1 data ... Field N+2 data ...  │   │
│  └──────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

Endianness

All multi-byte integers use big-endian (network byte order) for:

• Efficient network transmission
• Predictable cross-platform behavior
• Easier debugging with hex dumps

Fixed-Capacity String Layout

┌────────────────┬────────────────────────────────┐
│ Actual Length  │         UTF-8 Data             │
│   (4 bytes)    │    (fixed_capacity bytes)      │
│   big-endian   │   (padded with zeros)          │
└────────────────┴────────────────────────────────┘

Schema Evolution

Adding Fields

Safe additions:

• New optional fields at end of message
• New enum values

# Version 1
messages:
  - name: "Order"
    fields:
      - tag: 1
        name: "orderId"
        type: "int64"

# Version 2 (backward compatible)
messages:
  - name: "Order"
    fields:
      - tag: 1
        name: "orderId"
        type: "int64"
      - tag: 2           # New field
        name: "timestamp"
        type: "int64"
        optional: true   # Must be optional for compatibility

Deprecating Fields

- tag: 5
  name: "oldField"
  type: "string"
  deprecated: true
  deprecationNote: "Use newField instead, will be removed in v3.0"

Breaking Changes (Major Version)

These require a new schema version:

• Removing fields
• Changing field types
• Changing field tags
• Making optional fields required

Troubleshooting