White Paper: MyraTransport

A Modern, Zero-Copy Transport Library for Java 25+

Version: 1.0 (Draft)

Author: roraydev@gmail.com

1. Abstract

MyraTransport is a high-performance, io_uring-based network transport library designed for modern Java (JDK 25+). It provides the foundational byte-streaming layer for the mvp.express RPC framework, delivering extreme throughput and sub-microsecond latency through aggressive use of kernel features and zero-copy techniques.

The library is built exclusively on Java’s Foreign Function & Memory (FFM) API, eliminating JNI overhead while providing direct access to Linux io_uring for asynchronous I/O. It operates entirely off-heap using registered buffers, batch submission, and optional kernel bypass modes (XDP/DPDK).

2. Core Principles

• Zero-Copy Architecture: All I/O operations use registered MemorySegment buffers shared between codec and transport layers, eliminating intermediate copies.

• io_uring Native: Direct FFM bindings to liburing, leveraging async I/O, batch submission, and registered buffer optimizations for 100x syscall reduction.

• Backend Abstraction: Pluggable backend architecture (io_uring, NIO, XDP, DPDK) allows runtime selection based on deployment environment and performance requirements.

• Codec Agnostic: Transport operates on raw MemorySegment byte streams, supporting any encoding (myra-codec, protobuf, avro, etc.).

• Production Ready: Built-in TLS, connection pooling, backpressure management, and automatic reconnection with exponential backoff.

3. Architecture Overview

┌──────────────────────────────────────────────────────────┐
│              Application Layer                           │
│         (mvp.express RPC Framework)                      │
└────────────────────┬─────────────────────────────────────┘
                     │
                     ▼
┌──────────────────────────────────────────────────────────┐
│              MyraTransport API                           │
├──────────────────────────────────────────────────────────┤
│  • Transport interface (codec-agnostic)                  │
│  • Connection lifecycle management                       │
│  • Backpressure & flow control                           │
│  • TLS/SSL encryption layer                              │
│  • Connection pooling & reuse                            │
└────────────────────┬─────────────────────────────────────┘
                     │
      ┌──────────────┴──────────────┐
      ▼                             ▼
┌─────────────────┐         ┌─────────────────┐
│ IoUringBackend  │         │   NioBackend    │
│  (Primary)      │         │   (Fallback)    │
├─────────────────┤         ├─────────────────┤
│ • Registered    │         │ • Epoll/KQueue  │
│   buffers       │         │ • Standard NIO  │
│ • Batch submit  │         │ • Cross-platform│
│ • Zero-copy     │         │                 │
└────────┬────────┘         └────────┬────────┘
         │                           │
         ▼                           ▼
┌──────────────────────────────────────────────────────────┐
│           Kernel (Linux/BSD/Darwin)                      │
│  • TCP/IP Stack                                          │
│  • TLS (via OpenSSL)                                     │
│  • io_uring / epoll / kqueue                             │
└──────────────────────────────────────────────────────────┘

3.1. Backend Architecture

public interface TransportBackend extends AutoCloseable {
    CompletableFuture<MemorySegment> send(int fd, MemorySegment data);
    CompletableFuture<MemorySegment> recv(int fd, MemorySegment buffer);
    CompletableFuture<Integer> accept(int listenFd);
    CompletableFuture<Void> connect(SocketAddress address);
    void submitBatch(); // Flush pending operations
}

4. Registered Buffers: Zero-Copy Foundation

4.1. Concept

Traditional I/O requires kernel to:

1. Pin user pages in memory (expensive!)
2. Validate address range
3. Perform virtual→physical address translation

Registered buffers bypass this on every I/O:

// Setup once
struct iovec buffers[256];
for (int i = 0; i < 256; i++) {
    buffers[i].iov_base = malloc(65536);
    buffers[i].iov_len = 65536;
}
io_uring_register_buffers(&ring, buffers, 256);

// Use by index (no pinning cost!)
io_uring_prep_send_fixed(sqe, fd, buffer_idx, len, 0);

4.2. Performance Impact

4.3. Implementation

public class RegisteredBufferPool {
    private final Arena arena;
    private final MemorySegment[] buffers;
    private final ConcurrentLinkedQueue<Integer> availableIndices;
    private final MemorySegment ioUringBuffersArray;
    
    public RegisteredBufferPool(int numBuffers, long bufferSize) {
        this.arena = Arena.ofShared();
        this.buffers = new MemorySegment[numBuffers];
        this.availableIndices = new ConcurrentLinkedQueue<>();
        
        // Allocate buffers off-heap (64-byte aligned for cache lines)
        for (int i = 0; i < numBuffers; i++) {
            buffers[i] = arena.allocate(bufferSize, 64);
            availableIndices.offer(i);
        }
        
        // Prepare iovec array for io_uring registration
        this.ioUringBuffersArray = prepareIoUringBuffers();
    }
    
    private MemorySegment prepareIoUringBuffers() {
        // struct iovec { void *iov_base; size_t iov_len; }
        long iovecSize = 16; // 8 bytes pointer + 8 bytes length
        MemorySegment iovecs = arena.allocate(iovecSize * buffers.length);
        
        for (int i = 0; i < buffers.length; i++) {
            long offset = i * iovecSize;
            iovecs.set(ValueLayout.ADDRESS, offset, buffers[i]);
            iovecs.set(ValueLayout.JAVA_LONG, offset + 8, buffers[i].byteSize());
        }
        
        return iovecs;
    }
    
    public RegisteredBuffer acquire() {
        Integer index = availableIndices.poll();
        if (index == null) throw new OutOfMemoryError("Buffer pool exhausted");
        return new RegisteredBuffer(index, buffers[index], this);
    }
    
    void release(int index) {
        availableIndices.offer(index);
    }
}

public class RegisteredBuffer implements AutoCloseable {
    private final int index;
    private final MemorySegment segment;
    private final RegisteredBufferPool pool;
    
    public int bufferIndex() { return index; }
    public MemorySegment segment() { return segment; }
    
    @Override
    public void close() {
        pool.release(index);
    }
}

4.4. Zero-Copy Integration with myra-codec

// Application encodes directly into registered buffer
try (RegisteredBuffer buffer = bufferPool.acquire()) {
    // CODEC: Encode directly into registered buffer (no heap allocation)
    OrderRequestFlyweight flyweight = new OrderRequestFlyweight();
    flyweight.wrap(buffer.segment(), MessageHeader.HEADER_SIZE);
    flyweight.setClOrdId(orderId);
    flyweight.setSymbol(symbol);
    
    // TRANSPORT: Send via io_uring (kernel already has buffer pinned)
    CompletableFuture<MemorySegment> response = 
        transport.sendRegisteredBuffer(buffer.bufferIndex(), totalSize);
}

5. Batch Submission: 100x Syscall Reduction

5.1. The Problem

Traditional approach:

for (int i = 0; i < 1000; i++) {
    send(socket, data[i]);  // 1000 syscalls = 3.5ms overhead
}

5.2. The Solution

// Prepare all operations
for (int i = 0; i < 1000; i++) {
    io_uring_prep_send(sqe[i], socket, data[i]);
}
// Submit once
io_uring_submit(&ring);  // 1 syscall = 0.5μs overhead

5.3. Adaptive Batching

public class AdaptiveBatchScheduler {
    private final List<PendingOp> batchQueue = new ArrayList<>(128);
    private final int maxBatchSize = 128;
    private final Duration maxBatchDelay = Duration.ofMicros(100);
    
    public CompletableFuture<Void> submit(MemorySegment data) {
        CompletableFuture<Void> future = new CompletableFuture<>();
        
        synchronized (batchQueue) {
            batchQueue.add(new PendingOp(data, future));
            
            // Flush conditions
            if (batchQueue.size() >= maxBatchSize) {
                flushBatch();
            } else if (batchQueue.size() == 1) {
                scheduleFlush(maxBatchDelay);
            }
        }
        
        return future;
    }
    
    private void flushBatch() {
        int submitted = io_uring_submit(ring);
        batchQueue.clear();
    }
}

5.4. Performance Comparison

6. io_uring FFM Integration

6.1. Core Structures

// io_uring ring structure
public class IoUring {
    private final MemorySegment ring;     // struct io_uring
    private final MemorySegment sqRing;   // submission queue ring
    private final MemorySegment cqRing;   // completion queue ring
    private final Arena arena;
    
    public IoUring(int queueDepth) {
        this.arena = Arena.ofShared();
        this.ring = arena.allocate(/* io_uring size */);
        
        // Initialize via FFM
        SymbolLookup lib = SymbolLookup.loaderLookup();
        MethodHandle mhInit = linker.downcallHandle(
            lib.find("io_uring_queue_init").orElseThrow(),
            FunctionDescriptor.of(
                ValueLayout.JAVA_INT,     // return
                ValueLayout.JAVA_INT,     // queue_depth
                ValueLayout.ADDRESS,      // ring
                ValueLayout.JAVA_INT      // flags
            )
        );
        
        int result = (int) mhInit.invokeExact(queueDepth, ring, 0);
        if (result < 0) throw new IOException("io_uring_queue_init failed");
    }
    
    public MemorySegment getSqe() {
        // io_uring_get_sqe(&ring)
        MethodHandle mhGetSqe = /* ... */;
        return (MemorySegment) mhGetSqe.invokeExact(ring);
    }
    
    public int submit() {
        // io_uring_submit(&ring)
        MethodHandle mhSubmit = /* ... */;
        return (int) mhSubmit.invokeExact(ring);
    }
    
    public CompletionQueueEntry waitCqe() {
        // io_uring_wait_cqe(&ring, &cqe)
        MethodHandle mhWaitCqe = /* ... */;
        MemorySegment cqe = /* ... */;
        mhWaitCqe.invokeExact(ring, cqe);
        return new CompletionQueueEntry(cqe);
    }
}

6.2. Send Operation

public CompletableFuture<Void> sendRegisteredBuffer(int bufferIndex, int length) {
    MemorySegment sqe = ioUring.getSqe();
    
    // io_uring_prep_send_fixed(sqe, fd, buffer_idx, len, flags)
    sqe.set(ValueLayout.JAVA_BYTE, OPCODE_OFFSET, IORING_OP_SEND);
    sqe.set(ValueLayout.JAVA_BYTE, FLAGS_OFFSET, IOSQE_BUFFER_SELECT);
    sqe.set(ValueLayout.JAVA_SHORT, BUF_INDEX_OFFSET, (short) bufferIndex);
    sqe.set(ValueLayout.JAVA_INT, FD_OFFSET, socketFd);
    sqe.set(ValueLayout.JAVA_INT, LEN_OFFSET, length);
    
    CompletableFuture<Void> future = new CompletableFuture<>();
    pendingOps.put(sqe.address(), future);
    
    return future;
}

7. Connection Management

7.1. Connection Lifecycle

public class TcpConnection implements AutoCloseable {
    private enum State { DISCONNECTED, CONNECTING, CONNECTED, CLOSING, CLOSED }
    
    private final SocketAddress remoteAddress;
    private final IoUringTransport transport;
    private volatile State state = State.DISCONNECTED;
    private volatile int socketFd = -1;
    
    public CompletableFuture<Void> connect() {
        if (state != State.DISCONNECTED) {
            return CompletableFuture.failedFuture(
                new IllegalStateException("Already connecting/connected")
            );
        }
        
        state = State.CONNECTING;
        
        // Create socket
        socketFd = socket(AF_INET, SOCK_STREAM, 0);
        
        // Prepare io_uring connect operation
        return transport.backend.connect(socketFd, remoteAddress)
            .thenRun(() -> state = State.CONNECTED);
    }
    
    public CompletableFuture<Void> send(MemorySegment data) {
        if (state != State.CONNECTED) {
            return CompletableFuture.failedFuture(
                new IllegalStateException("Not connected")
            );
        }
        return transport.backend.send(socketFd, data);
    }
    
    @Override
    public void close() {
        if (state == State.CLOSED) return;
        state = State.CLOSING;
        // io_uring_prep_close(sqe, socketFd)
        // ...
        state = State.CLOSED;
    }
}

7.2. Connection Pool

public class ConnectionPool {
    private final ConcurrentMap<SocketAddress, Queue<TcpConnection>> pool;
    private final int maxConnectionsPerHost = 8;
    
    public CompletableFuture<TcpConnection> acquire(SocketAddress address) {
        Queue<TcpConnection> connections = pool.computeIfAbsent(
            address, k -> new ConcurrentLinkedQueue<>()
        );
        
        TcpConnection conn = connections.poll();
        if (conn != null && conn.isConnected()) {
            return CompletableFuture.completedFuture(conn);
        }
        
        // Create new connection
        TcpConnection newConn = new TcpConnection(address, transport);
        return newConn.connect().thenApply(v -> newConn);
    }
    
    public void release(TcpConnection conn) {
        Queue<TcpConnection> connections = pool.get(conn.remoteAddress());
        if (connections.size() < maxConnectionsPerHost) {
            connections.offer(conn);
        } else {
            conn.close();
        }
    }
}

8. TLS Support via OpenSSL FFM

public class TlsTransport implements TransportBackend {
    private final MemorySegment sslCtx;  // SSL_CTX*
    private final MemorySegment ssl;     // SSL*
    
    public TlsTransport(TlsConfig config) {
        // Load OpenSSL via FFM
        SymbolLookup openssl = SymbolLookup.libraryLookup("libssl.so", arena);
        
        // SSL_CTX_new(TLS_client_method())
        this.sslCtx = createSslContext(openssl, config);
        
        // SSL_new(ctx)
        this.ssl = SSL_new(sslCtx);
    }
    
    @Override
    public CompletableFuture<MemorySegment> send(int fd, MemorySegment data) {
        // SSL_write(ssl, data, len) via io_uring
        // (OpenSSL can use custom BIO for io_uring integration)
        return sslWrite(ssl, data);
    }
}

9. Kernel Bypass Modes (Future)

9.1. XDP (eXpress Data Path)

┌──────────────────────────────────────┐
│     MyraTransport (Java)             │
└─────────────┬────────────────────────┘
              │ AF_XDP socket
┌─────────────▼────────────────────────┐
│        Linux Kernel                  │
│  ┌────────────────────────────────┐  │
│  │  XDP Program (eBPF)            │  │
│  │  • Filter packets              │  │
│  │  • Redirect to AF_XDP          │  │
│  └──────────┬─────────────────────┘  │
└─────────────┼────────────────────────┘
              ▼
        NIC Driver (10-20x faster)

9.2. DPDK (Full Kernel Bypass)

┌──────────────────────────────────────┐
│     MyraTransport (Java + FFM)       │
└─────────────┬────────────────────────┘
              │ Direct memory access
┌─────────────▼────────────────────────┐
│      DPDK Library (userspace)        │
│  • PMD (Poll Mode Driver)            │
│  • rte_mbuf (packet buffers)         │
└─────────────┬────────────────────────┘
              │ DMA
┌─────────────▼────────────────────────┐
│     NIC Hardware (100ns latency)     │
└──────────────────────────────────────┘

10. Performance Benchmarks

10.1. Latency (P99)

10.2. Throughput

*DPDK uses dedicated cores

11. Configuration

transport:
  backend: io_uring  # io_uring | nio | xdp | dpdk
  
  io_uring:
    queue_depth: 256
    registered_buffers:
      enabled: true
      num_buffers: 256
      buffer_size: 65536
    batch_submission:
      enabled: true
      max_batch_size: 128
      max_delay_us: 100
  
  connection:
    pool_size: 16
    connect_timeout_ms: 5000
    idle_timeout_ms: 300000
    keepalive: true
  
  tls:
    enabled: false
    keystore: /path/to/keystore.p12
    truststore: /path/to/truststore.p12

12. Design Decisions

12.1. Why io_uring over NIO?

• 100x fewer syscalls via batch submission
• 1.7x faster I/O with registered buffers
• Zero-copy direct memory access
• Future-proof (Linux kernel evolution)

12.2. Why FFM over JNI?

• Type-safe native calls (no crashes)
• No marshalling overhead (direct memory)
• Maintainable (pure Java, no C glue code)
• Modern (officially supported since JDK 22)

12.3. Why Codec-Agnostic?

• Reusable across encoding schemes
• Clear separation of concerns
• Easier testing and maintenance
• Pluggable architecture

13. Future Roadmap

Phase 1 (Q1 2025) - COMPLETE ✅:

• ✅ io_uring with registered buffers
• ✅ Batch submission
• ✅ Native socket creation (FFM bindings)
• ✅ Async connect/accept via io_uring
• ✅ Timeout support
• ✅ Full client and server modes
• ✅ NIO fallback backend

Phase 2 (Q2 2025) - Advanced io_uring Features:

• Buffer Rings (io_uring 5.19+)

  • Kernel-managed buffer pools
  • Zero-copy receive with automatic buffer selection
  • Eliminates per-operation buffer allocation overhead
  • Target: >10K msgs/sec workloads (HFT, market data feeds)
  • Memory savings: kernel picks right-sized buffers dynamically

• Multi-shot Operations

  • IORING_OP_RECV_MULTISHOT: One SQE → continuous stream of receives
  • IORING_OP_ACCEPT_MULTISHOT: Accept multiple connections with single submission
  • Reduces submission overhead for high-frequency workloads
  • Combined with buffer rings for maximum efficiency

• Linked Operations

  • IOSQE_IO_LINK: Chain operations atomically
  • Use case: send-then-receive, connect-then-send
  • Ensures operation ordering without userspace coordination
  • Reduced latency for dependent operations

• Performance Tuning

  • Configurable queue sizes (currently hardcoded 256)
  • IORING_SETUP_IOPOLL: Polling mode for ultra-low latency
  • CPU pinning for completion thread
  • Per-connection ring management (vs current shared ring)
  • Ring overflow detection and recovery

Phase 3 (Q3 2025) - Connection Management:

• TCP + TLS support (OpenSSL FFM bindings)
• Connection pooling with reuse
• Backpressure and flow control
• Unix domain sockets (IPC)

Phase 4 (Q4 2025) - Alternative Transports:

• XDP backend for packet filtering
• QUIC transport (HTTP/3)
• UDP with connection tracking

Phase 5 (2026+) - Specialized Backends:

• DPDK backend (HFT niche, kernel bypass)
• RDMA support (InfiniBand for datacenter)
• Multicast (UDP group communication)

13.1 Cross-Repo Alignment (November 2025)

• Phase 1 (current focus): Land io_uring registered-buffer benchmarks so we can compare against Netty/gRPC while Myra Codec + roray-ffm-utils finalize adapters.
• Phase 2: Plug the transport into example-kvstore-app scenarios—Myra codec pairing plus alternative serialization combos (JSON, Avro, Thrift, Kryo, FlatBuffers, Protobuf, MessagePack).
• Phase 3: Provide the networking backbone for jia-cache, including clustering hooks (etcd/Apache Helix) and replication-ready transports.
• Phase 4: Roll these capabilities into the commercial mvp.express RPC platform and schema registry, ensuring the transport exposes tracing/metrics/retry knobs expected by enterprise adopters.

This design prioritizes performance without sacrificing safety, leveraging modern Java features to compete with C/C++ performance while maintaining Java’s productivity and ecosystem advantages.