Designing the Binary Wire Protocol for a C++ Limit Order Book Matching Engine
This article explains how I designed the binary wire protocol for my single threaded limit order book matching engine in C++. It covers struct padding, endianness, and strict aliasing, why I rejected JSON, XML, FIX, and schema compilers for the critical path, how Simple Binary Encoding (SBE) handles versioning without any runtime parsing, how the Linux kernel delivers network bytes to my process, and why a misaligned 4 byte integer costs extra CPU cycles while a correctly ordered one does not. Each section opens with a short explanation and follows with a deeper technical breakdown for readers who want the full detail. Every section uses the same example order: type='N', order_id=101, price=10050 (meaning $100.50), quantity=10, side='B'.
While building the network layer of my matching engine, I wrote a struct to represent an incoming order and cast raw network bytes directly onto it using reinterpret_cast. It looked correct in the source code:
struct OrderRequest {
char type;
int order_id;
int price;
int quantity;
char side;
};
I assumed this struct was exactly 14 bytes, matching the 14 bytes I intended to send over the network for my example order. It was not. The compiler had silently expanded it to 20 bytes by inserting invisible padding, which meant that if a client sent a tight 14 byte message, my server would read every field after type from the wrong offset. The engine would not crash. It would silently accept an order with a corrupted price or quantity and match it into the live book as if nothing was wrong. That single failure mode is what forced me to learn, precisely and from first principles, what a C++ compiler and a modern CPU actually do with a struct in memory, and how a wire protocol has to be designed around that reality rather than around how the code merely reads on the page. This article is the complete record of that investigation, covering every concept I worked through, from struct padding down to the individual clock cycles spent inside the CPU.
Table of Contents
The Breaking Problem: Adding a Field in the Middle of a Struct
Why I Rejected Text Formats and Schema Compilers for the Critical Path
Applying Versioning to My Order Book: A Self Trade Prevention Example
How Big Can the Kernel Buffer Get, and What Happens When It Fills Up
Why Text Formats Are Slow: A CPU Level Walkthrough of JSON, XML, and FIX
SBE Reality Check: What My Live Engine Actually Sees on the Wire
Random Access Memory: Why the CPU Can Teleport to Any Offset
What Happens When My 14 Byte Message Actually Arrives at the Server
Three Layouts Side by Side: No Padding, Injected Padding, Smart Ordering
1. Zero Copy Deserialization and Its Three Hidden Traps
The technique I was reaching for is called zero copy deserialization: interpreting raw network bytes directly as a structured C++ object without moving or transforming the data first. For my order book, this means treating the 14 raw bytes that arrive on the wire as if they already are an OrderRequest object, with no intermediate parsing step. This idea has three hidden problems that will cause bugs, memory corruption, or performance penalties depending on the compiler and the CPU architecture: data alignment and padding, endianness, and undefined behavior through strict aliasing.
Why All Three Traps Matter at Once
Each of these three problems shows up independently, but they compound. Padding can silently change my struct's size. Endianness can silently reverse the bytes inside a field. Strict aliasing can let the compiler's optimizer silently miscompile the cast itself. Any one of these, on its own, is enough to corrupt a price or quantity field without ever throwing an error or crashing the program. Section 2 covers the alignment trap in detail, Section 3 covers endianness, and Section 4 covers strict aliasing. Every other section in this article grows out of solving these three problems correctly for my specific project.
2. The Alignment Trap: How Padding Broke My Struct
CPUs do not like reading data from arbitrary memory addresses. A 4 byte integer usually needs to sit at a memory address that is a multiple of 4. If it does not, the CPU has to do extra work, or on some architectures such as ARM it can fail outright. To prevent this, the C++ compiler automatically inserts hidden bytes, called padding, into a struct to align its fields. This is exactly the bug described in the Problem Statement above: my struct looked like 14 bytes of data, but the compiler actually built 20 bytes, and my fields read from the wrong offsets.
The Exact Byte Layout, and the Fix
Here is what my original struct actually looked like to the compiler:
Field | Size | True Memory Offset | Purpose |
|---|---|---|---|
| 1 byte | Offset 0 | my data |
padding | 3 bytes | Offsets 1, 2, 3 | injected by compiler for alignment |
| 4 bytes | Offset 4 | my data, now safely aligned to 4 |
| 4 bytes | Offset 8 | my data |
| 4 bytes | Offset 12 | my data |
| 1 byte | Offset 16 | my data |
padding | 3 bytes | Offsets 17, 18, 19 | injected so total struct size is a multiple of 4 |
The total size in memory is 20 bytes, even though my actual data only adds up to 14 bytes. If the network sends exactly 14 tight bytes, my reinterpret_cast reads the wrong offsets: order_id accidentally reads 3 bytes of real order data and 1 byte of something else entirely.
The fix is to force packing, using #pragma pack(push, 1):
#pragma pack(push, 1) // No padding, pack tightly
struct OrderRequest {
char type; // 1 byte
int order_id; // 4 bytes, starts at offset 1
int price; // 4 bytes, starts at offset 5
int quantity; // 4 bytes, starts at offset 9
char side; // 1 byte, starts at offset 13
};
#pragma pack(pop)
// sizeof(OrderRequest) is now exactly 14 bytes
3. The Endianness Trap: Agreeing on Byte Order
Different hardware architectures store the individual bytes of a multi byte integer in different orders, a property called endianness. If my C++ application runs on a standard x86 Intel or AMD server, it uses little endian, meaning the least significant byte comes first. Network routers and some external exchanges use big endian, the traditional network byte order, meaning the most significant byte comes first. For my project, I made a deliberate, simple decision: both my client and my server run on the same little endian x86_64 architecture that I control, so I keep both sides in native little endian order and skip byte swapping entirely.
What Happens If the Two Sides Disagree
If my C++ application runs on a standard x86 Intel or AMD server using little endian, but a network router or an external exchange sends data in network byte order, which is big endian, my integer value 1 (0x00000001) would be read by my struct cast as 16777216 (0x01000000).
The rule of thumb I follow is: if I control both sides of the network, meaning my own server talking to my own client, I can keep both little endian. If I were talking to an external exchange, I would have to use functions like ntohl (network to host long) to flip the bytes before reading them.
4. The Strict Aliasing Trap: Why reinterpret_cast Is Unsafe
In modern C++, using reinterpret_cast to look at a raw char buffer as a struct is technically undefined behavior under the strict aliasing rule. The compiler assumes two different types cannot point to the same memory slot, and its optimization phase might completely break or omit code that violates this assumption. I replaced every one of these casts in my engine with std::memcpy, which a good compiler optimizes into a zero instruction CPU register move, with none of the undefined behavior risk.
The Safe Pattern I Use Everywhere in My Engine
The universally approved, zero overhead way to do this in modern C++ is std::bit_cast (C++20) or std::memcpy. A good compiler completely optimizes std::memcpy away into a zero instruction CPU register move. It looks like it copies memory, but it does not:
// The safe, modern, zero-overhead way:
OrderRequest req;
std::memcpy(&req, buffer, sizeof(OrderRequest)); // Compiler optimizes this perfectly, avoiding strict-aliasing bugs
Putting the alignment fix from Section 2 and the aliasing fix together, the summary checklist for my struct layout is:
#include // For std::memcpy
#include // For explicit sizing like int32_t
#pragma pack(push, 1)
struct OrderRequest {
char type; // 1 byte
int32_t order_id; // 4 bytes, explicit bit widths are safer than plain int
int32_t price; // 4 bytes
int32_t quantity; // 4 bytes
char side; // 1 byte
};
#pragma pack(pop)
void process_buffer(const char* buffer) {
OrderRequest req;
std::memcpy(&req, buffer, sizeof(OrderRequest)); // standard compliant, optimized to a raw register read
if (req.type == 'N') {
// req.price is ready to use immediately
}
}
5. Comparing My Struct Against Future Tooling
Before committing to hand written structs, I compared my approach against industry tools such as Simple Binary Encoding (SBE) and FlatBuffers. Both give up nothing on runtime speed, but they trade a small amount of setup complexity for automatic safety around alignment and versioning, which matters more as a system grows past a single team controlling both ends of the wire.
The Feature by Feature Comparison
Feature | My Fixed Binary Layout | Future Tools (SBE / FlatBuffers) |
|---|---|---|
Parsing overhead | Zero, direct memory access | Zero, designed for direct memory access |
Cross platform safety | Dangerous, I must manually handle endianness and padding | Automatic, the compiler tool handles byte flipping and alignment natively |
Backward compatibility | Brittle, adding a field in the middle breaks old software | Excellent, supports schema evolution so new fields can be added without breaking older code |
Dependencies | None, pure raw C++ | External, requires a code generation tool during the build |
My conclusion at this stage was to stick with my fixed width binary approach, since it is exactly how the fastest systems work under the hood, while making sure to use #pragma pack(1) and std::memcpy so the compiler cannot silently corrupt my layout. Section 25 covers the exact trade off I made and why I did not adopt SBE for this specific project.
6. A Concrete Wire Layout for My Example Order
To visualize this, I used a real example scenario: a new order with type='N' (new), order_id=101, price=$100.50 (stored as integer cents, 10050), and quantity=10. With #pragma pack(1) applied, my struct maps directly to a sequential line of 14 raw, continuous bytes, and reading any field out of it is a single, direct memory read.
The Exact Wire Diagram
Reading this in code is a direct memory pointer offset, using zero instructions to copy or parse:
int32_t current_price = req.price;
7. The Breaking Problem: Adding a Field in the Middle of a Struct
I then reasoned through a future scenario: six months from now, the exchange updates their system and requires every message to also include an Account ID (int32_t) right after order_id. Simply inserting a new field into the middle of my struct silently shifts every offset that comes after it, which is exactly the kind of change that breaks compatibility between an old client and a new server.
Exactly Where the Break Happens
If I update my struct like this:
struct OrderRequestV2 {
char type;
int32_t order_id;
int32_t account_id; // new field added here
int32_t price;
// ...
};
The break: if an old server sends me a Version 1 message, which is 14 bytes, but my new binary is expecting Version 2, my code tries to read account_id at byte offset 5. The actual price data is now misaligned. My code would read completely garbage prices, resulting in a catastrophic trading bug.
8. How Simple Binary Encoding Solves Versioning
SBE does not get rid of the fixed width layout concept, it still uses the exact same idea of direct memory mapping. The difference is that instead of relying on a fragile C++ struct layout, I would define a schema file (XML) that describes the message data structure, and a small header carrying a template ID and block length would ride in front of every message, letting an updated server safely detect when an older, shorter message has arrived.
The Schema, the Header, and the Decoder
The SBE compiler reads this XML and generates a helper C++ class called a Flyweight. SBE adds a fixed header, usually 4 to 8 bytes, to the front of every message container. The header explicitly states the template ID and the block length of the message:
Instead of a reinterpret_cast or std::memcpy directly to a struct, I would wrap the incoming network buffer in the SBE Flyweight viewer:
// SBE doesn't copy the data. It wraps the raw buffer with a clean interface.
OrderRequestDecoder orderDecoder;
orderDecoder.wrap(buffer, actingBlockLength, schemaVersion);
int32_t current_price = orderDecoder.price(); // Reads bytes directly from memory offsets
When the exchange later adds account_id, they append it to the end of the schema message block. When my updated SBE engine receives an older Version 1 message, the engine checks the header's BlockLength, sees the incoming message block is only 14 bytes long, and when my code calls orderDecoder.accountId(), the SBE framework realizes the field is not present in the incoming packet and automatically returns a default value such as 0, completely protecting my system from reading corrupt data offsets.
The full under the hood comparison:
Characteristic | My Packed C++ Struct | Future Tools (SBE) |
|---|---|---|
Data on the wire | Pure raw bytes, no meta information | Raw bytes plus a small structural header |
How data is parsed | Casting pointer offsets | Flyweight pointers decoding specific offsets |
What happens if a field is added | All memory offsets shift, crash or corruption unless every client recompiles simultaneously | Older engines safely ignore newer fields, newer engines gracefully default older missing fields |
My current code hardcodes the physical memory byte map directly into the executable. Future tools move the structural definitions out into an external schema architecture, using wrapping decoder classes that let me read raw memory blocks with automated safety guards.
9. Why My Original Design Was a House of Cards
I identified two massive practical reasons my raw struct approach is risky: hidden compiler changes, and system updates. Neither of these shows up as a compile error or a crash, both of them silently corrupt trading data instead, which is the worst possible failure mode for a matching engine.
The Two Specific Failure Scenarios
The invisible bug: moving to a different server. My design relies on a handshake deal with my current C++ compiler (g++, clang) and my current CPU, likely Intel or AMD x86. If I deployed the exact same code to a different server, for example an ARM based server, or simply updated my compiler version, the compiler is legally allowed to change how it structures my memory. Compiler A might lay out my OrderRequest using 20 bytes because it likes 4 byte boundaries. Compiler B might optimize it to 16 bytes. If my client is compiled with Compiler A and my server with Compiler B, the client sends what it thinks is a valid OrderRequest, the server casts the pointer directly to its own version, and because the hidden padding bytes do not match up, the server reads completely scrambled data. My price field would accidentally contain parts of my quantity field. The system would not crash, it would just trade the wrong quantities at completely wrong prices.
The nightmare bug: upgrading my system. If I need to add a single new feature, such as a tracking timestamp, in the middle of my struct:
struct OrderRequest {
char type;
int order_id;
int timestamp; // new 4-byte field added here
int price;
int quantity;
char side;
};
The moment I change this struct, every single byte offset after order_id shifts down by 4 bytes. If I deploy this update to my gateway server but have not forced every trading client to update at the exact same microsecond, the system breaks catastrophically: an un-updated client sends a message where bytes 9 to 12 represent price, but my updated server reads bytes 9 to 12 and treats them as timestamp, then reads bytes 13 to 16 for price. My system would blindly parse garbage data.
The reason frameworks like SBE exist is not because they are faster than a struct cast, they are not, my struct cast is the absolute speed limit of computer science. They exist because human beings write code, systems change, and software needs to be upgraded, and the generated code handles the offset correction automatically so data never gets corrupted.
10. Why I Rejected Text Formats and Schema Compilers for the Critical Path
This is the exact reasoning from my project's architecture notes, kept unchanged here since it is the decision that actually shipped in my engine.
Network messages need to be encoded for transmission. The main options:
Text formats (JSON, XML, FIX) are human-readable. The problem is parsing. Converting text fields to integers requires loops, branches, string operations, and sometimes memory allocation. A 50-byte JSON order message may require 500 CPU instructions to parse. Rejected.
Schema compilers (Protocol Buffers, FlatBuffers) generate efficient code but introduce external dependencies and hide the underlying mechanics. For this project I want to understand every layer. Rejected.
Fixed-width binary is what I am using. I design the wire message to match the in-memory C++ struct exactly. When bytes arrive from the network, I cast the buffer pointer directly to a struct pointer. No parsing at all.
struct OrderRequest {
char type; // 'N' = New order, 'C' = Cancel
int order_id; // 4 bytes
int price; // 4 bytes, stored as integer cents
int quantity; // 4 bytes
char side; // 'B' = Buy, 'S' = Sell
};
// After recv() fills the buffer:
const OrderRequest* req = reinterpret_cast(buffer);
// req->price is immediately usable. Zero parsing instructions.
One important note: I represent price as an integer (cents or ticks), never as a float. Floating-point arithmetic is non-associative and imprecise for decimal values. $100.01 stored as a float is actually 100.00999... internally. All real trading systems use integer price representation.
What the CPU Actually Does With Each Rejected Format
The reasoning above is the short version I use when explaining my architecture. The long version, meaning exactly what the CPU is doing instruction by instruction when it parses a JSON, XML, or FIX message instead of a fixed width struct, is detailed fully in Section 22, including the specific 500 to 2000 nanosecond cost of scanning, allocating, and converting text into integers, compared to the roughly 0.5 nanosecond cost of a single aligned memory read from my struct. The reinterpret_cast shown above is exactly the pattern I later replace with std::memcpy once I account for strict aliasing, covered in Section 4, and the exact struct layout shown above is exactly the layout that needs the padding fix from Section 2 and the field reordering from Section 26 to be both correct and fast.
11. The Golden Rule of Binary Protocols: Sequence Is Everything
Because a binary message has no keys like "order_id": to tell the computer what the data is, the client and the server must agree to a perfectly identical sequence of fields down to the exact byte. If my struct says order_id comes first and takes 4 bytes, the receiving side blindly grabs those exact 4 bytes and labels them order_id, regardless of what is actually there.
What a Field Order Mismatch Actually Looks Like
Imagine my client writes a message using this sequence: order_id (4 bytes, value 101, 0x00000065), then price (4 bytes, value 5000, 0x00001388). On the wire, the client sends these 8 raw bytes:
If my server programmer made a mistake and arranged the server struct differently, putting price first:
// SERVER STRUCT (Wrong Sequence!)
struct OrderRequest {
int price; // 4 bytes, expects this first
int order_id; // 4 bytes, expects this second
};
When those 8 bytes arrive, the server blindly maps its struct onto that raw memory buffer, and the result is:
The computer will not throw an error, will not crash, and will not warn me. It will simply execute a trade for an order ID it does not recognize at a completely absurd price. The law of fixed binary layouts I follow from this point on is: Rule 1, the data types must be exactly the same size on both sides; Rule 2, the fields must be in the exact same order on both sides; Rule 3, the spacing, meaning padding, between fields must be exactly the same on both sides, as covered in Section 2.
12. Framing the Stream: Message Header Plus Body
If the network card just receives a continuous pipeline of raw, anonymous bytes, my server needs to know where one message ends and the next begins, and which struct template to use to cast those bytes. The solution I use is a Framing Protocol: prepending a tiny, fixed size header to every single message, carrying a Message Type and a Message Length.
The Header, the Wire Layout, and the Dispatch Code
#pragma pack(push, 1)
// Every single message sent over the network MUST start with this header
struct MessageHeader {
uint16_t msg_type; // 2 bytes: what kind of struct is following?
uint16_t msg_length; // 2 bytes: how big is the following struct?
};
struct OrderRequest {
char type; // 1 byte
int32_t order_id; // 4 bytes
int32_t price; // 4 bytes
int32_t quantity; // 4 bytes
char side; // 1 byte
}; // Total size = 14 bytes
struct CancelRequest {
int32_t order_id; // 4 bytes
}; // Total size = 4 bytes
#pragma pack(pop)
When the client sends a new order, it transmits 18 bytes total, 4 bytes for the header plus 14 bytes for the order request body:
When bytes arrive, the server performs a two step, parsing free triage. Step 1: it reads the first 4 bytes and casts them only to the MessageHeader struct, safe because every header is identical in size and shape. Step 2: it reads msg_type, and if it equals 1 it safely casts the remaining bytes into OrderRequest, if it equals 2 it casts them into CancelRequest:
void handle_network_data(const char* network_buffer) {
// 1. Peek at the header first to identify what just arrived
const MessageHeader* header = reinterpret_cast(network_buffer);
// 2. Advance the pointer past the header to find where the body starts
const char* message_body = network_buffer + sizeof(MessageHeader);
// 3. Route the body to the correct struct based on the type ID
switch (header->msg_type) {
case 1: {
OrderRequest req;
std::memcpy(&req, message_body, sizeof(OrderRequest));
process_order(req);
break;
}
case 2: {
CancelRequest req;
std::memcpy(&req, message_body, sizeof(CancelRequest));
process_cancel(req);
break;
}
default:
// Error handling: Unknown message type received
break;
}
}
The msg_length field is my defense against network streaming issues. Because TCP is a streaming protocol, a network card might bundle several separate messages together into one single buffer read, as detailed in Section 16. By looking at header->msg_length, the server knows exactly how many bytes to skip forward to find the next message header in the stream. No layout body travels alone, every layout travels with a tiny 4 byte passport pinned to its front that announces its identity and size to the server's routing engine.
13. How SBE Avoids Parsing XML at Runtime
A natural question I had was whether SBE requires parsing that XML schema file while my trading engine is actually running. The answer is no, SBE requires absolutely zero XML parsing at runtime. The XML file is only used by the developer, on a laptop, while writing the code, and is thrown away once the C++ header it generates has been compiled in.
Build Phase vs Live Phase, and the Generated Flyweight
SBE splits the process into two separate phases:
Build Phase, happens once, on my laptop:
Live Phase, happens in production, inside my running engine:
In the Build Phase, I write the XML file defining the sequence of fields, run the SBE Compiler command line tool, and it reads the XML and writes an optimized C++ header file containing raw memory pointer offsets. Once that header file is generated, the XML file can be thrown away, it is no longer needed. In the Live Phase, my trading engine compiles the generated C++ code directly into the executable. When bytes arrive from the network, there is no XML, no strings, and no text parsing, the generated code looks at raw memory addresses exactly like my original struct cast did.
The generated code writes Flyweights, classes that just hold raw pointers, roughly like this:
// A simplified look at what the SBE-generated C++ code does:
class OrderRequestDecoder {
private:
const char* m_buffer; // Points directly to the raw network buffer
public:
void wrap(const char* buffer) {
m_buffer = buffer;
}
int32_t orderId() const {
int32_t val;
std::memcpy(&val, m_buffer + 1, sizeof(int32_t)); // Offset 1, past the type char
return val;
}
int32_t price() const {
int32_t val;
std::memcpy(&val, m_buffer + 5, sizeof(int32_t)); // Offset 5
return val;
}
};
Because the compiler optimizes away the wrapper classes, the actual machine instructions generated by SBE and by my own custom packed struct are nearly identical:
// My Struct Approach (1 CPU memory instruction):
int32_t p1 = req.price;
// SBE Flyweight Approach (1 CPU memory instruction):
int32_t p2 = decoder.price();
So why bother with SBE if the runtime speed is the same? SBE acts as an automated code generator for the exact same concept. Instead of me manually writing #pragma pack(1), manually figuring out if byte offsets are aligned, or manually handling version upgrades when a field shifts, the SBE compiler tool calculates all those byte offsets flawlessly via the XML blueprint.
14. How SBE Handles an Old Client Talking to a New Server
If I add a new field, the client code eventually needs to be updated to actually read or use it. But there is a massive difference between SBE and a custom C++ struct: SBE prevents the entire production system from crashing or corrupting data if one side updates before the other. I worked through a specific scenario to convince myself of this: I add a new field called trader_id (4 bytes) to the end of the OrderRequest layout, the server is updated to Version 2 and now expects it, and the client is still running Version 1.
Custom Struct Collapse vs SBE's Graceful Default
With a custom struct: the Version 1 client sends a 14 byte packet. The server receives 14 bytes but blindly executes std::memcpy(&req, buffer, sizeof(OrderRequestV2)), and sizeof(OrderRequestV2) is 18 bytes. The server reads 4 extra bytes of total garbage sitting next to it in memory, processes an order with a completely corrupted trader_id, and if multiple messages were packed tightly together in the TCP stream, every message after that reads scrambled values. My system would collapse instantly, in exactly the same way as the Section 7 scenario.
With SBE: when I add the field in SBE, I append it to the end of the block and tag it with a sinceVersion attribute in the XML:
When the Version 1 client sends its message, it includes a standard SBE header stating version = 1 and blockLength = 14. When the Version 2 server receives this, the generated SBE code performs an automatic safety check:
// Inside the SBE generated code on the Server:
int32_t traderId() const {
if (actingVersion < 2) {
return 0; // Automatically returns a safe, predictable default value
}
return *((int32_t*)(m_buffer + 14)); // Otherwise, safely read it from the buffer offset
}
The server does not crash, it reads the Version 1 message, realizes traderId is not there, assigns it a safe default of 0, and continues running flawlessly. The message stream stays aligned because SBE uses the blockLength from the header to know exactly how many bytes to advance to find the next message, completely preventing data alignment corruption. With a custom struct, both sides must be updated at the exact same microsecond, which is physically impossible with hundreds of clients. With SBE, servers can be updated over a weekend and clients can update over the next six months whenever they feel like it.
15. Applying Versioning to My Order Book: A Self Trade Prevention Example
To ground versioning inside my actual matching engine's data structure, which lives in memory as a highly optimized structure such as a map or array of price levels holding a linked list of resting orders, I worked through a specific feature: Self Trade Prevention. This requires every incoming order to carry an account_id, so that if an incoming buy order shares an account with a resting sell order already in the book, the engine cancels it instead of matching it against itself.
Custom Struct Catastrophe vs SBE Inside the Live Book
The server, meaning the order book engine, has been updated and now depends on reading account_id. The client, meaning an old trading bot, is old and sends orders without it.
The custom struct catastrophe inside the order book: my updated server struct now looks like this:
#pragma pack(push, 1)
struct OrderRequest {
char type; // 1 byte
int32_t order_id; // 4 bytes
int32_t price; // 4 bytes
int32_t quantity; // 4 bytes
int32_t account_id; // new field added, 4 bytes
char side; // 1 byte
}; // Total size = 18 bytes
#pragma pack(pop)
An old client sends a Version 1 order, 14 bytes tightly packed:
When the server copies those 14 bytes into its new 18 byte struct via std::memcpy, the byte offsets completely cross wires:
The server reads the last 4 bytes of the client's packet as account_id, but those bytes actually contained quantity and the side character. The side field now looks at an offset that does not exist in the packet and reads a random piece of garbage left over on the network card. The order book adds this order to the book, but because side is now a random garbage character instead of 'B' or 'S', my matching logic if (order.side == 'B') fails completely, and the order gets stuck in memory, corrupts the price index, or crashes the matching engine.
The SBE solution inside the order book: accountId is appended to the schema with sinceVersion="2". The old client's 14 byte message is flagged version 1, and the matching engine decodes it safely:
void on_order_received(const char* buffer) {
OrderRequestDecoder orderDecoder;
orderDecoder.wrap(buffer);
int32_t oid = orderDecoder.orderId();
int32_t price = orderDecoder.price();
int32_t qty = orderDecoder.quantity();
char side = orderDecoder.side();
// Because the incoming message header says Version 1, SBE automatically
// bypasses the buffer read and returns a safe fallback token (like 0)
int32_t account = orderDecoder.accountId();
if (account == 0) {
// The matching engine recognizes this is an old client and
// bypasses the Self-Trade Prevention check for this order
execute_matching_logic(oid, price, qty, side);
} else {
run_self_trade_prevention(account, oid, price, qty, side);
}
}
There is no data overlap, side is read flawlessly from its correct location. account_id defaults to 0 safely, and the matching engine does not blow up, it just treats the order as belonging to account 0. The order book stays completely clean and the price levels align perfectly, without kicking a single old client off the platform.
16. How the Kernel Actually Delivers Bytes to My Process
When raw electrical signals cross the network cable and hit the network card, the card converts those signals into binary bytes, addressed by IP address and port number. When my application calls bind() on port 8888, the Linux kernel locks that port to my process and routes every matching packet into a dedicated memory bucket called the Socket Receive Buffer, which my code later drains using recv().
The Streaming Tape, and Why Isolation Protects My Data
TCP is a Stream Protocol, not a packet protocol, behaving like a continuous water pipe. If three clients send messages, the kernel does not separate them with markers, it literally concatenates the raw bytes back to back:
When I call recv() or read() in my C++ code, I am pulling data out of that pipe into my own application's memory buffer. This is exactly why the Framing Protocol from Section 12 exists: my code looks at the first bytes of the buffer, casts them to MessageHeader, reads msg_length, knows the current message takes Header plus Body bytes total, processes those bytes, then shifts its pointer forward by exactly that many bytes to land on the start of the next message. Once my application reads bytes out of the kernel socket buffer via recv(), the kernel immediately deletes those bytes to free up space for incoming traffic.
Can another program steal my data? In a modern operating system, the answer is no, unless that program has root or administrator privileges. Modern CPUs and operating systems use Virtual Memory Space Protection. Every application lives in its own isolated sandbox:
Hardware isolation means the CPU physically prevents the malicious program from pointing its code at my app's memory, and if it tries, the CPU triggers a Segmentation Fault and kills it immediately. Kernel isolation means the kernel will only deliver data from Socket Buffer A to the exact Process ID that bound itself to port 8888. The one exception is if a malicious program gains root or admin privileges, it can put the network card into Promiscuous Mode, using a tool such as Wireshark, to sniff raw packets before they reach the socket buffers. High performance trading environments lock down, firewall, and isolate networks specifically to prevent this.
17. The Kernel Socket Buffer Is a FIFO Queue
The kernel buffer operates exactly like a FIFO queue (first in, first out). The moment my application successfully calls recv(), the kernel copies the requested bytes into my buffer and immediately pops, meaning deletes, those exact bytes from its own queue. Once popped, they are gone from the kernel forever, which is exactly why my framing logic in Section 12 has to read the precise number of bytes each header promises.
The Three States of a Pop, and the Partial Pop Edge Case
Imagine three 18 byte order messages sit in the kernel queue, 54 bytes total:
I ask for exactly one message, 18 bytes:
int bytes_received = recv(socket_fd, my_buffer, 14 + sizeof(MessageHeader), 0);
The moment recv() returns control to my program, Message 1 is wiped from the kernel buffer, and the read pointer automatically shifts forward. My next recv() call will grab Message 2.
Partial pops, an important edge case: what if 18 bytes are waiting in the kernel but my code only asks for 10 bytes?
char small_buffer[10];
recv(socket_fd, small_buffer, 10, 0);
The kernel copies the first 10 bytes of Message 1 into small_buffer and pops only those 10 bytes. The remaining 8 bytes of Message 1 stay stuck in the kernel at the front of the line. My next recv() call returns those leftover 8 bytes instead of Message 2. This is exactly why my framing logic has to be flawless: if I do not read the exact correct size dictated by my message headers, my binary offsets become permanently unaligned, and my order book starts parsing gibberish.
18. The Kernel Does Not Know What a Struct Is
The kernel does not know what a struct OrderRequest or struct CancelRequest is. To the kernel, my data is completely anonymous, just a raw sequence of bytes flowing into a single bucket. The kernel's only job is to deliver the raw bytes destined for port 8888 into my program's hands.
Where the Responsibility Actually Sits
Once the kernel hands that stream of raw bytes to my C++ program via recv(), the kernel's job is finished. It is entirely up to my C++ code to figure out which struct template applies to those bytes, which is exactly why the header with msg_type from Section 12 is necessary: my program reads the first few bytes, looks at the integer code for msg_type, and executes a switch statement to cast the bytes into the correct struct type.
19. Thread Safety Inside My Own Process
When my order book program launches, the operating system assigns it a Process. If my program has subprograms, in C++ called Threads, all of these threads live inside the exact same memory sandbox, sharing the same heap, the same variables, and the same network sockets. That sharing is convenient, but it is also the single biggest threat to data integrity inside my own application.
The Two Ways a Bad Thread Can Corrupt My Order Book
Scenario A, stealing from the socket: if a malfunctioning background thread, such as a logging thread, has access to my network socket file descriptor and blindly calls recv(), it pulls bytes directly out of the kernel socket buffer. Because recv() is a pop operation as described in Section 17, those bytes are instantly wiped from the kernel, and my main matching engine thread completely misses those orders, causing them to vanish.
Scenario B, memory corruption, the wild pointer: because all threads share the same RAM space, if a malfunctioning thread suffers from a pointer bug, such as writing past the boundary of an array, it can overwrite the memory address where my main thread just stored an OrderRequest, silently mutating a buy order into a sell order right under the matching engine's nose.
To prevent this, my engine follows a Single Threaded Execution design, also called the Actor Model:
The Network Thread is the only thread allowed to call recv() and pull data out of the kernel buffer. No other thread is given the socket handle. The Lock Free Queue is where the network thread places a pointer to the decoded OrderRequest, using a protected, blazing fast Lock Free Single Producer Single Consumer (SPSC) Queue. The Matching Engine Thread is a completely isolated thread that owns the live order book data structure, reading messages one by one from that queue with zero risk of any other thread crossing its memory space.
The kernel only routes raw bytes based on ports and has no concept of C++ structs, as covered in Section 18. Inside my process, any thread can access any memory block, so to stop a malfunctioning sub thread from corrupting or reading my data, I have to architect the code so that only a single, isolated thread is structurally permitted to touch the network buffers and the matching engine data.
20. Where the Kernel Buffer Actually Lives in RAM
The kernel buffer is not a magical piece of hidden hardware inside the network card, it lives directly inside the computer's main RAM. When the computer boots, the operating system kernel partitions RAM into two strictly isolated territories: Kernel Space and User Space, and moving data between them on every recv() call carries a real, measurable cost.
The Wall Between Kernel Space and User Space, and How to Bypass It
Kernel Space is a restricted zone: the kernel grabs a chunk of RAM for its own private use, and no standard program is physically allowed to read or write to it. The socket receive buffers live here, typically between 128 KB and a few megabytes per socket by default. User Space is the playground where all standard user programs live, including my C++ order book. When I write char buffer[1024];, that array is created in User Space RAM.
When the network card receives a packet, it copies the raw bytes into the Socket Receive Buffer inside Kernel RAM. My C++ program then executes the recv() system call, and the CPU has to stop my program, switch into a special high privilege mode, and physically copy the data across that virtual wall from Kernel RAM into my User RAM buffer. The CPU then switches back into standard mode and my program resumes. This process is called a Context Switch, and the memory copying is called a Kernel to User Space Copy. In high frequency trading, this copy operation is a real performance bottleneck, even though it happens entirely inside main memory.
The fastest firms solve this with a technique called Kernel Bypass, using specialized network cards and software libraries such as DPDK or Solarflare EF_VI:
With Kernel Bypass, the operating system steps aside entirely, and the network card is granted permission to write raw bytes directly into my C++ program's User Space memory buffer using hardware acceleration, called Direct Memory Access (DMA). There is no kernel buffer, no recv() system call, no context switch, and no memory copy in this mode.
21. How Big Can the Kernel Buffer Get, and What Happens When It Fills Up
The kernel does not borrow memory from user space, it manages RAM dynamically through the Kernel Page Allocator, and every socket has a strict cap on how much RAM it can consume so that one connection can never flood the whole machine. For my engine, this means I have to think deliberately about how large I let my socket buffer grow, and what happens if my matching loop ever falls behind the network.
Buffer Limits, the OOM Killer, and TCP Flow Control
Does the kernel limit the buffer size for a specific port? Yes. Every socket has a default cap and a hard maximum cap on its receive buffer size, viewable on Linux via /proc/sys/net/core/rmem_max. By default, a Linux socket receive buffer is usually allocated a few hundred kilobytes, for example around 212 KB. For my HFT style engine, I would use the C++ setsockopt() call with the SO_RCVBUF flag to manually force the kernel to expand port 8888's buffer to its maximum, often 16 MB or 32 MB, to handle heavy bursts of market traffic without dropping a byte. If the entire system starts running out of free memory more broadly, the kernel can shrink caches or kill low priority background applications via the Linux OOM Killer (Out Of Memory Killer) to protect itself from starving.
What happens when the buffer fills up completely? The kernel buffer acts like a physical bucket underneath a running faucet, and if I do not scoop data out fast enough, it fills to the brim. The behavior depends on whether I am using UDP or TCP.
If using UDP: UDP does not care about safety. The moment the bucket is full, any new incoming packets are instantly dropped and thrown away. My application never even knows those orders existed.
If using TCP: TCP guarantees no data is lost, handling overflow with TCP Flow Control, using a field in every TCP header called the Window Size. The server tells the client, "I currently have exactly X bytes of free space left." As my application slows down and the buffer fills, the kernel automatically shrinks the window size it reports to the client, and the exact moment the buffer hits 100 percent capacity, the server sends a packet saying
Window Size = 0. The client's operating system receives this Zero Window signal and physically freezes the client's outgoing transmission, blocking itssend()call, and the data backs up on the client's side instead.
While TCP protects me from losing data, a full kernel buffer is a disaster for an order book system: the moment it hits 100 percent and triggers a Zero Window, I am completely blind to the market while my engine is stuck clearing out old data, and prices have already moved. This is exactly why the processing loop after recv(), described in Section 19, has to be incredibly fast and lightweight, storing binary data instantly and deferring complex logic, to keep the kernel buffer as close to 0 percent full as possible.
22. Why Text Formats Are Slow: A CPU Level Walkthrough of JSON, XML, and FIX
Section 10 already summarizes why I rejected text formats: a 50 byte JSON order message may need 500 CPU instructions to parse. This section is the detailed version of that claim, walking through exactly what the CPU does, instruction by instruction, for JSON, XML, and FIX when it tries to read a price value, since the CPU only ever receives raw ASCII character bytes, never integers.
JSON, XML, and FIX, Step by Step
JSON, example message {"price":10050}:
The scanning loop: the CPU starts at byte 0 (
{) and steps forward byte by byte, checking every character to find a quotation mark indicating the start of a key.String extraction and memory allocation: it encounters
p,r,i,c,e, and a typical parser allocates temporary heap space (std::string) or creates a hash map key to hold these characters.Key matching, branching: the parser compares the extracted string
"price"against the requested field name, requiring string comparison loops (strcmp) and conditional branches inside the CPU.Locating the value: it skips the colon and finds the character
1.Text to integer conversion math loop: the CPU cannot do math with the character
'1'(ASCII byte value 49), so it runs a sequential loop, such asatoiorstd::stoi, computing((((0×10+1)×10+0)×10+0)×10+5)×10+0=10050.Closing up: the CPU continues scanning until it hits a comma or closing brace to know the number has ended.
XML, example message
State machine tracking: the parser scans for
<, loops throughp,r,i,c,euntil it finds>, and logs that it has entered a node namedprice.DOM tree building: most standard XML parsers build a memory tree (Document Object Model), constantly executing
newormallocto build parent child node pointers, which trashes the CPU cache.Extracting inner text: the parser encounters
10050and isolates this string slice from the surrounding tags.Validation check: the parser must look ahead, find the closing tag
, and string match it against the opening tag, throwing an error if it does not match.Text to integer conversion: the isolated string
['1','0','0','5','0']is passed through the same multiplication and addition loop as JSON.
FIX (Financial Information eXchange), example message 44=10050\x01, where tag 44 is the institutional trading tag for price and \x01 is the SOH control character delimiter. FIX is faster than JSON and XML because it throws away descriptive text keys and replaces them with raw numbers, but it is still a text format:
Tag indexing loop: the CPU reads
4,4and runs a small math loop just to parse the tag number itself from text into the integer44.Delimiter hunting: it scans forward looking for the
=sign to know the tag number has ended and the value is beginning.Value scanning: it loops character by character through
10050looking for the hidden separator character\x01.Routing, the big switch: once tag
44is extracted, it is passed into a largeswitch(tag_id)block, creating heavy branch prediction penalties as the hardware tries to guess which code path to jump to next.Text to integer conversion: the same conversion math loop runs on the value bytes.
The core problem common to all three: text formats force the CPU to behave like a detective, hunting through memory, guessing where things end, building structures dynamically, and manually calculating numbers from text digits. This causes two penalties: Branch Mispredictions, where loops and if statements cause the CPU's internal pipeline to constantly guess wrong and clear its cache, and Data Cache Misses, where hopping through text lookups forces the CPU to constantly fetch from slow main RAM instead of ultra fast L1 or L2 cache, covered in Section 36.
The binary contrast, my approach for the same field, wire message [ 130, 39, 0, 0 ] (the 4 byte hex representation of 10050): the CPU executes a single instruction, "read 4 bytes of memory located exactly at buffer + 5 and put it directly into CPU register EAX." There are no loops, no character tracking, no delimiter searching, no string allocations, and no math conversions.
Metric | My Binary Struct Approach | JSON Key Value Approach |
|---|---|---|
How the CPU finds | Instant offset, look exactly 5 bytes from the start | Search loop, scan character by character until finding |
CPU instructions required | 1 instruction, a single memory read | 500 or more instructions, loops, branches, character matching |
Time taken | About 0.5 nanoseconds | About 500 to 2000 nanoseconds |
Memory footprint | Fixed 14 bytes | 60 or more bytes of text plus temporary RAM overhead |
23. SBE Reality Check: What My Live Engine Actually Sees on the Wire
To finally resolve any lingering confusion between SBE's XML configuration and its runtime behavior, I compared what actually happens in RAM at the exact microsecond a message arrives, since Section 13 already showed the build phase but not the byte for byte comparison against a true text protocol.
XML on the Wire vs SBE on the Wire
In a true XML text protocol, the raw text bytes arriving over the network would look like
In SBE, the raw bytes arriving over the network look like [ 130, 39, 0, 0 ], the 4 byte binary integer for 10050. There is no text, no brackets, and no XML on the wire. When my code calls:
int32_t price = orderDecoder.price();
the SBE tool already read the XML blueprint weeks ago on my laptop and hardcoded the exact memory offset into the generated function. The compiler turns this into one assembly instruction:
MOV EAX, [RSI + 5] ; Load 4 bytes directly from memory offset 5 into the CPU register
If the runtime instruction is identical to a hand written struct, why use the XML blueprint at all? It exists entirely for engineering convenience and safety:
Challenge | My Custom Packed Struct | SBE (Using the XML Blueprint) |
|---|---|---|
Figuring out memory alignment | I must manually calculate byte offsets and use | The SBE compiler reads the XML and calculates exact byte offsets automatically |
Supporting multi language clients | If a client wants to write a bot in Java, Rust, or C#, I must manually translate my C++ layout by hand | I hand the XML blueprint to the client, they run the SBE compiler for their language, and it generates matching, native, zero parsing code |
System upgrades, adding fields | If I add a field, all memory offsets shift, and forgetting to update one internal service causes corrupt reads, as in Section 9 | I add the field to the XML with |
With text XML protocols, the XML tags travel across the wire and the CPU must parse them live. With SBE, the XML file is only a set of instructions for a code generation script on my laptop, and the wire only ever carries raw binary numbers.
24. Random Access Memory: Why the CPU Can Teleport to Any Offset
The remaining question I had was: inside the kernel buffer, there is no field labeled "price", just a continuous, nameless conveyor belt of binary data. How can orderDecoder.price() jump straight to the middle of that raw buffer and extract the correct 4 bytes without scanning anything else? The answer is that RAM is Random Access Memory: the CPU does not need to scan memory left to right, if it knows the exact memory address it can teleport instantly to any specific byte.
The Exact Arithmetic Behind the Teleport
When the network card dumps a message into my buffer, it lays the binary data down in an exact, predictable sequence:
When my XML file was passed to the SBE compiler, the compiler did simple arithmetic: type starts at offset 0 and takes 1 byte, order_id must start right after, so its offset is 0+1=1 and it takes 4 bytes, price must start right after order_id, so its offset is 1+4=5. That magic number 5 is hardcoded directly into the generated C++ source. When my code executes int32_t price = orderDecoder.price();, the CPU does not look at type or order_id, it performs instant math: target address equals base address plus offset, 0x1005 = 0x1000 + 5, and executes a single memory read at that address, translated into machine code as:
MOV EAX, [RSI + 5]
(RSI holds the base network buffer address 0x1000. Add 5 to get 0x1005. Take the 4 bytes sitting at that location and load them into the EAX register.)
JSON is sequential, or scanning: the CPU does not know where price is because the spacing changes depending on text length, so it must scan every byte left to right until it spots the letters p-r-i-c-e, as detailed in Section 22. SBE and my struct are random access, or teleportation: positions are set in stone at compile time, and the CPU instantly calculates the exact coordinates of the price field and grabs it in less than a nanosecond, completely ignoring the surrounding data.
25. My Two Alternatives to SBE
As Section 10 states directly from my project notes, I rejected schema compilers such as Protocol Buffers and FlatBuffers for the same reason I rejected text formats, though for the opposite technical reason: they are fast, but they hide the underlying mechanics, and this project exists specifically so I understand every layer myself. If I do not use SBE or a similar tool, I am back to managing raw byte layouts myself, and I have two possible paths.
Custom Struct vs Manual Pointer Arithmetic
Alternative A, stick with my custom packed struct:
#pragma pack(push, 1)
struct OrderRequest {
char type; // Offset 0
int32_t order_id; // Offset 1
int32_t price; // Offset 5
int32_t quantity; // Offset 9
char side; // Offset 13
};
#pragma pack(pop)
OrderRequest req;
std::memcpy(&req, buffer, sizeof(OrderRequest));
int32_t price = req.price; // The compiler uses offset 5 under the hood
What I have to manage myself: brittle deployments, since a new field forces every client to update at the exact same microsecond or the system reads garbage, as covered in Section 9, and no automatic translation to other languages such as TypeScript or Python, which I would have to code by hand.
Alternative B, manual pointer offset arithmetic:
// I hardcode the offsets as constants in my code
const int TYPE_OFFSET = 0;
const int ORDER_ID_OFFSET = 1;
const int PRICE_OFFSET = 5;
int32_t get_price(const char* buffer) {
int32_t price;
std::memcpy(&price, buffer + PRICE_OFFSET, sizeof(int32_t));
return price;
}
What I have to manage myself: human error, since typing const int PRICE_OFFSET = 4; instead of 5 introduces a silent alignment bug, and a maintenance nightmare if I have dozens of message types, each requiring thousands of lines of fragile offset constants where one typo collapses the entire data stream.
Both paths compile down to the exact same high performance machine instruction, MOV EAX, [RSI + 5], as shown in Section 24. The CPU does not care whether that offset was calculated by my brain inside a custom struct, written by hand as a constant, or generated by an SBE, Protocol Buffers, or FlatBuffers compiler. For a project like mine, where I control both the client and the server and specifically want to learn every layer rather than delegate it to a schema compiler, my custom packed struct approach is perfect and runs at the absolute speed limit of the hardware, as long as I am meticulous whenever I change the layout of my fields.
26. Why the CPU Loves Multiples of 4 and 8, and Hates 3
A CPU's memory controller reads data from RAM using a fixed physical highway called the Data Bus, 8 bytes wide on modern 64 bit processors, and it can only pull data across memory addresses that align to that highway's physical boundaries. This is the hardware reason behind the padding trap in Section 2, and it is also the reason the fix is not simply "always add padding," since padding costs network bandwidth.
Aligned vs Misaligned, and the Optimal Field Ordering Fix
Physical 4-Byte Memory Lanes in RAM:
Byte 0 Byte 1 Byte 2 Byte 3 <- Lane 1 (Address 0)
Byte 4 Byte 5 Byte 6 Byte 7 <- Lane 2 (Address 4)
The CPU can grab Lane 1 (addresses 0 to 3) or Lane 2 (addresses 4 to 7) in a single hardware cycle. It cannot grab addresses 1 to 4 or 3 to 6 in one shot, because those cross the physical lanes. Why not a multiple of 3? Computer hardware is binary, base 2. Shifting numbers or addresses by powers of 2 takes a single, instantaneous bit shift instruction inside the silicon. A multiple of 3 requires complex division and multiplication circuitry, which would drastically slow down the hardware.
Scenario A, aligned data with padding: a struct with a 1 byte char and a 4 byte int, with 3 bytes of padding so the integer lands perfectly at offset 4:
The CPU activates Lane 2, the 4 bytes drop directly into the register in 1 clock cycle, and the CPU immediately does math with it.
Scenario B, misaligned data, packed tight: using #pragma pack(1) with no padding, the same integer starts at offset 1:
The integer is now split across two physical lanes, and the CPU has to execute a multi step rescue mission, detailed fully in Section 32. Aligned data takes 1 memory fetch and 0 assembly cleanup instructions. Misaligned data takes 2 memory fetches plus bit shifting and bit masking arithmetic. On certain ARM architectures, such as routers, smartphones, or AWS Graviton servers, trying to read a misaligned integer directly can trigger a hardware exception and crash the program instantly.
The HFT dilemma: using padding gives the CPU maximum speed but inflates my message from 14 bytes to 20 bytes, wasting network bandwidth sending empty padding. Removing padding keeps the message a tight 14 bytes but forces the CPU to bit shift. The solution is Optimal Field Ordering, rearranging fields from largest type to smallest type:
// PERFECTLY PACKED AND PERFECTLY ALIGNED (No padding bytes needed!)
struct OrderRequest {
int32_t price; // 4 bytes, starts at 0, aligned to 4
int32_t order_id; // 4 bytes, starts at 4, aligned to 4
int32_t quantity; // 4 bytes, starts at 8, aligned to 4
char type; // 1 byte, starts at 12
char side; // 1 byte, starts at 13
}; // Total Size: 14 bytes.
By designing the sequence so every 4 byte integer naturally lands on a multiple of 4, the compiler adds zero padding, and the CPU can teleport to the fields in a single clock cycle, exactly as shown side by side in Section 31.
27. Why the CPU Cannot Simply Skip a Byte During a Fetch
If the CPU already knows the integer is located at addresses 1, 2, 3, and 4, why can it not just grab those exact bytes directly instead of shifting things around afterward? The answer is a hardware limitation: the CPU's execution units can only perform math on data that is aligned to the zero position inside a register, so any field that starts off center has to be dragged back into position before it is usable.
Register Slots and the Barrel Shifter, Byte by Byte
A CPU register is a rigid, physical array of transistors, for example a 4 byte (32 bit) register with 4 fixed slots. When the CPU executes a math instruction, the ALU (Arithmetic Logic Unit) is hardwired to look at Slot 0 for the lowest part of the number, Slot 1 for the next, and so on. It cannot do math if the number is sitting off center.
Walking through my misaligned order_id example, split across two physical lanes:
Step 1, the CPU reads Lane 1 into a temporary staging register: the hardware dumps all 4 bytes into a staging area, and because it is a fixed lane read, Char is stuck in Slot 0.
Step 2, the CPU reads Lane 2 into a second staging register: the rest of the integer, Int_B4, lands in Slot 0 of this second register.
Step 3, the need for bit shifting: the integer is split across two registers with bytes completely out of alignment. If I did math with Staging Register A directly, the CPU would accidentally include the Char byte, destroying my data. To fix this, the CPU passes the data through a physical circuit called a Barrel Shifter: it shifts Staging Register A to kick out Char and align Int_B1 with Slot 0, shifts Staging Register B to slide Int_B4 up into the top slot, and merges the two shifted results together with a bitwise OR, exactly as walked through cycle by cycle in Section 32.
The CPU does know where the data starts and ends. But because the wires connecting RAM to CPU registers can only transfer data in rigid, block aligned lanes, the CPU is physically incapable of skipping the first byte during a hardware RAM fetch. Bit shifting is not for locating the data, it is an assembly line cleanup process required to shift the data squarely into the center of the CPU's mathematical calculators.
28. What Happens When My 14 Byte Message Actually Arrives at the Server
Sending a 14 byte packed message gives minimum network latency, since no empty padding bytes are wasted on the wire, followed by careful buffer alignment handling on the server once it lands in RAM.
DMA, the 64 Byte Cache Line, and Cache Aligned Inflation
Step 1, the network card transits the 14 bytes: using a hardware mechanism called DMA, the network card writes those 14 bytes directly into my server's main RAM, and the network card always starts writing a new packet at an address aligned to a multiple of 64 bytes, a CPU Cache Line boundary, covered fully in Section 36. Even though my message is only 14 bytes, its starting position in my receive buffer is guaranteed to be perfectly aligned:
Step 2, how the CPU reads the fields, the packing penalty: because the buffer is tightly packed with no internal padding, reading type at offset 0 is 1 clock cycle. Reading order_id at offset 1 is misaligned, since 1 is not a multiple of 4, triggering the barrel shifter from Section 27. Reading price at offset 5 and quantity at offset 9 are also unaligned, both triggering the same hardware bit shifting routine. Modern Intel and AMD CPUs have advanced misalignment hardware units that hide this cost in silicon, so it does not crash, but it burns processing power and efficiency for every field looked up.
How the pros avoid the shifting penalty, Cache Aligned Inflation: instead of reading fields directly out of the misaligned network buffer, a technique used inside a single threaded processing loop is to allocate a clean, local struct on the CPU stack, ordered so it has zero padding but perfect alignment, then let the CPU copy and rearrange the 14 bytes into that perfectly aligned local struct:
// 1. The 14-byte misaligned packet is sitting in your network buffer
const char* net_buffer = get_network_packet();
// 2. Allocate a clean, local struct on the CPU Stack with optimal ordering
struct AlignedOrder {
int32_t price; // Offset 0 (Aligned!)
int32_t order_id; // Offset 4 (Aligned!)
int32_t quantity; // Offset 8 (Aligned!)
char type; // Offset 12
char side; // Offset 13
};
AlignedOrder local_order;
// 3. Inflate/re-arrange the 14 bytes into the perfectly aligned local struct
local_order.price = *((int32_t*)(net_buffer + 5));
local_order.order_id = *((int32_t*)(net_buffer + 1));
By doing this, the bit shifting penalty is absorbed exactly once, right at the doorstep of the application. Once the data is copied into the perfectly aligned local struct, the live order book matching logic can read prices, change quantities, and execute matching calculations thousands of times over at the absolute maximum speed of the hardware, with zero bit shifting loops holding it back.
29. Sending a Tight 14 Byte Message Without SBE
The network wire only carries raw, anonymous bytes, it has no idea whether those 14 bytes came from SBE, a custom C++ struct, a Python script, or a hand written array of characters. Since I do not use SBE in this project, I settled on one of two equally valid ways to send and receive that raw, tightly packed layout in pure C++.
Two Equivalent Ways to Send the Same 14 Bytes
Method 1, using #pragma pack(1), the struct approach:
#pragma pack(push, 1)
struct OrderRequest {
char type; // 1 byte, offset 0
int32_t order_id; // 4 bytes, offset 1
int32_t price; // 4 bytes, offset 5
int32_t quantity; // 4 bytes, offset 9
char side; // 1 byte, starts at 13
}; // Total size = exactly 14 bytes
#pragma pack(pop)
OrderRequest my_order = {'N', 101, 10050, 10, 'B'};
send(socket_fd, &my_order, sizeof(OrderRequest), 0); // Sends exactly 14 bytes
Method 2, manual serialization, the byte buffer approach, useful if I am worried about different operating systems handling packing differently:
char outbound_buffer[14];
char type = 'N';
int32_t order_id = 101;
int32_t price = 10050;
int32_t quantity = 10;
char side = 'B';
std::memcpy(outbound_buffer + 0, &type, 1);
std::memcpy(outbound_buffer + 1, &order_id, 4);
std::memcpy(outbound_buffer + 5, &price, 4);
std::memcpy(outbound_buffer + 9, &quantity, 4);
std::memcpy(outbound_buffer + 13, &side, 1);
send(socket_fd, outbound_buffer, 14, 0); // Sends exactly 14 bytes
If I looked at these network packets using a tool such as Wireshark, a 14 byte message sent using SBE, Method 1, or Method 2 would look 100 percent identical on the wire, the exact same 14 bytes shown in the diagram in Section 6.
So why would anyone choose SBE if the wire data is the same? It comes back to backward compatibility, exactly as in Section 14 and Section 15. Without SBE, if I add account_id next month, my 14 byte layout becomes an 18 byte layout, and the moment my updated server reads a 14 byte packet from an old client, the memcpy or struct cast crosses memory lines and crashes my matching engine. With SBE, the exact same 14 bytes travel the wire, but because SBE auto generates safety wrappers around the offsets based on message version, my updated server can safely process a 14 byte packet from an old client and an 18 byte packet from a new client simultaneously without a single instruction misaligning. For a controlled, solo project where I write both the client and the server myself, skipping SBE and using Method 1 is completely fine, fast, and efficient.
30. Do I Need to Add Padding Myself
I do not need to add padding if I do not want to. If I pack my data tightly into 14 bytes, my code works perfectly, the network card sends it, and my order book matches trades cleanly. The only reason to add padding is to optimize for raw CPU execution speed, and it comes down to a classic trade off: network speed versus CPU speed.
The Trade Off Table, and the Cheat Code
Choice | Wire Size | CPU Shifting Penalty | Best Used For |
|---|---|---|---|
Tightly packed, no padding, 14 bytes | 14 bytes, minimum transit time | Yes, the CPU must use its barrel shifter on every field read | WAN or internet connections where bandwidth is the main bottleneck |
Word aligned with padding, 16 to 20 bytes | 16 to 20 bytes, slightly larger transit time | No, the CPU teleports to fields and loads them in 1 cycle | Co located HFT servers inside the exchange building, where network speed is instant and every picosecond of CPU matching time counts |
The cheat code: I can have both, sending 14 bytes over the wire with zero padding while still letting the CPU read the fields with zero bit shifting penalties, by arranging struct fields from largest type to smallest type, exactly as introduced in Section 26:
// 14 BYTES TOTAL - ZERO PADDING WASTED ON THE WIRE
// BUT EVERY SINGLE INTEGER LANDS ON A MULTIPLE OF 4!
struct OptimizedOrderRequest {
int32_t price; // 4 bytes, starts at offset 0, aligned
int32_t order_id; // 4 bytes, starts at offset 4, aligned
int32_t quantity; // 4 bytes, starts at offset 8, aligned
char type; // 1 byte, starts at offset 12
char side; // 1 byte, starts at offset 13
}; // Total size = 14 bytes
Because price, order_id, and quantity are all 4 byte integers landing on offsets 0, 4, and 8, which are all clean multiples of 4, the CPU hardware does not need any bit shifting or masking, and the total size remains exactly 14 bytes, so no network bandwidth is wasted on empty padding.
31. Three Layouts Side by Side: No Padding, Injected Padding, Smart Ordering
To see the definitive, visual difference, here is how my 14 bytes arrange across the physical 4 byte memory lanes for three variations of the same struct: no padding, injected padding, and smart ordering.
The Three Layouts, Side by Side
1. No padding (#pragma pack(1)), original field order (type, order_id, price, quantity, side): total wire size 14 bytes, but critically penalized, since order_id, price, and quantity all bleed across lane boundaries, forcing multiple memory cycles and bit shifting arithmetic:
2. Injected padding, default compiler alignment, original field order: total wire size 20 bytes, wasting 6 bytes of bandwidth sending empty zeroes, but maximum CPU speed, since the compiler forces 3 bytes of dead space right after type so order_id lands perfectly at byte 4:
3. Smart ordering, the HFT solution, fields rearranged largest to smallest, price, order_id, quantity, type, side: total wire size 14 bytes, zero bytes wasted, and maximum CPU speed, since the integers start at bytes 0, 4, and 8, naturally aligning with the hardware lines with zero padding:
Feature | 1. No Padding | 2. Injected Padding | 3. Smart Field Ordering |
|---|---|---|---|
Bytes sent over wire | 14 bytes, ultra lightweight | 20 bytes, bloated | 14 bytes, ultra lightweight |
CPU memory fetches | Two reads per integer field | One single read per integer field | One single read per integer field |
Hardware bit shifting | Required, consumes CPU cycles | None, instant execution | None, instant execution |
Overall efficiency | Good for internet bandwidth, bad for CPU matching speed | Bad for network latency, excellent for CPU matching speed | Perfect network efficiency paired with perfect CPU execution speed |
By simply ordering fields from largest data size to smallest data size, I gain absolute alignment speed inside the CPU registers while maintaining an ultra lean 14 byte footprint on the network. This is the exact rule I apply in Section 41, the final struct my engine uses.
32. The Micro Level Life of a Misaligned Read
To see exactly how the transistor circuits process my unpadded layout, I traced through order_id, a 4 byte integer sitting at offsets 1, 2, 3, and 4 in my raw memory buffer, which is the exact field flagged as misaligned back in Section 26 and Section 27.
The Full Clock by Clock Timeline
The component that talks to RAM is called the Load-Store Unit (LSU). Attached to it is a specialized circuit called the Barrel Shifter. The RAM cache delivers data in rigid 4 byte physical lanes:
My 4 byte order_id is trapped: its first 3 bytes sit at the end of Lane A, and its final byte sits at the start of Lane B. Since a single assembly read instruction cannot handle this, the CPU's internal microcode splits it into a series of micro operations:
Clock Cycle 1, fetching the first fragment: the LSU activates the address lines for Lane A, and all 4 bytes rush across the data bus into Staging Register 1:
Here type is unfortunately sitting in Slot 0, right alongside 3 of the 4 order_id bytes.
Clock Cycle 2, fetching the second fragment: the LSU requests Lane B, and its bytes land in Staging Register 2:
Clock Cycle 3, the micro cleanup, bit shifting and masking: the CPU has all the pieces, but they are a mess, with type cluttering the first register and parts of price cluttering the second. Both staging registers are routed through the Barrel Shifter: Register 1 is shifted right by 8 bits (1 byte) to flush out type and slide the three order_id bytes down; Register 2 is shifted left by 24 bits (3 bytes) to wipe out the unrelated price bytes and force the lonely order_id_4 byte up into the highest slot. The results are then merged through an AND/OR gate array, covered in full in Section 33.
Clock Cycle 4, delivery to the calculator: the reconstructed, 4 byte integer lands cleanly inside the visible CPU register:
Only now can the execution unit perform math on it.
By contrast, in the smart field ordering approach from Section 26, order_id sits cleanly inside its own physical lane without bridging a boundary: Clock Cycle 1, the LSU activates the data lines for its clean lane, and the 4 bytes drop instantly into the final register in their correct slots; Clock Cycle 2, the CPU is already calculating trades.
While the LSU is playing detective and piecing unaligned integer fragments back together over multiple clock cycles, the CPU's primary arithmetic calculator sits completely idle, waiting for its data to show up.
33. How the CPU Discards Unwanted Bytes: Shifting and Masking
The CPU is physically incapable of saying "just send me bytes 1, 2, and 3," since the wiring on the motherboard and inside the silicon is like a train track that only stops at fixed stations, as introduced in Section 26. Because it can only load in multiples of 4 or 8 bytes, it is forced to bring unrelated data, such as the type byte or parts of price, along for the ride into its internal staging registers, and it has to actively discard that unwanted data before it can do math.
The Shift, and the Mask
Phase 1, kicking out the unrelated data, the shift: when Lane A arrives in Staging Register 1, the bytes look like [order_id_3] [order_id_2] [order_id_1] [type]. The CPU passes this through a Shift Right by 8 bits instruction, which in hardware is literally just wires angled to the right, pushing the type byte off the cliff at the end, while zeroes feed into the top:
Phase 2, protecting against residual garbage, the mask: to guarantee absolutely zero contamination from any other unrelated bytes, the CPU's control unit can also apply a hardware bitmask, forcing the data through an array of AND logic gates using a specific binary filter, for example 0x00FFFFFF to discard the top byte and ensure it is pure zero:
Shifted Data: 01011010 [order_id_3] [order_id_2] [order_id_1] (garbage that may have leaked into the top)
Bitwise MASK: 00000000 11111111 11111111 11111111 (0x00FFFFFF)
AND Gate Out: 00000000 [order_id_3] [order_id_2] [order_id_1]
Anywhere the mask has a 0, the transistor gate shuts off, forcing that byte to become 00000000, discarded. Anywhere the mask has a 1, the gate stays open, allowing the precious order_id bytes to pass through untouched.
Because the hardware can only move data in rigid 4 byte or 8 byte blocks, it wastes electricity and data bus bandwidth loading bytes it does not want, and it wastes execution time running shift and mask operations to clean up and throw away that irrelevant data. Smart field ordering, from Section 26, eliminates the cleanup crew entirely, and 100 percent of the bytes that enter the register are exactly the ones asked for.
34. The Real Cost in Clock Cycles
In a regular software application, a 3 to 4 clock cycle delay is imperceptible to a human. But in high frequency trading, the entire matching loop needs to execute in nanoseconds, so the exact clock cycle cost of the misaligned read from Section 32 and Section 33 is worth counting precisely.
The Cycle Count, and the Nanosecond Math
With smart ordering or injected padding, aligned: Cycle 1, load the 4 bytes straight into the destination register; Cycle 2, done, the CPU immediately starts executing matching engine logic.
With no padding, original field order, misaligned: Cycle 1, fetch Lane A from memory or cache, bringing along the irrelevant type byte; Cycle 2, fetch Lane B, bringing along irrelevant price bytes; Cycle 3, the discarding cycle, pass Lane A through the shifter to discard type; Cycle 4, the discarding cycle, pass Lane B through the shifter to discard price; Cycle 5, merge the cleaned fragments into the final register; Cycle 6, done, now the matching engine logic can execute.
Modern CPUs run at around 4 to 5 GHz, meaning 1 clock cycle takes roughly 0.2 nanoseconds. If I have to waste 3 or 4 extra clock cycles just to discard irrelevant bytes every single time I read a price, an order ID, or a quantity, I am burning approximately 0.8 nanoseconds of pure waste per field. If my order book processes millions of messages a second, those fractions of a nanosecond compound into massive latency delays. While my CPU is busy running logic gates to scrub away irrelevant data, a competitor with perfectly aligned memory has already processed the packet and stolen the trade ahead of me. Padding, or smart ordering, is a trick to bypass the discarding cycle. By making sure every field sits perfectly at a multiple of its own size, 100 percent of the data the CPU grabs is relevant, allowing it to skip the cleanup phase entirely.
35. Scaling the Problem Up to an 8 Byte Field
An 8 byte data type, such as int64_t or a double for fractional prices, scales the exact same hardware logic from Section 26 up to a larger physical highway. On all modern 64 bit processors, the internal data bus and the L1 cache lines are physically wired to move data in 8 byte lanes (64 bits) at a time, which changes the size of the penalty when a field is misaligned.
Aligned vs Misaligned at 8 Bytes
Aligned scenario, multiple of 8: if my 8 byte integer sits at an address that is a perfect multiple of 8, for example address 0, 8, 16, or 24, it sits perfectly inside a single physical lane:
Clock Cycle 1, the LSU fires electricity down the address lines for the lane, the hardware moves all 8 bytes simultaneously across the 64 bit wide data bus, dropping them instantly into a 64 bit register such as RAX, with zero bits shifted and zero bytes discarded, ready for math on Cycle 2.
Misaligned scenario, the split lane penalty: if I pack my data tightly without padding and my 8 byte integer lands on an odd address such as offset 4, because the hardware only recognizes boundaries at 0 and 8, my 8 byte integer is split perfectly in half across two distinct physical lanes:
Clock Cycle 1, fetch Lane 1, forced to load 4 bytes of completely unrelated data just to get Int_B1 through Int_B4; Clock Cycle 2, fetch Lane 2, again loading 4 bytes of unrelated garbage to get Int_B5 through Int_B8; Clock Cycle 3, discarding cycle for Lane 1, routing Fragment 1 through a 64 bit Barrel Shifter, shifting right to discard the unrelated data and sliding Int_B1...B4 into the bottom slots; Clock Cycle 4, discarding cycle for Lane 2, shifting left to discard the top unrelated bytes and sliding Int_B5...B8 into the upper slots; Clock Cycle 5, the merge, passing both cleaned registers through a bitwise OR gate array, using the same shift and mask mechanics detailed in Section 33.
When a 4 byte integer was misaligned, it usually only bled 1 or 2 bytes over a boundary. When an 8 byte integer is misaligned, it almost always shatters into two massive pieces across the 8 byte lane grid, causing double the cache traffic and forcing the 64 bit shifter circuitry to run longer cleanup routines. The solution is the same Smart Field Ordering rule from Section 26, adapted: always put 8 byte fields at the absolute top of the structure:
struct UltimateHFTRequest {
int64_t high_res_timestamp; // 8 bytes, starts at 0, perfect 8-byte alignment
int32_t price; // 4 bytes, starts at 8, perfect 4-byte alignment
int32_t order_id; // 4 bytes, starts at 12, perfect 4-byte alignment
char type; // 1 byte, starts at 16
char side; // 1 byte, starts at 17
}; // Total size = 18 bytes. Zero padding bytes wasted, zero CPU discarding cycles.
36. Where the Data Lives After That: The L1 Cache
Main RAM is far too slow for an HFT engine, taking about 60 to 100 nanoseconds to read from. Instead, the moment I access any byte in my message, the CPU automatically grabs a massive 64 byte chunk of memory surrounding it and loads it into the ultra fast L1 Data Cache built right next to the execution core, which is exactly the 64 byte alignment boundary referenced back in Section 28.
The Cache Line, and Why the Second Field Is Nearly Free
This 64 byte chunk is called a Cache Line:
The L1 cache holds onto the entire 64 byte line. If my message is 14 bytes, the whole message, plus whatever data happens to be sitting next to it in RAM, is stored in the L1 cache instantly. When I read price right after reading order_id, the CPU does not go back to RAM, it checks the L1 cache, sees the 64 byte line is already there, a Cache Hit, and pulls the data in less than 1 nanosecond.
37. The 64 Bit Sub Register Trick
On a true 64 bit architecture, which is what all modern Intel, AMD, and ARM servers are, the physical lanes are strictly 8 bytes (64 bits) wide, which changes the math for how a 4 byte integer such as price is handled compared to the simpler 4 byte lane model in Section 26.
Case A, Case B, and the 64 Bit Rule of Thumb
Case A, the 4 byte integer naturally inside an 8 byte lane: imagine price sits at offset 8, a perfect multiple of 8, so it sits at the start of physical Lane 2:
When I read that 4 byte integer, the CPU fetches all 8 bytes of Lane 2 in one cycle, and even though it fetched 8 bytes, the 64 bit hardware automatically routes just the lower 4 bytes directly into my 32 bit register, ignoring the top 4 bytes. There is no discarding cycle here, because the field started exactly at the lane boundary.
Case B, the 4 byte integer crosses the 8 byte lane boundary: if price lands at offset 6, two bytes sit at the end of Lane 1 and two bytes bleed into the start of Lane 2:
Even on a powerful 64 bit machine, this triggers the exact same multi cycle rescue mission from Section 32: fetch Lane 1, fetch Lane 2, shift Lane 1 left, shift Lane 2 right to discard the unrelated data cluttering both lanes, then merge.
The 64 bit rule of thumb: on a 64 bit architecture, the hardware highway moves 8 bytes at a time, an 8 byte variable (int64_t) must land on a multiple of 8 to avoid the discarding cycle, and a 4 byte variable (int32_t) must land on a multiple of 4 to avoid the discarding cycle, as long as it fits entirely inside one of those 8 byte halves, the 64 bit hardware cuts it clean in a single cycle. This is why Smart Field Ordering, putting 8 byte types first, then 4 byte, then 2 byte, then 1 byte, is the gold standard for 64 bit HFT engines, covered fully in Section 38.
38. Do I Need Padding to Make a 4 Byte Field 8 Byte Aligned
I do not need to add an extra 4 bytes of padding to make a 4 byte field 8 byte aligned. This is exactly the spot where developers get tripped up: they think if the physical lanes are 8 bytes wide, then every 4 byte field needs to be a multiple of 8. But a 64 bit CPU has a built in feature that handles 4 byte integers completely penalty free, as long as they land on a multiple of 4, exactly as previewed in Section 37.
Slot A, Slot B, and the Final Struct
A physical 8 byte lane is divided into two equal 4 byte slots, Slot A and Slot B, with dedicated copper wiring cutting the lane exactly in half, so the CPU can read either slot independently in one single clock cycle with zero shifting.
Imagine order_id followed immediately by price, together taking exactly 8 bytes:
order_id starts at offset 0 (a multiple of 4) and fills Slot A perfectly. price starts at offset 4 (a multiple of 4) and fills Slot B perfectly. When my code asks for price at offset 4, the CPU fetches the entire Lane 1 in 1 clock cycle, and the hardware wires automatically route Slot B directly into the register while completely ignoring Slot A. Total cost, 1 clock cycle, padding added, 0 bytes.
When does a 4 byte integer face a penalty on a 64 bit system? Only if it starts on an offset that crosses that middle line or the outer boundaries, for example starting at offset 6, which shatters it across the boundary between Lane 1 and Lane 2, exactly as shown in Case B of Section 37.
The verdict: I do not need to inflate my message size with 4 byte padding blocks to satisfy a 64 bit CPU. Using Smart Field Ordering, the compiler naturally packs 4 byte fields two by two into 8 byte lanes:
struct Perfect64BitStruct {
int64_t timestamp; // 8 bytes, starts at 0, fits Lane 1 perfectly
int32_t order_id; // 4 bytes, starts at 8, fits Lane 2 Slot A perfectly
int32_t price; // 4 bytes, starts at 12, fits Lane 2 Slot B perfectly
char type; // 1 byte, starts at 16
char side; // 1 byte, starts at 17
}; // Total size = 18 bytes.
In this layout, every field can be read by a 64 bit CPU in exactly 1 clock cycle, without wasting a single byte of network bandwidth on empty padding.
39. Why a 1 Byte Field Never Suffers a Penalty
Reading a 1 byte field, such as type or side, is the easiest job the CPU can get on a 64 bit system, always taking exactly 1 clock cycle regardless of what offset it sits on, unlike the 4 byte and 8 byte fields covered in Section 26 and Section 35.
Why a Single Byte Can Never Straddle a Boundary
Imagine type and side sitting in Lane 3, handling offsets 16 to 23:
When my code requests type at offset 16, or side at offset 17, the CPU executes a single 1 cycle memory fetch: the Load-Store Unit pulls all 8 bytes of Lane 3 out of the L1 cache and flings them across the 64 bit data bus into the execution core, and because it is a 1 byte read, the hardware has dedicated logic lines wired directly to each individual byte slot, instantly routing just that single byte into the register while ignoring the other 7 bytes.
A 64 bit register such as RAX is nested like Russian dolls, accessible as a full 64 bit register, a 32 bit register (EAX), a 16 bit register (AX), or a single 8 bit register (AL). The compiler generates a single assembly instruction called MOVZX (Move with Zero-Extend):
MOVZX EAX, byte ptr [RSI + 16]
This instruction tells the CPU: go to address RSI + 16, grab that single byte, place it in the 8 bit AL slot, and fill the rest of EAX with zeroes. A 4 byte integer faces a penalty because it can split across lanes, as in Section 26. An 8 byte integer faces a penalty because it can split across lanes, as in Section 35. A 1 byte field is a single, indivisible atom, and it is physically impossible for it to bridge a boundary, because a boundary only exists between bytes, so it always sits entirely inside one slot of the hardware lane. Whether a char sits at offset 16, 17, 19, or 23, the CPU always reads it out of the L1 cache line in exactly 1 clock cycle, with zero bit shifting cleanup required.
40. The Odd Case: Reading a 10 Byte Field
A 10 byte read, for example a fixed size char array[10] used for a stock ticker symbol like "AAPL USD ", will almost always face a penalty on modern 64 bit hardware, since there is no such thing as a 10 byte CPU register or physical data lane, unlike the clean 1, 4, and 8 byte cases in Section 26, Section 35, and Section 39.
Clean 10 Bytes vs Shattered 10 Bytes, and the SIMD Fix
Since native hardware lanes and registers only exist in powers of 2 (1, 2, 4, 8, 16, 32), a 64 bit CPU physically cannot load 10 bytes in a single instruction, so the compiler breaks the request into two separate operations: an 8 byte read for the first 8 bytes, and a 2 byte read for the remaining 2 bytes.
Scenario A, the clean 10 byte read, aligned to 8: if I place the 10 byte field at the very beginning of the struct, offset 0, it aligns perfectly with the start of the hardware lanes:
Instruction 1 (Cycle 1): fetch all of Lane 1, landing perfectly in an 8 byte register with zero shifting. Instruction 2 (Cycle 2): fetch Lane 2, and internal wiring isolates the first 2 bytes cleanly into a 2 byte register. Total cost, 2 clock cycles, completely clean.
Scenario B, the nightmare 10 byte read, packed tight, misaligned: using #pragma pack(1) and placing this 10 byte string at an odd offset such as offset 3, the 10 bytes scatter across three different physical hardware lanes:
The 8 byte read spans from offset 3 to offset 10, crossing from Lane 1 into Lane 2, requiring a multi cycle shift and mask rescue mission exactly like Section 32 and Section 33. The 2 byte read spans from offset 11 to offset 12, crossing from Lane 2 into Lane 3, requiring another shift and mask operation. Instead of a clean 2 cycles, the CPU bogs down for 6 to 8 clock cycles, spinning its internal logic gates just to stitch the 10 byte string back together.
The HFT trick for 10 bytes, using SIMD: if I absolutely must copy or check a 10 byte string, such as a symbol, instantly, the professional approach is to pad the field to 16 bytes in the struct layout, ensuring it starts on an offset that is a multiple of 16:
struct OptimizedTicker {
char symbol[16]; // Expanded from 10 to 16!
};
By rounding up to 16 bytes, a power of 2, I can use a single SIMD (Single Instruction Multiple Data) vector register instruction, such as MOVAPS or an AVX instruction, and the CPU handles all 16 bytes simultaneously in 1 single clock cycle. Wasting 6 bytes of space to turn a 10 byte field into a 16 byte field feels wrong on paper, but in silicon it allows the hardware to skip the multi instruction breakup entirely.
41. Conclusion: The Final Ordering Rule
Every concept in this article traces back to a single root cause: the C++ struct I write on the page is not the same thing as the memory layout the compiler and the CPU actually produce and read. Data alignment and padding (Section 2), endianness (Section 3), and strict aliasing (Section 4) are the three hidden traps in zero copy deserialization (Section 1). Text formats such as JSON, XML, and FIX cost hundreds of CPU instructions per field, as shown in Section 10 and Section 22, while a fixed width binary layout lets the CPU teleport directly to any offset in a single instruction, because RAM is random access memory (Section 24).
The Full Recap, and the Rule I Actually Ship
A framing protocol, a small header carrying message type and message length (Section 12), is what lets a single TCP stream safely carry multiple message types and lets my server find the boundary of the next message (Section 16). The kernel treats all of this as anonymous bytes (Section 18), routed purely by port number into a FIFO socket buffer that lives in kernel space RAM (Section 20), popped destructively on every recv() call (Section 17), and subject to flow control and buffer limits that can stall my engine if my processing loop is too slow (Section 21). Inside my own process, only a single thread should ever touch the network socket and the live order book (Section 19), since all threads share the same memory and a misbehaving thread can silently corrupt or steal order data.
At the hardware level, the CPU can only fetch memory in fixed 4 byte or 8 byte lanes (Section 26), so any field that does not start on a clean multiple of its own size forces a multi cycle rescue mission of shifting and masking (Section 27, Section 32, Section 33), whether that field is 1 byte (Section 39), 4 bytes (Section 26), 8 bytes (Section 35), or an odd size like 10 bytes (Section 40).
The single rule that resolves nearly every one of these traps at once, and the rule I apply throughout my matching engine's wire protocol, is to order struct fields from largest data type to smallest data type. This produces a message that is exactly as small as the data it contains, with zero wasted padding bytes on the network, while every multi byte field naturally lands on a boundary the CPU can read in a single clock cycle, with zero bit shifting and zero masking:
#pragma pack(push, 1)
struct MessageHeader {
uint16_t msg_type;
uint16_t msg_length;
};
struct OrderRequest {
int32_t price; // offset 0 within body, aligned to 4
int32_t order_id; // offset 4, aligned to 4
int32_t quantity; // offset 8, aligned to 4
char type; // offset 12
char side; // offset 13
};
#pragma pack(pop)
void handle_message(const char* buffer) {
MessageHeader header;
std::memcpy(&header, buffer, sizeof(MessageHeader));
const char* body = buffer + sizeof(MessageHeader);
if (header.msg_type == 1) {
OrderRequest req;
std::memcpy(&req, body, sizeof(OrderRequest));
// req.price, req.order_id, req.quantity are ready to use immediately
}
}
For my running example, a full new buy order for 10 shares at $100.50, this is 18 bytes total, sent once over the wire, decoded with a single memcpy per message, with every integer field naturally aligned to the CPU's own hardware lanes and zero heap allocation anywhere in the path. Simple Binary Encoding, covered in Section 8, Section 13, Section 14, and Section 23, automates this same idea with schema versioning for systems where the client and server cannot be upgraded at the same moment, which I deliberately chose not to adopt for the reasons in Section 25. For a project where I control both ends of the connection, a carefully ordered, tightly packed C++ struct reaches the exact same hardware speed with none of the external tooling.
The full source code and design notes for this project are available here: Single Threaded Limit Order Book Matching Engine in C++