Topics — Full Reference

A self-driving forklift has a LiDAR scanning at 10 Hz, cameras streaming 30 FPS, wheel encoders firing at 1 kHz, and a safety system that must react within microseconds. All of this data needs to flow between nodes — but the forklift can't afford the 50–100 µs per message that sockets and pipes add. At 1 kHz, that overhead alone eats 5–10% of every control cycle. And when the vision team splits their processing into a separate process for fault isolation, you can't afford to rewrite every communication path.

HORUS topics solve this with automatic shared-memory IPC. You call send() and recv(). Every topic is shared-memory-backed from the moment it's created; HORUS counts how many publishers and subscribers exist (and whether the payload is plain-old-data) and selects one of four lock-free SHM backends. Latency is dominated by core locality, not by the backend: when producer and consumer share an executor thread the data stays L1-hot (~12–19 ns one-way); across threads or processes it's ~220 ns one-way — cache-coherence physics that no backend choice can beat. Because every topic is already SHM-backed, a consumer in another process can attach later with no migration. Your code never changes.

For a gentler introduction, start with Topics: How Nodes Talk.

Automatic Backend Selection

HORUS doesn't expose one rigid communication backend — it has four shared-memory backends, each matched to a messaging pattern. Every topic is shared-memory-backed from the moment it's created; the system picks the backend automatically based on two factors:

1. How many publishers and subscribers exist?
2. What payload — a plain-old-data (POD) type, or one that needs serialization?

Note that these factors select the pattern, not the speed. Latency is set by core locality (see below): a producer and consumer on the same executor thread run ~12–19 ns one-way; anything cross-thread or cross-process is ~220 ns one-way, regardless of which backend is in play.

Backend Reference

All four backends are shared-memory-backed. They differ in the messaging pattern they serve, not in their latency tier:

Latency Is Core Locality

Which backend you land on does not set your latency — where the producer and consumer run does. The cliff is the CPU cache boundary, not intra-process-vs-shared-memory:

When producer and consumer share a core the payload stays L1-hot (~12–19 ns). The moment they sit on different cores — whether that's two threads or two processes — you pay ~220 ns of cache-coherence traffic, and no backend or layout choice moves that number. An in-process heap ring would not beat SHM here. The fastest realistic route is to co-locate a producer and consumer on one executor thread.

Joining and Fan-out Changes

Because every topic is shared-memory-backed from creation, a process split needs no migration at all: a consumer in another process simply attaches to the existing shared memory — no rewiring, no dropped messages. This is exactly why there is no separate "intra-process" path to migrate away from.

When the fan-out itself changes — a second subscriber joins a 1:1 topic, so the pattern shifts from SpscShm toward a broadcast backend (PodShm / FanoutShm) — HORUS re-selects the matching backend transparently. You never need to configure or trigger this; it happens automatically on the next send() or recv() after the change is detected.

Communication Patterns

One-to-One (SpscShm — ~19 ns same-thread, ~220 ns cross-thread/process)

The simplest pattern: one publisher, one subscriber. Auto-selects SpscShm (single-producer, single-consumer) — the leanest backend. Co-locate both nodes on one executor thread and it runs L1-hot (~19 ns one-way); split them across cores or processes and it's ~220 ns either way.

// simplified
use horus::prelude::*;

// Publisher
struct Sensor { data_pub: Topic<f32> }
impl Node for Sensor {
    fn name(&self) -> &str { "Sensor" }
    fn tick(&mut self) { self.data_pub.send(42.0); }
}

// Subscriber
struct Display { data_sub: Topic<f32> }
impl Node for Display {
    fn name(&self) -> &str { "Display" }
    fn tick(&mut self) {
        if let Some(v) = self.data_sub.recv() { println!("{}", v); }
    }
}

One-to-Many / Broadcast (PodShm or FanoutShm — ~19 ns same-thread, ~220 ns cross-thread/process)

One publisher, multiple subscribers. All subscribers independently receive every message. Auto-selects PodShm for plain-old-data payloads (memory-mapped broadcast) or FanoutShm for types that need serialization.

// simplified
// One publisher
struct Broadcaster { alert_pub: Topic<String> }

// Three independent subscribers — all receive the same data
struct Logger { alert_sub: Topic<String> }
struct Dashboard { alert_sub: Topic<String> }
struct Recorder { alert_sub: Topic<String> }

This is the standard pattern for sensor data: one camera node publishes, and the vision system, logger, and display all subscribe independently.

Many-to-One / Aggregation (MpscShm — ~19 ns same-thread, ~220 ns cross-thread/process)

Multiple publishers, one subscriber. Auto-selects MpscShm (multiple-producer, single-consumer). Messages are interleaved in arrival order.

// simplified
// Multiple sensors publishing to the same topic
struct TempSensor1 { pub_temp: Topic<f32> }
struct TempSensor2 { pub_temp: Topic<f32> }

// Single aggregator drains all readings
struct Aggregator { sub_temp: Topic<f32> }

impl Node for Aggregator {
    fn name(&self) -> &str { "Aggregator" }
    fn tick(&mut self) {
        // Drain all readings — may come from either sensor
        while let Some(reading) = self.sub_temp.recv() {
            self.process(reading);
        }
    }
}

Many-to-Many (FanoutShm — ~19 ns same-thread, ~220 ns cross-thread/process)

Multiple publishers and subscribers (or any non-POD broadcast). Auto-selects FanoutShm, which wires a per-pair SHM SPSC matrix — the most general path. The per-hop latency still follows the core-locality ladder above, but every added publisher/subscriber pair is extra work, so prefer a narrower pattern when you don't genuinely need N:M.

Topic Naming

Use Dots, Not Slashes

HORUS uses dots (.) for topic name hierarchy:

// simplified
// CORRECT
Topic::<f32>::new("sensor.temperature")?;
Topic::<CmdVel>::new("robot.motor.cmd_vel")?;
Topic::<LaserScan>::new("lidar.front.scan")?;

// WRONG — fails on macOS
Topic::<f32>::new("sensor/temperature")?;

Naming Conventions

Avoid: names starting with _ (reserved for internal use), names containing special characters (!@#$%^&*()), and names containing /.

Type-Safe Topic Descriptors

The topics! macro defines compile-time topic descriptors that prevent name typos and type mismatches:

// simplified
use horus::prelude::*;
use horus_robotics::prelude::*;

topics! {
    pub CMD_VEL: CmdVel = "cmd_vel",
    pub SENSOR_DATA: f32 = "sensor.data",
    pub MOTOR_STATUS: MotorStatus = "motor.status",
}

// Usage — type is enforced at compile time
let pub_topic = Topic::<CmdVel>::new(CMD_VEL.name())?;
let sub_topic = Topic::<CmdVel>::new(CMD_VEL.name())?;

This catches topic name typos and type mismatches at compile time instead of runtime.

Message Types

What Types Work?

Any type that implements Clone + Send + Sync + Serialize + Deserialize + 'static works with topics. In practice, this means almost everything:

// simplified
use serde::{Serialize, Deserialize};

// Primitive types — work out of the box
let topic: Topic<f32> = Topic::new("float")?;
let topic: Topic<bool> = Topic::new("flag")?;

// Custom structs — add standard derives
#[derive(Clone, Serialize, Deserialize)]
struct MyMessage {
    x: f32,
    y: f32,
    label: String,
}
let topic: Topic<MyMessage> = Topic::new("custom")?;

The TopicMessage Trait

Under the hood, all types go through the TopicMessage trait, which defines a wire format for transmission:

// simplified
pub trait TopicMessage: Sized + Send + 'static {
    type Wire: Clone + Send + Sync + Serialize + DeserializeOwned + 'static;
    fn to_wire(&self, pool: &Option<Arc<TensorPool>>) -> Self::Wire;
    fn from_wire(wire: Self::Wire, pool: &Option<Arc<TensorPool>>) -> Self;
    fn needs_pool() -> bool;
}

For most types, the blanket implementation makes Wire = T — a direct pass-through with zero overhead. Pool-backed types (Image, PointCloud, DepthImage, Tensor) use a small descriptor as the wire format and transfer the actual data through shared memory pools.

You never need to implement this trait manually — the blanket impl covers all standard types.

Pool-Backed Types (Zero-Copy for Large Data)

For large data types, copying through the ring buffer would be too slow. A 1920×1080 RGB image is 6 MB — copying it at 30 FPS would consume 180 MB/s of memory bandwidth just for one topic.

HORUS solves this with pool-backed types. The actual data lives in a shared memory pool. The topic transfers only a small descriptor (~200–336 bytes) that points to the pool slot. The receiver reads the pixel data directly from the same memory — zero copies regardless of image size.

The API is identical — send(), recv(), try_send() all work the same:

// simplified
use horus::prelude::*;

let camera = Topic::<Image>::new("camera.rgb")?;

// Send (moves Image into pool slot, sends descriptor through ring)
camera.send(image);

// IMPORTANT: drain every tick — images are large and stale frames waste pool slots
if let Some(img) = camera.recv() {
    println!("{}x{} image received", img.width(), img.height());
}

Memory and Capacity

Ring Buffer Model

Each topic is backed by a ring buffer with a fixed number of slots. When a publisher calls send() and all slots are occupied, a point-to-point (1:1) topic rejects the new message (keeping the queued messages in publish order); a broadcast (multi-subscriber) topic overwrites the oldest.

Ring buffer (capacity = 4):
┌───┬───┬───┬───┐
│ 3 │ 4 │ 5 │ _ │  ← slot 0 was overwritten by message 4
└───┴───┴───┴───┘
  ↑           ↑
  read        write
  cursor      cursor

The default capacity is 4 slots. This is intentionally small — robotics control loops care about the latest data, not history. Increase capacity when you need more buffering:

// simplified
// Default: 4 slots, auto-sized
let topic = Topic::<Imu>::new("imu")?;

// Custom: 16 slots, auto-sized — for high-frequency bursty publishers
let topic = Topic::<Imu>::with_capacity("imu", 16, None)?;

// Custom: 8 slots, 4 KB each — for large variable-size messages
let topic = Topic::<LaserScan>::with_capacity("lidar.scan", 8, Some(4096))?;

Memory Usage

For most robotics applications, total topic memory is well under 1 MB.

Cleaning Up Shared Memory

Shared memory files persist after processes exit — by design, so new processes can join existing topics. Clean up between sessions:

horus clean --shm            # Clean shared memory
horus clean --shm --dry-run  # Preview what would be cleaned
horus clean --all            # Clean everything (SHM + build cache)

HORUS also performs automatic stale-topic cleanup — files with no active process are removed when new topics are created.

Runtime Debugging

CLI Tools

horus topic list              # List all active topics
horus topic echo sensor.data  # Print messages on a topic
horus topic hz sensor.data    # Show publish rate
horus monitor --tui           # Interactive dashboard

TUI Debug Logging

Debug logging is toggled at runtime from the TUI monitor — no code changes or recompilation needed. Select a topic in the Topics tab and press Enter to start logging; press Esc to stop.

When active, every send() and recv() records timing and message summaries. When disabled, there is zero overhead — introspection is fully separated from the hot path.

Programmatic Monitoring

// simplified
use horus::prelude::*;
use horus_robotics::prelude::*;

let topic = Topic::<CmdVel>::new("cmd_vel")?;

// Check for drops (subscriber too slow)
if topic.dropped_count() > 0 {
    hlog!(warn, "Dropped {} messages on '{}'", topic.dropped_count(), topic.name());
}

// Aggregate statistics
let m = topic.metrics();
println!("Sent: {}, Received: {}, Failures: {}",
    m.messages_sent(), m.messages_received(), m.send_failures());

// Connection info
println!("Backend: {}, Pubs: {}, Subs: {}",
    topic.backend_name(), topic.pub_count(), topic.sub_count());

Design Decisions

Why four backends instead of just one "shared memory"? A single shared-memory implementation that handles every pattern must use the most conservative synchronization — CAS on both head and tail — even for the common case of one publisher and one subscriber, where a simple Release/Acquire ring (SpscShm) does the job. The four backends (SpscShm, MpscShm, PodShm, FanoutShm) let each messaging pattern use the minimal synchronization it needs, with automatic selection so you never think about it. Note that this is about synchronization overhead, not the latency tier: latency is governed by core locality (~12–19 ns same executor thread, ~220 ns cross-core), and co-locating a producer and consumer on one thread is the real lever.

Why ring buffers instead of unbounded queues? Unbounded queues grow without limit. A slow subscriber on a fast publisher creates a memory leak that eventually crashes the process. Ring buffers have fixed, predictable memory usage. When full, a 1:1 topic rejects new sends (keeping its queued messages in order) and a broadcast topic overwrites the oldest — the publisher never blocks either way. For guaranteed delivery use send_blocking(); for freshest-wins on a 1:1 stream, drain with recv_all()[-1].

Why automatic backend selection instead of manual configuration? During development, everything runs in one process. In production, you split across processes for isolation. In simulation, you might run everything on one thread for determinism. If you had to manually configure the backend for every topology, you'd need different configurations for development, testing, and production. Automatic selection means the same code runs optimally everywhere.

Why SHM-backed from creation? A new subscriber joining shouldn't require restarting the publisher, and a process splitting in two shouldn't require code changes. Because every topic is shared-memory-backed the instant it's created (the creator's PID is stamped at init), a consumer in another process just attaches to the existing shared memory — no migration, no coordination. When the fan-out pattern itself changes, HORUS re-selects the matching backend at runtime, without dropping messages.

Why dots instead of slashes for naming? POSIX shm_open on macOS interprets slashes as directory separators. A topic named "sensor/lidar" creates a file at /dev/shm/sensor/lidar instead of /dev/shm/sensor.lidar, which fails if the sensor/ directory doesn't exist. Dots work identically on Linux and macOS. This was a pragmatic cross-platform decision, not a stylistic one.

Trade-offs

See Also

• Topics: How Nodes Talk — Beginner introduction
• Topic API — Complete API reference with every method signature
• Python Topic API — Topic API from Python
• Communication Overview — When to use topics vs services vs actions
• Image API — Pool-backed camera images
• PointCloud API — Pool-backed 3D point clouds
• Message Types — Standard robotics message types