Living Document Notice Published 2026-09-14. The evolving architecture and revisions for this dispatch live in the Stax Digital Garden.

Event Streaming and Drop-on-Lag Ring Buffers

Event Streaming and Drop-on-Lag Ring Buffers: Monochromatic ice blue phosphor P7 vector CRT macro depicting a circular drop-on-lag ring buffer with sweep pointers shedding lagging overflow packets

Pushing real-time graph updates to desktop modules via naive WebSocket or Server-Sent Events (SSE) pipelines risks unbounded memory expansion. If a connected module stalls during message consumption, unconsumed event buffers accumulate in server process memory, in practice triggering an out-of-memory crash of the host daemon.

Harbormaster resolves slow-client memory exhaustion by enforcing drop-on-lag bounded ring buffers. The event dispatcher caps per-client queues at fixed capacities. When a lagging client exceeds buffer thresholds, Harbormaster drops older messages and signals an overflow state, protecting the core daemon memory footprint.

Event Dispatch Topology and Queue Isolation

Each connected module consumes from an isolated memory buffer:

[ Harbor Knowledge Engine ] ??? Emits graph change event
                                          ?
                                          ?
                         ???????????????????????????????????
                         ?  Harbormaster Event Dispatcher  ?
                         ???????????????????????????????????
                                 ?                 ?
                 Bounded Queue A ?                 ? Bounded Queue B (Slow)
                 (Capacity: 128) ?                 ? (Overflow Triggered)
                                 ?                 ?
                        [ Status Widget ]      [ Lagging Client ]
                         Receives event         Drops oldest events
                                                Signals ERR_STREAM_LAG

Parameter	Unbounded Stream Pattern	Harbormaster Drop-on-Lag Buffer
Buffer Boundary	Infinite FIFO list	Fixed ring buffer (128 slots)
Slow Client Impact	Consumes host RAM until OOM	Drops oldest event; logs warning
Disconnection Threshold	Never disconnects automatically	Disconnects on persistent 30s stall
Memory Floor	Unpredictable (up to hundreds of MB)	Fixed ceiling (< 2 MB total across clients)

Bounded Ring Buffer Implementation

import collections
from typing import Any, Dict

class BoundedEventBuffer:
    def __init__(self, capacity: int = 128):
        self.capacity = capacity
        self.queue = collections.deque(maxlen=capacity)
        self.dropped_events_count = 0

    def push_event(self, event: Dict[str, Any]) -> bool:
        if len(self.queue) == self.capacity:
            self.dropped_events_count += 1
            # Evicts oldest automatically via deque maxlen
            self.queue.append(event)
            return False
        self.queue.append(event)
        return True

    def pop_event(self) -> Dict[str, Any]:
        if not self.queue:
            return None
        return self.queue.popleft()

Streaming Event Probing via cURL

Subscribe to real-time events over loopback HTTP:

# Connect to event stream with valid capability token
curl -N -H "Authorization: Bearer tok_stream_valid_089f" \
     -H "Host: 127.0.0.1:8765" \
     http://127.0.0.1:8765/protocol/v1/events/stream