The parts of the design that decide whether results are correct.
This page is written for engineers and architects evaluating the platform. It describes structure and behaviour without benchmark claims — measured figures depend on your workload and deployment, and we would rather establish them with you than publish numbers you cannot reproduce.
From append to readable result
Append
The event is written to a partition determined by its key and acknowledged once durable.
Advance
The owning operator instance reads the next offset in sequence for that partition.
Apply
Local state for the key is read and updated as part of processing the event.
Checkpoint
State is committed together with the offset it corresponds to.
Expose
The derived value becomes readable on the view and emitted to subscribers.
- t0AppendThe event is written to a partition determined by its key and acknowledged once durable.
- t1AdvanceThe owning operator instance reads the next offset in sequence for that partition.
- t2ApplyLocal state for the key is read and updated as part of processing the event.
- t3CheckpointState is committed together with the offset it corresponds to.
- t4ExposeThe derived value becomes readable on the view and emitted to subscribers.
Ordering is scoped to a key, on purpose
Global ordering across an entire stream would serialise the system. Ferrite guarantees order where correctness needs it — within a key — and processes keys in parallel.
You choose the partition key: account, device, order, tenant. It determines both ordering scope and parallelism.
Uneven key distribution is exposed per partition, so hot keys can be identified rather than inferred from lag.
Partition ownership can move between instances; the new owner resumes from the committed checkpoint.
Windows carry an explicit policy for events that arrive after the window advanced, instead of silently discarding them.
Stated plainly, because it is usually stated vaguely
Acknowledged after the event is durably recorded in its partition.
Offsets are committed with the state they produced, so recovery resumes without reapplying committed work.
Depends on the destination: idempotent or transactional sinks preserve the guarantee; others require a de-duplication key.
We do not claim a single end-to-end guarantee for every configuration, because the weakest link is normally the external system at the edge of the pipeline. During evaluation we go through your sinks individually.
What happens when something breaks
Partitions owned by the lost instance are reassigned; processing resumes from the last checkpoint for those keys.
Backpressure propagates upstream. Producers observe it rather than the system quietly buffering without bound.
Records that repeatedly fail are diverted for inspection so a single event cannot stall a partition indefinitely.
A workload version can be stopped and replayed from a chosen offset once corrected, because the log retains the input.
Where it runs
Deployment topology is decided with you rather than fixed by us. What we can describe is the shape of the options and their operational consequences.
The runtime and log are designed to run within infrastructure you control, so data residency follows your existing boundaries.
Throughput scales with partitions and operator instances; state size scales with keys retained per operator.
Workloads are versioned units, so a component can be upgraded without redeploying the whole pipeline.
Multi-region behaviour is a design conversation about ordering and latency trade-offs, not a checkbox.
Take the architecture apart with us.
We are comfortable spending a session on partition keys, checkpoint intervals and sink semantics before anyone talks about commercials.