01 / 06 — 06 / 06
offset 000101 / 06
Produce, Fetch, ListOffsets, Metadata, OffsetCommit, OffsetFetch, FindCoordinator, JoinGroup, SyncGroup, Heartbeat, CreateTopics and ApiVersions, at the versions modern clients negotiate. kafka-python, sarama, confluent-kafka and franz-go all connect.
offset 000202 / 06
One topic partition is one durable PubSub stream. Retention, replicas and disk are that stream's settings, so the thing actually holding your events is a system you already run and already back up.
offset 000303 / 06
One Go binary that keeps no state of its own. Restart it, run several, roll it mid-deploy — the log is somewhere else, so the gateway is never the piece you are afraid to touch.
offset 000404 / 06
A group's committed offsets live in a PubSub key-value bucket, so a consumer that dies comes back where it left off instead of at the top. Ask for an offset that has aged out and you get OFFSET_OUT_OF_RANGE, so the client resets rather than hangs.
offset 000505 / 06
Every Kafka offset is written into the record batch header when the batch is produced, and read straight back from there. Deriving them from the store's own sequence numbers works right up until the sequence has gaps in it — and then one record can make every fetch after it unreadable.
offset 000606 / 06
A produce whose batch chain does not line up is refused with INVALID_RECORD instead of being written and discovered later by a consumer that cannot get past it. The scan that recovers a partition's bounds is capped, so a damaged topic degrades rather than stalls.
Type a Kafka API.
import json
from kafka import KafkaProducer, KafkaConsumer
# The only line that changes
producer = KafkaProducer(
bootstrap_servers=["stream.hanzo.ai:9092"],
value_serializer=lambda v: json.dumps(v).encode(),
)
producer.send("events", {"type": "user.signup", "user_id": "usr_123"})
# Consumer groups work identically
consumer = KafkaConsumer(
"events",
bootstrap_servers=["stream.hanzo.ai:9092"],
group_id="my-service",
auto_offset_reset="earliest",
)
for message in consumer:
process(message.value)
The clients stay as they are. The cluster underneath them goes away.
Event stream
Paste and ship
Paste this into any agent. It reads the skill manifest and calls Stream from there.
Read https://hanzo.ai/skill.md and use Hanzo Stream in my project.