Skip to content
Try Hanzo
Language: English (United States)

A column store for analytical SQL

Datastore keeps each column of a table in its own file, compressed on its own terms, so a query naming four columns out of a hundred reads four.

Hanzo Datastore

events exampleReads 1 of 12 columns
SELECT count() FROM events WHERE event = 'signup'

Press a column to add it to the query.

What it does

Scan a great many rows. Return a small answer. Do it while the rows are still arriving.

  1. tsdeltas
    eventdictionaries
    countrydictionaries
    revenuegeneral

    01 / 06

    Columns, not rows

    A table is stored one column at a time, so a query reads only the columns it names. Each column gets a codec suited to what it holds — deltas for a rising timestamp, dictionaries for a repeated label, general compression over the result.

  2. node

    node

    node

    partitioned and spread across nodes

    02 / 06

    Shard it and it keeps up

    Tables are partitioned and spread across nodes; add nodes and both the storage and the scan spread with them. Parts merge in the background, so ingest never stops for a compaction window.

  3. Raft quorum

    replica
    replica
    replica

    catches up part by part

    03 / 06

    Replicas that agree

    Replicated tables coordinate through a Raft quorum running inside the server itself — there is no separate coordination service to stand up beside the database. A replica that falls behind catches up part by part.

  4. node
    node
    node
    object storage
    partitionpartitionpartitionpartitionTTL

    04 / 06

    Local disks, or object storage

    Run a shared-nothing cluster on local NVMe, or put the table's data on object storage and scale query nodes on their own. Old partitions expire on a TTL you write into the schema.

  5. Events

    metrics

    logs

    partitioned by time

    Materialized views

    05 / 06

    Events, metrics and logs are one shape

    Append-heavy, partitioned by time, queried by range — the same engine serves all three. Materialized views keep rollups current as rows land, so the summary exists before anyone asks for it.

    • HTTP8123
    • native protocol9000
    • MySQLwire port
    • PostgreSQLwire port

    CSVParquetJSON stream

    06 / 06

    SQL, on several protocols

    Query over HTTP on 8123, on the native protocol on 9000, or through the MySQL and PostgreSQL wire ports — which is how BI tools and drivers that know nothing about it still connect. A CSV, a Parquet file or a JSON stream loads without a conversion step first.

Where the speed comes from

Nothing exotic. Read less, decompress less, and touch memory in the order the CPU wants it.

Read only what the query names

A row store reads the whole row to answer a question about two of its fields. A column store reads the two.

Fewer bytes off the disk, fewer bytes to decompress, and more of the working set fits in memory.

01

One instruction, many values

Values from a single column sit next to each other in memory in the same representation, so a sum or a filter runs across a block of them with vector instructions instead of once per row.

That layout is also what keeps the CPU cache full rather than chasing pointers.

02

Sorted, so a range is contiguous

Rows are written in the order of the table's sorting key and grouped into parts by partition, so a query bounded by time reads a run of adjacent blocks instead of seeking around the disk.

Marks inside each part let the scan jump straight to the block that could hold the answer.

03

Compression chosen per column

A rising timestamp compresses as deltas. A repeated label compresses as a dictionary. A general-purpose compressor runs over whatever is left.

Less data on disk is also less data on the wire once the query is spread across nodes.

04

What people point it at

Different industries, the same shape of problem: a great many rows in, a small answer out, asked again a second later.

  1. 01Web and app analytics
  2. 02E-commerce and finance
  3. 03Time series
  4. 04Advertising networks and RTB
  5. 05Information security
  6. 06Business intelligence
  7. 07Telecommunications
  8. 08Monitoring and telemetry
  9. 09Online games
  10. 10Internet of Things (IoT)
  11. 11Observability
  12. 12User behavior analytics

Which way in?

SQL, on several protocols

HTTP · port

8123

Query over HTTP on 8123.

A CSV, a Parquet file or a JSON stream loads without a conversion step first.

Get Datastore

hanzoai/datastore

Document datastore

License: Apache-2.0hanzoai/datastore4 stars

Paste and ship

Paste this into any agent. It reads the skill manifest and calls Datastore from there.

api /v1/provisioning/datastore

Agent prompt
Read https://hanzo.ai/skill.md and use Hanzo Datastore in my project. Start with: hanzo provisioning datastore create