A column store for analytical SQL
Datastore keeps each column of a table in its own file, compressed on its own terms, so a query naming four columns out of a hundred reads four. Aggregations run over compressed blocks with vector instructions rather than a row at a time. It is built for the queries that scan a great deal and return very little — dashboards, event analytics, metrics, logs and traces. It is an analytics database and only that, not the place your application's rows live.
What it does
Scan a great many rows. Return a small answer. Do it while the rows are still arriving.
Columns, not rows
A table is stored one column at a time, so a query reads only the columns it names. Each column gets a codec suited to what it holds — deltas for a rising timestamp, dictionaries for a repeated label, general compression over the result.
Shard it and it keeps up
Tables are partitioned and spread across nodes; add nodes and both the storage and the scan spread with them. Parts merge in the background, so ingest never stops for a compaction window.
Replicas that agree
Replicated tables coordinate through a Raft quorum running inside the server itself — there is no separate coordination service to stand up beside the database. A replica that falls behind catches up part by part.
Local disks, or object storage
Run a shared-nothing cluster on local NVMe, or put the table's data on object storage and scale query nodes on their own. Old partitions expire on a TTL you write into the schema.
Events, metrics and logs are one shape
Append-heavy, partitioned by time, queried by range — the same engine serves all three. Materialized views keep rollups current as rows land, so the summary exists before anyone asks for it.
SQL, on several protocols
Query over HTTP on 8123, on the native protocol on 9000, or through the MySQL and PostgreSQL wire ports — which is how BI tools and drivers that know nothing about it still connect. A CSV, a Parquet file or a JSON stream loads without a conversion step first.
Where the speed comes from
Nothing exotic. Read less, decompress less, and touch memory in the order the CPU wants it.
Read only what the query names
A row store reads the whole row to answer a question about two of its fields. A column store reads the two.
Fewer bytes off the disk, fewer bytes to decompress, and more of the working set fits in memory.
One instruction, many values
Values from a single column sit next to each other in memory in the same representation, so a sum or a filter runs across a block of them with vector instructions instead of once per row.
That layout is also what keeps the CPU cache full rather than chasing pointers.
Sorted, so a range is contiguous
Rows are written in the order of the table's sorting key and grouped into parts by partition, so a query bounded by time reads a run of adjacent blocks instead of seeking around the disk.
Marks inside each part let the scan jump straight to the block that could hold the answer.
Compression chosen per column
A rising timestamp compresses as deltas. A repeated label compresses as a dictionary. A general-purpose compressor runs over whatever is left.
Less data on disk is also less data on the wire once the query is spread across nodes.
What people point it at
Different industries, the same shape of problem: a great many rows in, a small answer out, asked again a second later.
Web and app analytics
E-commerce and finance
Time series
Advertising networks and RTB
Information security
Business intelligence
Telecommunications
Monitoring and telemetry
Online games
Internet of Things (IoT)
Observability
User behavior analytics
Up to 5% of compute goes back to open source
Every deployment is SBOM-verified. Contributors to Datastore earn a share of compute revenue — transparent, on-chain, and customizable by the community.
Start with one process
The server, the client and the single-process local mode are one binary under different names. Run the container, or install it and start datastore-server — then point datastore-client at it and load a file.