Why we need time series database
The key property is temporal ordering — the same measurement (e.g., cpu_usage) repeats many times, with each point tied to a timestamp.
You don’t update old values; you keep adding new ones as time passes.
This pattern shows up everywhere:
- Metrics (CPU, memory, network traffic)
- Sensors (IoT, weather, industrial data)
- Financial ticks (price over time)
- Application logs, events, telemetry
| Property | Description |
|---|---|
| Append-only | New data is constantly appended (insert-heavy), rarely updated or deleted. |
| Time-ordered | Queries are usually bounded by time ranges (“last 5 minutes”, “past week”). |
| High volume | Each metric can generate thousands of points per second. |
| Aggregation-oriented | Most queries summarize data: “average temperature per hour”, not single row lookups. |
| Retention | Old data often expires automatically (e.g., keep only 30 days). |
Storage layout
-
Time-partitioned blocks (also called chunks or segments). Data points are grouped by time windows (e.g., one file per hour/day).
-
Each block is immutable once written (fits append-only nature).
-
Compression is optimized for sequential timestamps and numeric similarity, e.g.:
- Delta encoding (store time difference instead of full timestamp)
- Gorilla compression (Facebook’s technique for float deltas)
Indexing
- Instead of indexing every row, TSDBs usually index by metric name + tags (labels), not by timestamp.
- Within a metric, timestamps are implicitly ordered, so no need for a full index.
Ingestion
- Bulk, batched writes rather than single-row inserts.
- Often in-memory buffers (“write-ahead log”) before compacting to disk segments.
Query model
-
Focus on range scans + aggregation, e.g.:
SELECT avg(cpu_usage) FROM metrics WHERE host = 'server1' AND time BETWEEN now() - interval '1h' AND now(); -
Optimized for reading continuous time intervals, not random rows.
Retention and downsampling
- Automatic data lifecycle: → raw data for 7 days → hourly averages for 30 days → daily averages for 1 year
- Traditional databases require manual cleanup or partition management.
Built using Go