Overview
A time-series database (TSDB) is optimized for sensor data reported by devices in IoT scenarios according to timestamps and is the core of IoT data storage. Unlike relational databases that store data by rows, time-series databases use columnar storage, time partitioning, and dedicated compression algorithms, leveraging the characteristics of time-series data being "time-ordered, multi-metric, high-write, and primarily range queries" to achieve million-points-per-second write throughput and millisecond-level time range queries. In scenarios such as energy monitoring, industrial collection, environmental monitoring, and smart metering, time-series databases are an irreplaceable storage engine.
Characteristics of IoT time-series data: Data points are generated in the form (device, metric, timestamp, value), and multiple metrics for the same device are often reported together; queries are mostly "aggregate or detailed data of a certain metric of a certain device within a certain time range"; data is time-sensitive, with recent data being hot and historical data being cold. Time-series databases are specifically optimized in storage, indexing, and querying for these characteristics.
Core Features
High Write and Compression
Million-points-per-second write, supports batch insert optimization. Columnar storage + Delta encoding + compression algorithms can achieve a compression ratio of over 10:1. Time partitioning facilitates expiration deletion and parallel queries.
Downsampling and Aggregation
Built-in aggregate functions such as AVG, SUM, MAX, MIN. Supports downsampling: aggregate data at 1-minute granularity into 1-hour or 1-day intervals, reducing storage and speeding up large-range queries. Supports continuous query (Continuous Query) automatic pre-aggregation.
Retention policies and layering
Automatically expire and delete by time (e.g., retain for 90 days). Supports tiered storage with hot data on SSD and cold data on HDD. Closely integrated with data microservices, stream processing, and data visualization.
Typical Products and Selection
InfluxDB: open-source, popular, and rich ecosystem. TDengine: domestic, high-performance, high compression ratio, outstanding in IoT scenarios. TimescaleDB: PostgreSQL-based extension, good SQL compatibility. OpenTSDB: HBase-based, suitable for big data ecosystem. Selection needs to consider write performance, query capability, compression ratio, maintenance cost, and integration with the existing tech stack.
How a timestamp series is laid out
Measurements are stored compressed, with time as the key
A time-series database stores who measured what, and when, by how close those times sit to each other. Writes are mostly appends. Queries read a time range and compress by the difference between neighboring points. It serves detail or aggregates over a period. It does not take transactional updates on arbitrary fields. Device records and customer contracts stay in a relational database. Pictures and images stay in object storage. Tags in the database filter devices or measurements. Tag values should be stable, and the number of distinct values should stay limited.
Several measurements reported by the same device at the same time can land as several columns of one row, or be split into several series. When there are many columns and they are sparse, a query for one measurement reads in a large number of empty columns. When series are split too finely, fetching every measurement for that device has to merge too many series. Choose the layout from the query that matters: a long range of one measurement, or several measurements at one moment. Convert units before ingest. When Celsius and Fahrenheit, or seconds and milliseconds, exist in the database together, an aggregate function cannot tell which value can be used. How units and empty values are arranged is covered in Data Cleaning.
Design downsampling and retention together
Downsample only when raw precision is worth tracing in the near term and the later term only needs the trend. A continuous query, or a job outside the database, writes a coarser series by the hour and by the day, and the raw series uses a shorter retention of its own. Agree the coarse function with the business first. Energy uses a sum. Temperature should keep the extremes as well as the mean. A state value cannot use a mean to stand in for the state it occupied most of the time.
When retention expires, the data is deleted. It is not archived on its own. If an audit still needs the raw points, export them before they expire, and record the time range that was exported and the checksum. After hot and cold data are split into tiers, a query that crosses tiers must be allowed a longer response. An interface cannot promise cold data the same delay as hot data. Keep a single writer between a continuous query inside the database and a job outside it, so two processes do not write the same coarse series at once. How the export files are stored is covered in Distributed Storage. Put the time range in the key so a span can be fetched later.
Timestamp units, out-of-order points, and tag cardinality
Some devices use seconds, some use milliseconds, and some use a local-time string with no time zone. If precision and time zone are not made the same on ingest, points land near 1970, or are scaled into the far future, and partitions are stretched into large empty blocks. The write interface should reject a time it cannot parse, and it should not write the zero instant. When an out-of-order point passes the window the database allows, it may be rejected, or written into the wrong compressed block, and the query curve jumps backward. Points outside that window go into a rejection log, and they are filled in after the device clock is corrected.
Tag cardinality is another capacity problem. Putting a random identifier from each request, a full payload, or an alarm text that keeps growing into a tag makes the index swell with the device count times the number of text variants, and compression stops working. A tag should hold only a limited set of values used for filtering. Content that changes often goes into the measurement value. If a device's series are left in place after the device is deleted, the old series still occupy space for the rest of the retention period. The deletion list for device master data and time-series expiry should be the same list. A query that asks for the raw points of every device over a very long span bypasses the series prepared for downsampling. The interface should force a long span onto the coarse series.