Overview
Realtime Analytics performs second-level or minute-level aggregation, statistics, and prediction on IoT device data to provide real-time decision support for business. Unlike offline batch processing, real-time analytics requires that aggregation results be queried shortly after data is generated, supporting scenarios such as dashboards, real-time alerts, and dynamic scheduling. By combining stream computing (Flink, Spark Streaming), time-series databases (InfluxDB, TDengine), and OLAP engines (ClickHouse, Doris), it enables device count statistics, energy consumption summary, anomaly prediction, trend analysis, and other capabilities. It is applied in energy management, industrial monitoring, intelligent operations, IoV, and other scenarios.
Challenges of IoT real-time analytics: large data volume, multiple dimensions (device × metric × time × region), and high query latency requirements. Techniques such as pre-aggregation, materialized views, and unified stream-batch processing are needed to complete aggregation during write or near real-time, so that queries can directly read precomputed results, avoiding full table scans.
Core Capabilities
Real-time Aggregation
Aggregation functions such as SUM, AVG, COUNT, MAX/MIN are computed in rolling or sliding windows (1 minute, 5 minutes, 1 hour). Supports multi-dimensional grouping by device, product, group, region, etc. Stream computing engines incrementally update aggregation results as data arrives, writing them into time-series databases or OLAP tables for querying.
Multi-dimensional Analysis and Drill-down
Drill down by device, time, region, and other dimensions, from summary to detail. Supports year-on-year, month-on-month, proportion, and other analysis. Downsampling and aggregation queries in OLAP engines or time-series databases support multi-dimensional analysis. Combined with data visualization, it enables interactive exploration.
Predictive Analytics and Anomaly Detection
Trend prediction and usage forecasting based on historical data. Time series forecasting using machine learning models such as LSTM and Prophet. Anomaly detection: statistical methods (3σ, IQR) or machine learning to identify outliers. Embedding model inference in real-time analytics pipelines to achieve instant alerts.
Typical IoT Applications
Real-time energy consumption statistics: electricity and water meter data aggregated by building, area, and time, displaying today's usage and year-over-year comparison on a dashboard. Equipment health analysis: vibration, temperature, and other indicators calculate health scores in real time, with alerts for scores below the threshold. Real-time production summary: aggregating production line counts and yield rates to support production dashboards. Integrated with data visualization, alerts, and reports to form a complete real-time decision-making system.
Which aggregation layer a query sees
The business question a window statistic answers
What a query interface returns in a short time should be statistics already aligned to one definition, not a fresh scan of the raw points. The question it answers is what the sum, mean, extreme, or count was for a group of devices over the last window. A window can tumble, slide, or be cut by an idle gap, and the grouping key can be device, product, region, or tenant. After the result is written to the query layer, wall displays and APIs read that number. Raw points can be kept for investigation. An interactive query should not assemble the same metric by scanning the whole table again.
Write each metric as a formula before calling it real time. Whether a missing point is ignored or counted as zero, whether a duplicate report overwrites or accumulates, and which time zone cuts buckets across midnight all belong in the formula. Different formulas turn the same raw stream into different figures for "usage today." The analytics service fixes the formula on the write path, and queries only read the result. If prediction and anomaly detection join this path, their input must be a series prepared with that same formula. Otherwise the model learns the gap-filling method, and the field process is buried.
Set the metric definition before the pre-aggregation grain
Pre-aggregation suits metrics that are queried often, have a limited set of grouping keys, and can accept the precision loss that comes with a bucket width. The bucket width should stay within the smallest change that must be resolved, and the write volume of one set of grouping keys must still be sustainable in storage. Making every device, every sub-sensor, and every minute its own key makes the pre-aggregation table larger than the raw table, and queries get slower. Leave that detail in time-series storage. The analytics layer keeps only stable summary keys.
Measure how late data may arrive before the job is enabled. If field devices cache for half an hour and then backfill, a five-minute window can still be rewritten after it closes. Treating a window that has just closed as a bill will not match a recalculation the next day. Mark each window as still open to revision until the watermark passes the backfill deadline. The split with Data Cleaning is that cleaning removes impossible values and unifies units, while analytics aggregates the clean series with the window formula. The two sides should not each keep a separate definition of a valid point.
Late points rewrite a window that has already closed
When a device clock runs ahead, points fall into a future window and the query at that moment is too small. After the clock is corrected they flood the current window, and the curve shows a step. When the clock lags, points are counted into the wrong historical bucket. The analytics job should record the gap between event time and ingest time. Points past the threshold go to a side path and do not directly rewrite a window that has already been published. If the business allows a revision, keep a revision version. Do not quietly overwrite the old number.
Adding a sum of raw points to a sum of pre-aggregates counts the same quantity twice. An interface should state which layer the result comes from. If alerts use the raw peak and the dashboard uses the mean inside the bucket, the two stay apart for a long time. Alert rules should cite the same bucket in the analytics layer, and delivery of the rule belongs in the Alert Microservice, so each place does not keep its own copy of the threshold. If the step of a sliding window is much smaller than the window length, neighboring results overlap heavily, and one fluctuation is counted many times, which looks like a continuous anomaly. Set the step and the window together for the sensitivity required. After a grouping key changes from region to device, old materialized results do not expire on their own. Write down the time range to rebuild, so the old and new definitions are not spliced into one curve.