Overview
Edge Storage provides local data caching and persistence capabilities on edge gateways and is a key component of edge computing architecture. In industrial sites, remote areas, and mobile scenarios, networks are often unstable or intermittently available. If device data is uploaded directly to the cloud, it will be lost during network downtime. Edge storage first writes data locally and automatically synchronizes it to the cloud once the network is restored, achieving "resume on disconnection" and ensuring data integrity. In addition, edge storage supports local queries for use by edge rule engines and edge AI, enabling local decision-making and historical analysis in offline scenarios.
Edge devices have limited storage resources (usually eMMC, SD cards, or small-capacity SSDs), so edge storage needs to be lightweight and efficient. SQLite, RocksDB, and time-series databases (such as the embedded version of QuestDB) are common choices. Capacity management must be supported: circular overwriting, expiration cleaning, and archiving according to policies to avoid disk full issues.
Core Functions
Disconnection Caching and Synchronization
When the network is interrupted, device data is written to a local queue or database. Two levels of buffering are supported: memory buffering + persistence, with overflow written to disk when memory is full. When the network recovers, background tasks upload data in order, supporting breakpoint resume and deduplication (to avoid repeated uploads). Synchronization strategies are configurable: real-time, batch, scheduled. Retry on failure and exponential backoff are supported.
Data filtering and aggregation
Data can be filtered according to policy before uploading: only abnormal data or data that exceeds a threshold is uploaded. Supports time window aggregation: for example, aggregating samples within 1 minute into a single record before uploading, reducing data volume. Works with edge rule engines to achieve "edge preprocessing, cloud stores the key data".
Local Query
When offline, the edge rule engine, edge AI, and local applications can query historical data. They support queries by device, time range, and metrics. Time series data can be aggregated and downsampled. Meets the requirement of "local analysis can be performed even when disconnected from the network."
Capacity Management
Circular Overwrite: Overwrite the oldest data when storage is full. Expiration Cleanup: Automatically delete according to retention policies (e.g., keep for 7 days). Compression: Compress historical data for storage. Monitor disk usage, trigger alerts or aggressive cleanup when exceeding thresholds.
Typical IoT Applications
Industrial Sites: Production line data is locally cached when offline and synchronized when restored. Supports local historical queries for edge quality analysis.
Remote Areas: In scenarios like agriculture, mines, and oil fields with unstable networks, edge storage ensures data is not lost.
Mobile Scenarios: Data collected by vehicle- and ship-mounted devices in tunnels, offshore areas, and other no-network zones is cached locally and synchronized when network is available.
Technology Selection
SQLite: Lightweight, single-file, no server required, suitable for simple KV or table structures. RocksDB: High-performance KV, supports high write rates. QuestDB, InfluxDB Embedded: Optimized for time series data. EdgeX Foundry supports configurable storage backends. Selection should consider write performance, query capability, storage usage, and integration with the edge framework.