Smart technology for a strong manufacturing industry with 24/7 uninterrupted production of IIoT solutions for companies.

Blog

NoSQL - Timeseries databases

Time Series Databases

Time series databases are developed to support specific use cases related to storing and analyzing time series data. They can efficiently store measured values from sensor devices, but they can also be used in a much wider range of applications (e.g., stock market monitoring). Examples of these NoSQL databases include InfluxDB, Amazon Timestream, and Prometheus.

Most time series databases use compression algorithms to manage data efficiently or make use of retention periods and data aggregation.

In a time series database, discrete attributes are typically separated from dynamic, continuous measurements. An example is storing CPU usage: discrete attributes are the name (CPU usage), unit (%), and range (0–100), while the dynamic values are the timestamp and the actual measured percentage.

This separation is intended to store and index data efficiently. Most time series databases also provide functions such as calculations, interpolation, filtering, and analysis of time series data.

In addition, NoSQL time series databases are designed to scale vertically and provide high throughput while maintaining strong performance for analytics.

Most time series databases are schema-agnostic: additional dimensions or tags can be added when ingesting data without modifying the existing schema. This allows measurements within the same table to have different contexts or metadata, making them highly suitable for dynamic and generic applications.

When to apply?

Advantages:

  • Efficient storage and compression of time series

  • Fast aggregations over time windows

  • Built-in functions for downsampling and retention policies

  • Optimized for “append-only” data

Disadvantages:

  • Not suitable for relational data

  • Limited query language outside time series

  • Sometimes difficult to integrate with traditional tools

Use cases:

  • Monitoring and metrics (server metrics, sensor data, IoT)

  • Financial time series (stocks, prices)

  • Anything that needs to be expressed over time in graphs

Comparison with Process Historians

Historians have traditionally been the choice for storing equipment measurements and offer high compatibility with industrial software such as HMIs, PLCs, OPC drivers, etc. They also support store-and-forward capabilities that prevent data loss. However, the full potential of the data in these Process Historians has largely remained untapped, partly because it is difficult to provide context to the data.

Process historians such as OSISoft PI cannot scale horizontally. Tags and contextual dimensions must be known in advance, making them unsuitable for dynamic or generic systems.

They are often fairly closed systems where it is difficult to extract data, or where connectors require special licenses to access the data.

Nonetheless, they remain invaluable, especially since much historical data is still waiting to be used for pattern recognition, determining the “golden batch,” anomaly detection, preventive maintenance, etc., based on machine learning models.

Strengths of Process Historians:

  • Robust: proven in chemical, pharmaceutical, and energy sectors

  • Strong data consistency and reliability: specifically built for real-time industrial processes

  • Strict schema and tags: consistent data structures for sensors and calculations

  • Built-in functionality: event frames, alarms, interpolations, compressions, roll-ups, analytics

  • Integrations with SCADA, DCS, and MES systems: ready-to-use connectors

Disadvantages of Process Historians:

  • Less flexible: all tags and dimensions must be predefined; new sensor structures are harder to add

  • Cost: Historians are license-intensive and often expensive

  • Limited horizontal scalability; cloud integration is often complex