Category Archives: BigData/NoSQL

SCADA Real Time Data and Apache SPARK

The integration of SCADA with Spark and WinCC Open Architecture offers a powerful and versatile solution that combines real-time data processing, advanced analytics, scalability, and flexibility. This combination empowers you to optimize industrial processes, make data-driven decisions, and stay ahead … Continue reading

Posted in Allgemein, BigData/NoSQL, WinCC OA | Comments Off on SCADA Real Time Data and Apache SPARK

WinCC Unified GraphQL Server for Data Integration

With the GraphQL Server in WinCC Unified V18 we can now integrate various (IT) data platforms with simple programs. Those programs can be written in Python, Java, Kotlin, Go, JavaScript, or whatever kind of programming language you prefer. In my … Continue reading

Posted in BigData/NoSQL, WinCC Unified | Comments Off on WinCC Unified GraphQL Server for Data Integration

Size of tables in PostgreSQL vs Apache Cassandra…

PostgreSQL table with ts+key as primary key:  ~43GB PostgreSQL wide column table with ts as primary key : 247GB Cassandra wide column table with ts as primary  key: 4.5GB Strange that in PostgreSQL a table with much less rows (but … Continue reading

Posted in BigData/NoSQL | Comments Off on Size of tables in PostgreSQL vs Apache Cassandra…

Streaming SQL for Apache Kafka & WinCC OA with Docker…

KSQL makes it easy to read, write, and process streaming data in real-time, at scale, using SQL-like semantics. It offers an easy way to express stream processing transformations as an alternative to writing an application in a programming language such … Continue reading

Posted in BigData/NoSQL, WinCC OA | Comments Off on Streaming SQL for Apache Kafka & WinCC OA with Docker…

Monitoring with Logstash and WinCC OA…

  For example we do this with the Oracle Alert Log. Very often an Oracle Database is used with WinCC OA to store history values. But a lot of times no one takes care of the Oracle database. At least the … Continue reading

Posted in BigData/NoSQL, WinCC OA | Comments Off on Monitoring with Logstash and WinCC OA…

Observing WinCC OA Logs with Elasticsearch and Logstash…

With Logstash we can collect the logs of WinCC OA systems and write it to Elasticsearch. Multiple WinCC OA system’s can be observed with a central log database… With Kibana the logs can be easily discovered – I now see errors what i … Continue reading

Posted in Allgemein, BigData/NoSQL, WinCC OA | Comments Off on Observing WinCC OA Logs with Elasticsearch and Logstash…

WinCC OA connected to Apache Kafka…

A WinCC OA Driver to connect WinCC OA to Apache Kafka (input & output). Available on Github https://github.com/vogler75/oa4j

Posted in Allgemein, BigData/NoSQL, WinCC OA | Comments Off on WinCC OA connected to Apache Kafka…

WinCC OA RDB-Manager with Oracle vs MongoDB

Keep in Mind: It is not a comparison of the databases only. With Oracle we used the WinCC OA RDB Manager with OA Query-RDB Direct option, and the RDB-Manager has lot of more functionalities than the NoSQL Prototyp! The other … Continue reading

Posted in BigData/NoSQL, WinCC OA | Comments Off on WinCC OA RDB-Manager with Oracle vs MongoDB

Streaming & Complex Event Processing (CEP) & EPL/CQL with WinCC OA…

Complex Event Processing (CEP) and event series analysis are used for detecting situations among events. EsperTech provides the Event Processing Language (EPL) designed for concisely expressing situations and fast execution against both historical and currently-arriving events (Espertech.com). The Esper EPL … Continue reading

Posted in BigData/NoSQL, WinCC OA | Comments Off on Streaming & Complex Event Processing (CEP) & EPL/CQL with WinCC OA…

Streaming WinCC OA Events to Apache Kafka and Spark…

We connected a Event Data Logger, written in Java and connected to WinCC OA by JNI, to Apache Kafka, a A high-throughput distributed messaging system. Apache Spark can read data from Apache Kafka streams. Apache Spark is a fast and general engine for … Continue reading

Posted in Allgemein, BigData/NoSQL, WinCC OA | Comments Off on Streaming WinCC OA Events to Apache Kafka and Spark…