Category Archives: BigData/NoSQL

SCADA Real Time Data and Apache SPARK

Posted on 13. May 2023 by vogler

The integration of SCADA with Spark and WinCC Open Architecture offers a powerful and versatile solution that combines real-time data processing, advanced analytics, scalability, and flexibility. This combination empowers you to optimize industrial processes, make data-driven decisions, and stay ahead … Continue reading →

Posted in Allgemein, BigData/NoSQL, WinCC OA | Comments Off on SCADA Real Time Data and Apache SPARK

WinCC Unified GraphQL Server for Data Integration

Posted on 22. November 2022 by vogler

With the GraphQL Server in WinCC Unified V18 we can now integrate various (IT) data platforms with simple programs. Those programs can be written in Python, Java, Kotlin, Go, JavaScript, or whatever kind of programming language you prefer. In my … Continue reading →

Posted in BigData/NoSQL, WinCC Unified | Comments Off on WinCC Unified GraphQL Server for Data Integration

Size of tables in PostgreSQL vs Apache Cassandra…

Posted on 4. July 2018 by vogler

PostgreSQL table with ts+key as primary key:  ~43GB PostgreSQL wide column table with ts as primary key : 247GB Cassandra wide column table with ts as primary  key: 4.5GB Strange that in PostgreSQL a table with much less rows (but … Continue reading →

Posted in BigData/NoSQL | Comments Off on Size of tables in PostgreSQL vs Apache Cassandra…

Streaming SQL for Apache Kafka & WinCC OA with Docker…

Posted on 4. September 2017 by vogler

KSQL makes it easy to read, write, and process streaming data in real-time, at scale, using SQL-like semantics. It offers an easy way to express stream processing transformations as an alternative to writing an application in a programming language such … Continue reading →

Posted in BigData/NoSQL, WinCC OA | Comments Off on Streaming SQL for Apache Kafka & WinCC OA with Docker…

Monitoring with Logstash and WinCC OA…

Posted on 11. October 2016 by vogler

  For example we do this with the Oracle Alert Log. Very often an Oracle Database is used with WinCC OA to store history values. But a lot of times no one takes care of the Oracle database. At least the … Continue reading →

Posted in BigData/NoSQL, WinCC OA | Comments Off on Monitoring with Logstash and WinCC OA…

Observing WinCC OA Logs with Elasticsearch and Logstash…

Posted on 10. October 2016 by vogler

With Logstash we can collect the logs of WinCC OA systems and write it to Elasticsearch. Multiple WinCC OA system’s can be observed with a central log database… With Kibana the logs can be easily discovered – I now see errors what i … Continue reading →

Posted in Allgemein, BigData/NoSQL, WinCC OA | Comments Off on Observing WinCC OA Logs with Elasticsearch and Logstash…

WinCC OA connected to Apache Kafka…

Posted on 30. September 2016 by vogler

A WinCC OA Driver to connect WinCC OA to Apache Kafka (input & output). Available on Github https://github.com/vogler75/oa4j

Posted in Allgemein, BigData/NoSQL, WinCC OA | Comments Off on WinCC OA connected to Apache Kafka…

WinCC OA RDB-Manager with Oracle vs MongoDB

Posted on 29. July 2016 by vogler

Keep in Mind: It is not a comparison of the databases only. With Oracle we used the WinCC OA RDB Manager with OA Query-RDB Direct option, and the RDB-Manager has lot of more functionalities than the NoSQL Prototyp! The other … Continue reading →

Posted in BigData/NoSQL, WinCC OA | Comments Off on WinCC OA RDB-Manager with Oracle vs MongoDB

Streaming & Complex Event Processing (CEP) & EPL/CQL with WinCC OA…

Posted on 18. July 2016 by vogler

Complex Event Processing (CEP) and event series analysis are used for detecting situations among events. EsperTech provides the Event Processing Language (EPL) designed for concisely expressing situations and fast execution against both historical and currently-arriving events (Espertech.com). The Esper EPL … Continue reading →

Posted in BigData/NoSQL, WinCC OA | Comments Off on Streaming & Complex Event Processing (CEP) & EPL/CQL with WinCC OA…

Streaming WinCC OA Events to Apache Kafka and Spark…

Posted on 30. May 2016 by vogler

We connected a Event Data Logger, written in Java and connected to WinCC OA by JNI, to Apache Kafka, a A high-throughput distributed messaging system. Apache Spark can read data from Apache Kafka streams. Apache Spark is a fast and general engine for … Continue reading →

Posted in Allgemein, BigData/NoSQL, WinCC OA | Comments Off on Streaming WinCC OA Events to Apache Kafka and Spark…