Allgemein, BigData/NoSQL, WinCC OA

Streaming WinCC OA Events to Apache Kafka and Spark…

We connected a Event Data Logger, written in Java and connected to WinCC OA by JNI, to Apache Kafka, a A high-throughput distributed messaging system.

Apache Spark can read data from Apache Kafka streams. Apache Spark is a fast and general engine for large-scale data processing. Combine SQL, streaming, and complex analytics. Spark powers a stack of libraries including SQL and DataFrames, MLlib for machine learning, GraphX, and Spark Streaming. You can combine these libraries seamlessly in the same application.

WinCCOA-Kafka-Spark

Allgemein, BigData/NoSQL, WinCC OA

Java is everywhere… WinCC OA Java Manager…

From laptops to datacenters, game consoles to scientific supercomputers, cell phones to the Internet, Java is everywhere! Now connected to WinCC OA.

Here you will find an implementation of an API to connect WinCC OA to Java. It is based on the WinCC OA native API and JNI. This version is for WinCC OA 3.14 on Windows. Same works on Linux, but is not in the Zip, if you want it for Linux, just send me an email.

An example for a dpSet in Java see blow, more examples are in the zip/source directory. The JClient class is an easy to use static class. It should be thread safe and callback functions are processed in a separate thread, so that the main WinCC OA thread/loop will not be blocked by callback functions.

Based on that a NoSQL database logger was created, it is able to handle up to 40000-50000 value changes by dpQueryConnect. MQTT was connected to WinCC OA with a few lines of code (see WinCC OA and MQTT).

Download from GitHub: https://github.com/vogler75/oa4j.git

How to use/install? In the zip is a ReadMe.txt with a step by step instruction.

JManager m = new JManager();
m.init(args).start(); 
ret = JClient.dpSet()
 .add("System1:ExampleDP_Trend1.:_original.._value", new FloatVar(Math.random()))
 .add("System1:ExampleDP_SumAlert.:_original.._value", new TextVar("hello world"))
 .await()
 .getRetCode();
Debug.out.log(Level.INFO, "retCode={0}", ret); 
m.stop();

WCCOAjava

BigData/NoSQL, WinCC OA

Internet of Things (IoT) and WinCC OA with MQTT …

Recently i implemented a JAVA API for WinCC Open Architecture. So, it is possible to implement WinCC OA API Manager in Java. I used this to implement a simple API Manager in Java to connect WinCC OA to MQTT.

MQTT is a machine-to-machine (M2M) “Internet of Things” connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium (http://mqtt.org/).

The example is connected to a WinCC OA datapoint in system 1, sends incoming values through MQTT to HiveMQ (http://www.hivemq.com/try-out/, there is a public available MQTT broker for testing available). Another Java API Manager is subscribed to the MQTT broker, receives the values changes by MQTT and sets it to the WinCC OA system 2…

OA-MQTT-OA

 

Allgemein, BigData/NoSQL, WinCC OA

ElasticSearch and WinCC OA …

With a Java-Datalogger we can write events and alerts to different data sinks. For example we can use ElasticSearch and Kibana to discover alerts collected by WinCC OA. Elasticsearch provides full text search, analytics, dashboards. And now it also supports graphs!

“Graph introduces a new way to discover, and explore, the relationships that live in your data. Adding relevance to the process of exploration, a unique combination of graph and search means you can do things like detect fraud, recommend content, and more.”  https://www.elastic.co/de/products/graph

Kibana-Dashboard

Screen Shot 2016-03-31 at 20.48.52Screen Shot 2016-03-31 at 20.34.52

WinCC OA

Voice commands to WinCC OA / Speech recognition…

Start the Control-Manager “BigDataServiceBus.ctl” from the attached ZIP at the server.

SpeechRecognition1

Create a Datapoint-Type with following structure:

Untitled

(Listen: dyn_string; Recognized:string; Speak:string)

Create a Datapoint “SpeechRecognition_<client-hostname>” (of the type you created before).

 

Start the executable from the ZIP: SpeechRecognition.exe <servername>:8080 at the client.

Speech-3

(<servername> is the host where the Control Manager “BigDataServiceBus.ctl” is running)

The program will listen to the words you set at the datapoint element “Listen” (you have to set the datapoint element at least once after starting the executable).  When a word is recognized it will be set to the “Recognized” datapoint element.

Speech-2

You can connect to the Recognized datapoint element and do any action – open an alert screen, open a trend, …

You can also set a text to the “Speak” datapoint element and the text will be spoken by the program at the client.

Download the ZIP: SpeechRecognition-ZIP

BigData/NoSQL, WinCC OA

Cassandra for WinCC OA & Performance…

Did a first test with Cassandra as NoSQL database store for WinCC OA event logging.

Cassandra was easy to install, pretty good write performance and very good read performance. Compared to e.g. OpenTSDB the CPU load on client and server are higher.

Test-Setup: 5000 values/second, 24h, 392 Mil.Records, One Hyper-V server on an i7 3.1Ghz with 4 Cassandra nodes, each node with two cores, disks on a shared storage space device with 4 disks and raid 0 (i know that’s not a real life scenario for a NoSQL cluster). WinCC OA and the Java-Historian-DB-Logger running on another Hyper-V server on an old i5 CPU 750 @ 2.67 Ghz.

Cassandra Performance Stats

Cassandra Workload

 

BigData/NoSQL, WinCC OA

Databases for WinCC OA Logging….

I have implemented a Database Logger for WinCC OA which supports different databases.

I did now a first test to compare some databases with WinCC OA and time series logging.

The logger is writing 1000 values/second to four databases in parallel: MSSQL Server, InfluxDB, Phoenix (based on Hadoop/HBase) and Oracle.

There are now about 160.000.000 values in each database. About 3.600.000 values are added every hour…

Four Ctrl-Scripts are querying data with dpGetPeriod every 10 seconds, random time frames (between 1 and 2 hours of data).

Screen Shot 2015-12-28 at 09.33.27

Oracle seems to be slower (i think this is related to my test environment), but the access and response times are stable with growing amount of data in the database.

Screen Shot 2015-12-28 at 10.37.13

remark: at 10pm the Oracle DB backup starts.

BigData/NoSQL, WinCC OA

WinCC OA Docker Image with InfluxDB & Grafana …on a Mac…

On a Mac install the Docker Toolbox.

https://www.docker.com/docker-toolbox

And click on the “DOCKER CLI”

Screen Shot 2015-12-07 at 11.37.49

The type: docker run -d -p 3000:3000 –name oa1 vogler/winccoa:influxdb startup

Afterwards you will see your new container and at the preview you can already see the Grafana dashboard, click on it and it will open in your browser.

Screen Shot 2015-12-07 at 11.41.24

Screen Shot 2015-12-07 at 16.31.58

You have now a running WinCC OA system which is logging some datapoints to the InfluxDB. There is a simple simulation script running which will set random values to Drive1.*

To open the GEDI you have to go to the DOCKER CLI again and type:

bash-3.2$ docker exec -ti oa1 /bin/bash

You now have a bash in the WinCC OA Container. Set the DISPLAY variable to your client (mac). You need a X-Windows-Server on your client (and allow clients to open windows with “xhost +”). At the Mac you may use xquartz: http://www.xquartz.org

dba3d5e6193f:/ # export DISPLAY=macbookpro:0

dba3d5e6193f:/ # gedi &

Screen Shot 2015-12-07 at 11.48.47or type “console”, it will open the WinCC OA Console.

The NoSQL Logger’s configuration file is: /proj/NoSQL/config/config.logger

There you can set the query part of what you wanna log into the InfluxDB. You can specify the FROM clause of a WinCC OA query, also remote queries are possible (if you connect another OA system to this system (distributed system)).

Then you have to restart the NoSQL Database Logger – it is the WCCOAjava program you see in the console. Right, it is implemented in Java with a native API to WinCC OA.

Screen Shot 2015-12-07 at 11.51.44

Grafana can be used to create dashboards in a fast and easy way…

Screen Shot 2015-12-07 at 17.09.13

WinCC OA

Docker Image for WinCC OA…

I pushed a docker image for WinCC OA to the docker hub.

docker pull vogler/winccoa

It is based on opensuse and WinCC OA 3.13 P009.

At the first startup it will ask for a system number and the system name.

root@debian1:~# docker run -t -i –name=oa1 vogler/winccoa startup
Please enter your desired system name and system number…
sysnum? 1
sysname? oa1
Do you wish to set the project to oa1 with num 1?
1) Yes
2) No
#? 1

It will set the system number and the system name of the project. The projectname itself is currently always “WinCCOA”, but this does not matter, because every WinCC OA project will run in a seperate container.

To startup the container and the WinCC OA project, type:

root@debian1:~# docker start oa1

To enter the container:

docker exec -it oa1 bash

 

To get the ip of the container:

4304c34d368e:/ # ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:AC:11:00:02
inet addr:172.17.0.2 Bcast:0.0.0.0 Mask:255.255.0.0

To connect from a client to your container OA system you can expose ports from the container to the docker host (by using -p <source-port>:<destination-port> option at docker run…) or you may want to add a route to the bridged container network, in my case the bridged network is 172.17.0.0. On a Windows client you can add a route (192.168.1.115 is my Docker-Host-Ip):

route add 172.17.0.0 mask 255.255.255.0 192.168.1.115

for linux: route add -net 172.17.0.0 netmask 255.255.255.0 gw 192.168.1.115

Then you are able to ping your docker container network 172.17.0.2 from your client network (eg. 192.168.1.x) and so you are able to connect a WinCC OA client to the docker container project.

To start the WinCC OA console or the project administration enter the container (as shown before) and set the DISPLAY variable to your client (where you have to run a X-Window-System) and then you can start the console “startConsole”…