I recently started using my panel at home and felt lost without having such API’s.
So I decided to change that:
✅ I built a lightweight REST and a GraphQL server for WinCC Unified.
✅ Implemented in Rust, they’re native, lightweight, single executables.
✅ Low memory and CPU footprint – thanks to Rust!
✅ With GraphQL, there is also support for tag & alarm subscriptions! 🔄
I always prefer using GraphQL — I find it tedious to constantly refer to separate REST documentation. But for some simple use cases, REST is perfectly fine.
In industrial automation, logging and monitoring are crucial for maintaining system health and troubleshooting issues. Siemens WinCC Unified provides built-in tracing capabilities that. In this post I will show how to capture that traces to Elasticsearch to allow seamless log collection, storage, and visualization.
Step 1: Capturing WinCC Unified Traces
WinCC Unified provides a trace tool that simplifies the process of collecting traces. The tool allows logs to be written to files, which can then be read by Logstash (a tool to process log files).
In that example we will write the log files to C:\Tools\logstash-siemens\logs directory.
For WinCC Unified Panels, trace forwarding can be enabled, allowing traces to be captured with the WinCC Unified trace tool on a PC. The traces will then be also be written to files on the same PC (by the tool you started at Step 1).
Recently, I conducted a test to analyze the bandwidth usage of MQTT, and one feature stood out as particularly impactful: the Topic Alias feature in MQTT 5. This feature can significantly reduce the overhead associated with long topic names, which is especially relevant in UNS (Unified Namespace) implementations using ISA-95-style topics, where topic names tend to be lengthy and sometimes consume more bytes than the payload itself.
💵 It can make an impact if you are being charged based on the amount of data transferred (cloud).
🚨 The Importance of Topic Alias
The Topic Alias feature allows a client to map a topic name to a shorter alias, reducing the amount of data transmitted. This can drastically lower bandwidth usage when transmitting messages with long topic names.
Using Topic Alias during publishing is straightforward but requires the topic-to-alias mapping logic to be implemented in the client program (which is not difficult to do).
On the subscriber side, the implementation should ideally be seamless. In theory, a subscriber needs only to set the maximum allowed alias number during the connection phase. This should make the feature easy to adopt for receiving applications.
👉 During my tests, I discovered something surprising: EMQX was the only broker (of the ones I have tested) to support Topic Alias for subscriptions(!) out of the box. With others I was unable to enable this functionality.
👉 To note: most articles about Topic Alias focus primarily on its use during publishing, not on subscriptions. I was focused on subscriptions.
Within this thesis we implement a prototype of a smart home process control system for increasing the consumption of self-produced photovoltaic energy. To increase the consumption of self-produced energy, the energy is stored in a thermal storage by charging it automatically in case there is excess energy.
Due to the increasing popularity of small energy-producing plants (mainly photovoltaic systems) the allowances for energy fed into the public electricity grid decreases. The income from this kind of electricity is lower than the price of purchased electricity. By increasing the self-consumption of energy, the profitability of a photovoltaic system can be significantly improved.
Based on the findings of the thesis 1 „Eigenverbrauchsoptimierung vonPhotovoltaikstrom in Einfamilienhäusern“ (Vogler, 2014), the components to build such an integrated control system are implemented. The goal is to build an automated control system, that enforces power consumption when there is enough self-produced electricity available.
Due to long product cycles, companies in the process control-business are comparatively innovation averse and expensive. But with systems such as Arduino or Raspberry Pi, which allow an easy entry to the embedded programming and control technology, an intelligent control system for self-consumption of photovoltaic electricity by thermal storage can be implemented with low cost.
Our system consists of loosely coupled separated components. By the use of service interfaces, the components are highly distributed, which follows the current IT trend of ” IoT – Internet of Things “.
Thesis 1: Optimizing self-consumption of photovoltaic electricity in single-family homes
In that example we take SparkplugB messages from a MQTT Broker, decode it and write it to Snowflake. And we take some OPC UA nodes and write it also to the same table.
Create a database and a schema for your destination table.
CREATE OR REPLACE SCHEMA scada;
Create the table for the incoming data:
CREATE TABLE IF NOT EXISTS scada.gateway (
system character varying(1000) NOT NULL,
address character varying(1000) NOT NULL,
sourcetime timestamp with time zone NOT NULL,
servertime timestamp with time zone NOT NULL,
numericvalue float,
stringvalue text,
status character varying(30),
CONSTRAINT gateway_pk PRIMARY KEY (system, address, sourcetime)
);
Add a Snowflake logger section to the gateways config.yml. In that example we take SparkplugB messages from a MQTT Broker, decode it and write it to Snowflake. And we take some OPC UA nodes and write it also to the same table.
The Url you can find in the Snowflake web console by going to Admin/Accounts and then hover over the “Locator” column.
Start the Gateway
> git checkout snowflake
> cd automation-gateway/source/app
> ../gradlew run
Note: Using gradlew to start the gateway is not recommended for production. Instead, consider using a Docker image or the files from the build/distribution for a more robust setup..
It’s super easy to get started. Use the public available MonsterMQ at test.monstermq.com at port 1883. Just publish a JSON string to any topic under “Test”, like “Test/Sensor1” with a payload like this: {“value”: 1}, and you’ll see the value reflected in the Grafana dashboard in real time.
I’m currently publishing temperature sensor values to the public broker from my home automation using automation-gateway.com. Just that you see some values in the dashboard.
So, if you need a broker to store your IoT data directly into TimescaleDB without the need for any additional components, consider using MonsterMQ. It’s free and available at MonsterMQ.com.
Here is the exmaple docker-compose.yml file of the public availble test.monstermq.com broker:
My first tests with QuestDB on 10 years of home automation data (1.4 billion rows) are promising.
👉 Fast ingestion of parquet files (~1 hour on an old Intel NUC i5)
I have stored my data in parquet files, one per month, and imported it with a simple Python script. The import on my really old Intel NUC i5 took only about one hour. I’ve never been able to do this so quickly with any other database.
🤔 Btw.: I think storing data in #parquet files, or any other open table format, like Apache Iceberg, is one of the best choices to keep data. Because it’s independent of a database engine.
👉 Familiar SQL syntax. I experienced that QuestDB has a powerful SQL engine. I converted some Postgres SQL statements to QuestDB without big issues or changes. And I love SQL 💚
👉 Great query response times – see image. Not a representative query, but still impressive speed.
👉 By using ZFS with compression the used disk space can be reduced to a good value.
Do you want to log your OPCUA data to QuestDB? I have added this option to the automation-gateway.com seven days ago.
For the OpenSearch and Elasticsearch fans: the automation-gateway.com can now log OPC UA, MQTT, PLC4X values to OpenSearch!
Makes sense if you have lot of text values, to leverage the full text search capabilities. But for timeseries it is nice as well, the Kibana tool is great to create dashboards.
🥳 Last weekend I found some time to try out an upcoming feature in WinCC Open Architecture 3.20. With the Node.js integration you can write your business logic in JavaScript with native connectivity to WinCC OA. You can take full advantage of the Node.js ecosystem.
🧐 But I am a Java enthusiast and I love the JVM ecosystem. Have you ever heard about GraalVM? It is an advanced JDK written in Java. And it has a Node.js Runtime, which gives you the power of Node.js plus the power of polyglot programming, you can mix JavaScript with Java.
👍 And it turned out that the GraalVM Node.js Runtime also works with WinCC OA! It took me some time to figure out how the polyglot interoperability works, but now I have a first draft of a Java-Library which makes it easy to use Java and OA in the Node.js environment.
🤩 I can now use Java to develop great solutions with WinCC OA.
You can now add a Node. In that example we will use the dpQuery node and use “SELECT ‘_online.._value’ FROM ‘Meter_Input_WattAct.'” as query. So we just query the online value of one tag.
You have to configure the Server by clicking on the pencil button. This points to the before started Websocket Control Manager and you have to set the username and password we have added in one of the previous steps.