{"id":734,"date":"2016-05-30T10:54:26","date_gmt":"2016-05-30T08:54:26","guid":{"rendered":"http:\/\/www.rocworks.at\/wordpress\/?p=734"},"modified":"2016-05-30T10:54:26","modified_gmt":"2016-05-30T08:54:26","slug":"streaming-wincc-oa-events-to-apache-kafka-and-spark","status":"publish","type":"post","link":"https:\/\/www.rocworks.at\/wordpress\/?p=734","title":{"rendered":"Streaming WinCC OA Events to Apache Kafka and Spark&#8230;"},"content":{"rendered":"<p>We connected a Event Data Logger, written in Java and connected to WinCC OA by JNI, to <strong>Apache Kafka<\/strong>, a\u00a0<span class=\"subtitle\"><a href=\"http:\/\/kafka.apache.org\/\">A high-throughput distributed messaging system.<\/a><\/span><\/p>\n<p><strong>Apache Spark<\/strong> can read data from Apache Kafka streams.\u00a0Apache Spark\u00a0is a fast and general engine for large-scale data processing.\u00a0Combine SQL, streaming, and complex analytics.\u00a0Spark powers a stack of libraries including <a href=\"http:\/\/spark.apache.org\/sql\/\">SQL and DataFrames<\/a>, <a href=\"http:\/\/spark.apache.org\/mllib\/\">MLlib<\/a> for machine learning, <a href=\"http:\/\/spark.apache.org\/graphx\/\">GraphX<\/a>, and <a href=\"http:\/\/spark.apache.org\/streaming\/\">Spark Streaming<\/a>. You can combine these libraries seamlessly in the same application.<\/p>\n<p><a href=\"http:\/\/www.rocworks.at\/wordpress\/?attachment_id=738\" rel=\"attachment wp-att-738\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-738\" src=\"http:\/\/www.rocworks.at\/wordpress\/wp-content\/uploads\/2016\/05\/WinCCOA-Kafka-Spark.png\" alt=\"WinCCOA-Kafka-Spark\" width=\"529\" height=\"429\" srcset=\"https:\/\/www.rocworks.at\/wordpress\/wp-content\/uploads\/2016\/05\/WinCCOA-Kafka-Spark.png 529w, https:\/\/www.rocworks.at\/wordpress\/wp-content\/uploads\/2016\/05\/WinCCOA-Kafka-Spark-300x243.png 300w\" sizes=\"auto, (max-width: 529px) 100vw, 529px\" \/><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>We connected a Event Data Logger, written in Java and connected to WinCC OA by JNI, to Apache Kafka, a\u00a0A high-throughput distributed messaging system. Apache Spark can read data from Apache Kafka streams.\u00a0Apache Spark\u00a0is a fast and general engine for large-scale data processing.\u00a0Combine SQL, streaming, and complex analytics.\u00a0Spark powers a stack of libraries including SQL &hellip; <a href=\"https:\/\/www.rocworks.at\/wordpress\/?p=734\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Streaming WinCC OA Events to Apache Kafka and Spark&#8230;<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1,29,3],"tags":[],"class_list":["post-734","post","type-post","status-publish","format-standard","hentry","category-allgemein","category-bigdatanosql","category-wincc-oa"],"_links":{"self":[{"href":"https:\/\/www.rocworks.at\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/734","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rocworks.at\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rocworks.at\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rocworks.at\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rocworks.at\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=734"}],"version-history":[{"count":3,"href":"https:\/\/www.rocworks.at\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/734\/revisions"}],"predecessor-version":[{"id":740,"href":"https:\/\/www.rocworks.at\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/734\/revisions\/740"}],"wp:attachment":[{"href":"https:\/\/www.rocworks.at\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=734"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rocworks.at\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=734"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rocworks.at\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=734"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}