Apache Kafka Plug-in

Apache Kafka Connection

The Apache Kafka Software from the Apache Foundation is a free software for streaming data packets. A Kafka system is composed of clusters that connect and interlink to provide a messaging system that allows data to be transported losslessly from sender (producer) to receiver (consumer). The system works according to the logic of Publish/Subscribe. Brokers hold data for consumers so that processing can be controlled by the client. The order of messages remains the same. Messages are marked in Kafka with a topic and optionally with a key. Sent messages contain a payload whose format is not provided, similar to MQTT.
You can find a dedicated video about this topic by visiting our tutorial stream.

Apache Kafka Plug-in

That’s how easy it is.

Apache Kafka connection via drag & drop

Connecting your systems via the Apache Kafka software is straightforward and intuitive with the OPC Router using drag & drop. The two screenshots show you an example of the Kafka application. In the first screenshot, data is fetched with the help of KepServerEX, translated into JSON and sent to an Apache Kafka Broker. The second screenshot shows how data is received from an Apache Kafka, evaluated in JSON and transferred to an MS-SQL database.

Example: OPC data of a cutter are retrieved from KEPServerEX, translated into JSON and sent to an Apache Kafka Broker
Example: Parameter data for a cutter is received as a consumer by Apache Kafka, evaluated (JSON) and transferred to an MS-SQL database so that it is available to the process.

Bringing Industry 4.0 to life

  • Achieving more with coupled systems:
    Coupled systems can exchange information in real time and thus immediately adapt to all requirements. Waiting times and inefficiencies are eliminated by including and viewing all available information.
  • Industry 4.0 with Apache Kafka:
    Industry 4.0 means the best possible networking, self-control and self-optimisation of plants based on information. Apache Kafka provides the necessary infrastructure to achieve this goal on a large scale.
  • About OPC Router:
    OPC Router provides a powerful link for industrial systems to Apache Kafka. Its graphical configuration and robust runtime make it ideal for industrial use.

Let your systems talk to each other via Apache Kafka.

Test OPC Router with the Apache Kafka Plug-in without any obligation and request your free demo now.

Apache Kafka Plug-in application examples

Apache Kafka Plug-in
Plug-in Connections
MS SQL Server

Master data distribution across all plants

For many processes, material master data is an important basis for the parameterisation of individual production processes. For a fast and autonomous decision, these are kept close to the plant as shadow copies to avoid problems and delays when retrieving central data. With Apache Kafka, master data can be made available securely to all interested “consumers”. A cyclical transmission, for example as a JSON package, ensures that the current status is always available directly in the databases at the plants. There is no dependence on the central system. Current production parameters can also be sent back from the plants in the same way, so that an intelligent evaluation and processing can take place centrally (Big Data/AI).

Apache Kafka Plug-in
Plug-in Connections
OPC UA Symbol

Process Data Streaming

Kafka offers a high degree of security and tolerance towards connection problems for central monitoring over large geographical distances and across a wide variety of network architectures. If OPC Router is used to transfer current status/fault and runtime information from systems to Kafka, the cluster system ensures that data is transferred correctly and seamlessly to the central monitoring system. Compared to lightweight IoT infrastructures such as MQTT, Kafka ensures reliable transmission, as is necessary for industrial messaging systems.

Connecting Systems with the OPC Router Apache Kafka Plug-in

Transfer-Objects

The transfer object maps the Kafka producer in the OPC Router. It offers the fields Topic, Key and Payload. Messages are marked with topic and key so that consumers are informed about new messages if they have subscribed to the respective topic. Any data block can be transferred to the payload in OPC Router Connection. Common sources are the JSON or the XML object.

Trigger

If OPC Router should act as a Kafka Consumer, the Kafka Trigger is used. A topic configured for the trigger, which is used by OPC Router to register as a subscriber in Kafka. The trigger activates its connection, in which it is configured, as soon as a new message is available for the topic. If several messages are available, the trigger is retriggered for each message. Even after a system restart, the trigger retrieves all messages not yet retrieved from broker.

Browsing

Browsing of the topics in the broker is possible in the transfer object and in the trigger. After successful configuration of the broker connection, available topics are loaded and made accessible to the operator in list form for selection.

Haven’t found the right information yet?

Here you will find more information on our software as well as an overview of all available OPC Router Plug-ins. We also provide a free demo with which you can test a fully functional version of OPC Router.