Skip to content

A stream processing runtime that allows connecting any streaming data source to any destination and act on it

License

Notifications You must be signed in to change notification settings

wso2/product-streaming-integrator

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4,059 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

WSO2 logo

WSO2 Streaming Integrator

License stackoverflow Join the community on Discord X

WSO2 Streaming Integrator (SI) is a streaming data processing server that allows you to integrate streaming data and take action based on streaming data.

Streaming Integrator/ Workflow

WSO2 SI is powered by Siddhi.io, a well-known cloud native open source stream processing engine. Siddhi lets users write complex stream processing logic using a SQL-like language known as SiddhiQL.

To learn more about WSO2 Micro Integrator please visit WSO2 Streaming Integrator.

Why Streaming Integrator?

WSO2 SI can be effectively used for:

  • Realtime ETL with files, DBs, SaaS apps, HTTP endpoints, etc.
  • Working with streaming messaging systems such as Kafka and NATS
  • Streaming data integration
  • Executing complex integrations based on streaming data

You can aggregate, transform, enrich, analyze, cleanse, and correlate streams of data on the fly using Siddhi queries and constructs.

SI tooling provides a web-based IDE that allows you to build Siddhi applications with a drag-and-drop graphical editor or a streaming SQL code editor.

SI has native support for Kubernetes with a K8s Operator designed to provide a convenient way of deploying SI on K8s.

Get started

See the following topics to get started with the product:

See the tutorials to try out the main integration use cases.

Contribute to Streaming Integrator

Build from the source

To build the WSO2 Streaming Integrator from source, follow the steps below.

  1. Clone or download the source code from this repository.
  2. Run mvn clean install from the root directory of the repository.
  3. The generated Streaming Integrator distribution can be found at streaming-integrator/modules/distribution/target/-streaming-integrator-<version>.zip

When building with dependencies, you need to build in the following order.

  1. siddhi
  2. carbon-analytics-common
  3. carbon-analytics
  4. carbon-dashboards
  5. carbon-ui-server

Deploy Streaming Integrator

Deploy in Docker

WSO2 Streaming Integrator has a Docker distribution so that it can be deployed in any container-orchestration system. The Docker image can be built from the source, or downloaded directly from Docker Hub.

Build the Docker image

Before building the docker image, install Docker on the host machine.

To build the Docker image from the source, issue the following command from the root directory.
mvn clean install -Ddocker.skip=false

Get the image from Docker Hub

To get the Docker image from Docker Hub, issue the following command.

docker pull wso2/wso2si

Deploy in Kubernetes

WSO2 Streaming Integrator can be deployed in a Kubernetes cluster using Siddhi Operator. Siddhi operator enables the deployment of Siddhi applications directly in your Kubernetes cluster using a Kubernetes Custom Resource. For more details see Installing Streaming Integrator Using Kubernetes.

Report issues

Open an issue

Help us make our software better! Submit any bug reports or feature requests through WSO2 Streaming Integrator GitHub repo.

Report security issues

Please do not report security issues via GitHub issues. Instead, follow the WSO2 Security Vulnerability Reporting Guidelines.

Join the community!

Get commercial support

You can take advantage of a WSO2 on-prem product subscription for the full range of software product benefits needed in your enterprise, like expert support, continuous product updates, vulnerability monitoring, and access to the licensed distribution for commercial use.

To learn more, check WSO2 Subscription.

Can you fill out this survey?

WSO2 wants to learn more about our open source software (OSS) community and your communication preferences to serve you better. In addition, we may reach out to a small number of respondents to ask additional questions and offer a small gift.

The survey is available at: WSO2 Open Source Software Communication Survey


(c) Copyright 2012 - 2025 WSO2 Inc.