Stream Applications

In this repository, you will find a collection of components that can meet various data integration use cases and requirements.

This repository builds on top of the Spring Functions Catalog and generates standalone Spring Cloud Stream applications that can run against Spring Cloud Stream’s RabbitMQ or Apache Kafka binder implementations. It is also possible to extend the generator to bundle the Java functions with the other supported binder implementations.

These applications can run standalone or as part of a data flow, such as the one orchestrated using Spring Cloud Data Flow.

Project Structure

The following are the various components of this repository.

• Common Core Components for Applications

• Spring Cloud Stream Applications

• Build Parent

• Release Train

Reusable Spring Cloud Stream Applications

The following table shows the currently available stream applications:

Source	Processor	Sink
Debezium	Aggregator	Analytics
File	Bridge	Cassandra
FTP	Filter	Elasticsearch
HTTP	Groovy	File
JDBC	Header Enricher	FTP
JMS	Header Filter	JDBC
Kafka	HTTP Request	Kafka
Load Generator	Script	Log
Mail	Splitter	MongoDB
MongoDB	Transform	MQTT
MQTT	Twitter Trend	Pgcopy
RabbitMQ		RabbitMQ
AWS S3		Redis
SFTP		Router
Syslog		RSocket
TCP		AWS S3
Time		SFTP
Twitter Message		TCP
Twitter Search		Throughput
Twitter Stream		Twitter Message
Websocket		Twitter Update
XMPP		Wavefront
ZeroMQ		Webscoket
		XMPP
		ZeroMQ

Build

This build requires GraalVM JDK 17 distribution in order to build the script function and processor.

If you are building locally and don’t want to publish to a repository set: export LOCAL=true

Quick Compile

This will compile all jars and publish to local repo.

./quick-compile.sh

Boil the Ocean

This will build and test / integration test all the modules.

./full-build-test.sh

Boil the Ocean (sans tests)

This will build all the modules (including apps) but skip tests.

./full-build-no-test.sh

Core

This will build the core functionality and all functions.

./build-core.sh

You can then build the desired apps.

Application

./build-app.sh . applications/sink/log-sink

Note	In order to disable metrics by default there needs to be application properties configured like in default-application.properties. The build-app.sh script will copy default-application.properties into src/main/resources if no application.properties,yml,yaml or json is present.

Additional Resources

Here is a list of resources where you can find out more about using and developing functions and stream applications:

• Function Composition

• Contributing a New Function or Application to this Repository

Code of Conduct

Please see our Code of Conduct