Open Source Data Engineering & Integration Software
Data Engineering & Integration software automates the movement and transformation of information between databases, APIs, and data warehouses. These tools manage complex ETL and ELT pipelines by scheduling tasks and orchestrating workflows through directed acyclic graphs. This type of software serves data engineers and analysts who need to synchronize large datasets and transform raw logs into structured reports. Self-hosting these apps ensures that sensitive database credentials and internal pipeline configurations remain on private infrastructure rather than in external cloud environments.
This page lists 11 open source tools in the Data Engineering & Integration category. The most popular are Apache Airflow, CyberChef and Kestra. Most use the Apache-2.0 or Elastic-2.0 license, and 11 offer an official Docker image.
Apache Airflow
Programmatically author, schedule, and monitor workflows using directed acyclic graphs to orchestrate complex data pipelines.
CyberChef
A web application for performing data manipulation tasks including encoding, encryption, compression, and parsing in a browser.
Kestra
Orchestrate data, AI, and infrastructure workflows using a declarative YAML interface and a large plugin ecosystem.
Airbyte
Moves data from APIs, databases, and files into warehouses, lakes, and AI applications using ELT pipelines.
Apache Druid
A high performance real-time analytics database designed for fast queries and high concurrency data ingestion.
Mage
Build and orchestrate data pipelines using a notebook style interface with support for Python, SQL, and R.
CloudQuery
An open source ELT platform that syncs cloud infrastructure metadata from various SaaS sources into data warehouses.
Lightdash
Transforms dbt models into interactive dashboards to enable self-serve data analytics and business intelligence reporting.
Jitsu
Collect event data from websites and applications to stream into data warehouses and other external services.
RudderStack
Collect customer data from applications and websites to activate it in warehouses and business tools.
Elementary Data
A dbt-native data observability tool for detecting anomalies and monitoring data quality within data warehouse pipelines.
Join our newsletter to get shiny new open source software delivered to your inbox. Unsubscribe anytime.