A native ETL and integration engine for high-performance, data-intensive workflows
ETALIS is a native ETL and data integration framework designed for developers who need high-performance, reliable data pipelines without dragging in a full-blown enterprise platform. The core engine is written in modern C++, exposing a set of composable components for HTTP calls, JSON/XML parsing, data transformations and orchestration, with the goal of being usable both directly from C++ and through language connectors (starting with PHP). The vision is to offer a compact, efficient engine that can be embedded into existing architectures or used as a standalone service, with pipelines defined through configuration instead of ad-hoc scripts scattered across systems.
Most organisations that need to integrate systems or move data between APIs, databases and files face the same trade-off:
heavyweight ETL platforms that are powerful but complex, opinionated and hard to embed,
or a fragile collection of custom scripts, cron jobs and one-off integrations that are difficult to maintain.
From a developer perspective, many existing tools:
are not performance-centric,
are hard to test and version,
or don’t integrate well with existing C++/PHP stacks.
ETALIS was created to explore a different approach: a native engine, focused on performance and control, with clear building blocks for HTTP, transformation and extraction, that can be used as a foundation for repeatable, testable and maintainable data pipelines.
Role & responsibilities
As founder and product & technical lead, my responsibilities span both product and engineering:
defining the product vision, positioning and scope of ETALIS as a framework
analysing competitors and existing tools to identify gaps and opportunities
designing the architecture of the C++ core and its main components (HTTP client, extract/transform modules, pipeline orchestration)
defining how configuration (e.g. YAML) can be used to describe reusable pipelines and variables
implementing core modules and their integration points
setting up tests, micro-benchmarks and validation scenarios
planning the roadmap for language connectors (starting from PHP) and early-adopter usage in real projects
Approach
The work on ETALIS combines product thinking, software architecture and hands-on systems programming:
Competitive and problem analysis Reviewed existing ETL and integration platforms to understand what they solve well and where they are too heavy, too opaque or too hard to embed in a developer-centric workflow.
Modular C++ core Designed the framework around a set of core modules (HTTP client, JSON/XML processing, transformation utilities, pipeline execution) that can be used independently or composed in more complex flows.
Configuration-driven pipelines Defined a configuration approach (e.g. YAML) to describe variables, resources, activities and pipeline logic, so that behaviour can be expressed declaratively rather than hard-coded in application logic.
Focus on testability and performance Structured components so they can be tested in isolation with repeatable scenarios, and used performance tests and benchmarks to guide design decisions in the core.
Developer-first mindset Kept the experience of the developer at the centre: clear APIs, predictable behaviour, avoid “magic” that makes debugging impossible, and make it realistic to embed ETALIS in existing stacks rather than forcing a full migration.
Results & impact
ETALIS is currently in active development, with a working core that already supports:
building HTTP-driven data flows using a consistent client abstraction,
transforming and handling structured data (JSON, XML and related formats) through dedicated components,
structuring integrations in a way that can be tested, evolved and eventually exposed to other languages.
Even in its current stage, the framework is already used as a playground to explore:
how to consolidate integration logic that would otherwise be spread across multiple services,
how to achieve better performance and control than typical “glue script” approaches,
how to give developers a clearer mental model of their data pipelines.
The long-term impact targeted by ETALIS is to enable product teams and developers to:
build integrations and ETL processes that are lean, predictable and maintainable,
reuse the same engine across multiple products and environments,
and reduce the amount of custom, one-off integration code that is hard to test and even harder to replace.
Technologies & methods
Technologies: Modern C++, CMake, HTTP client abstractions, JSON/XML processing, configuration via YAML, testing and benchmarking tools
Interested in a lean, developer-centric approach to ETL and integrations? Let’s talk about your context and explore whether ETALIS can play a role in your data and integration strategy.