Query Compilation for Modern Data Processing Environments
Philipp Grulich, TU Berlin
Abstract
Over the last years, the data processing environment has significantly changed.
Nowadays, data-centric applications perform polyglot queries over increasingly large volumes of data at high velocities.
These workloads became increasingly complex, exceed traditional analytical queries and involve stream processing, machine learning, and
UDF-based operators.
Through this talk, we will revisit the architecture of compilation-based execution engines and propose building blocks to enable efficient data processing in such environments.
Our architecture mitigates current system bottlenecks, enhances hardware utilization, and improves performance without sacrificing the productivity of system engineers.
Finally, we highlight the impact of our contributions on modern data processing platforms like NebulaStream.