Thriving in the No Man's Land between Compilers and Databases

When developing new data-intensive applications, one faces a build-or-buy decision: use an existing off-the-shelf data management system (DMS) or implement a custom solution. While off-the-shelf systems offer quick results, they lack the flexibility to accommodate the changing requirements of long-term projects. Building a solution from scratch in a general-purpose programming language, however, comes with long-term development costs that may not be justified. What is lacking is a middle ground or, more precisely, a clear migration path from off-the-shelf Data Management Systems to customized applications in general-purpose programming languages. There is, in effect, a no man’s land that neither compiler nor database researchers have claimed.

We believe that this problem is an opportunity for the database community to claim a stake. We need to invest effort to transfer the outcomes of data management research into fields of programming languages and compilers. The common complaint that other fields are re-inventing database techniques bears witness to the need for that knowledge transfer. In this paper, we motivate the necessity for data management techniques in general-purpose programming languages and outline a number of specific opportunities for knowledge transfer. This effort will not only cover the no man’s land but also broaden the impact of data management research.

Biennial Conference on Innovative Data Systems Research (CIDR)
Publication Year