HackerPapers

arXiv:1905.12133cs.DB

One SQL to Rule Them All

Edmon Begoli, Tyler Akidau, Fabian Hueske, Julian Hyde, Kathryn Knight, Kenneth Knowles

Abstract

Real-time data analysis and management are increasingly critical for today`s businesses. SQL is the de facto lingua franca for these endeavors, yet support for robust streaming analysis and management with SQL remains limited. Many approaches restrict semantics to a reduced subset of features and/or require a suite of non-standard constructs. Additionally, use of event timestamps to provide native support for analyzing events according to when they actually occurred is not pervasive, and often comes with important limitations. We present a three-part proposal for integrating robust streaming into the SQL standard, namely: (1) time-varying relations as a foundation for classical tables as well as streaming data, (2) event time semantics, (3) a limited set of optional keyword extensions to control the materialization of time-varying query results. Motivated and illustrated using examples and lessons learned from implementations in Apache Calcite, Apache Flink, and Apache Beam, we show how with these minimal additions it is possible to utilize the complete suite of standard SQL semantics to perform robust stream processing.

arXiv comments
Journal reference
DOI
doi:10.1145/3299869.3314040
Subjects
cs.DBDatabases

On Hacker News

Linked by 6 identifiable commenters in 6 comments across 5 threads, from Jul 2019 to Jun 2021. Found in 1 HN submission from May 2019 to May 2019. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments5 threads

  1. Q – Run SQL Directly on CSV or TSV Files 1 comment

    • Looking at the comments I see there are more and more such projects, that's great! Declarative interfaces rule! Anyways, I too felt a void to be filled with tools like these, so I (and a couple of other folks) developed OctoSQL[0], check it out if you like this. It lets you…

      cube2222View on HNParent context

  2. Internal Consistency in Streaming Systems 1 comment

    • > this would still leave the problem where each transactions causes two deletes and two inserts Indeed! You might be interested in the '6.5 Materialization Controls' section of this paper: https://arxiv.org/pdf/1905.12133.pdf They intend to add these extensions: SELECT ... EMIT…

      BenoitPView on HNParent context

  3. Why isn't differential dataflow more popular? 1 comment

    • Where and how in dataflow is late data being handled? How can I configure in which ways refinements relate? These questions are the standard "What Where When How" I want to answer and put into code when dealing with streaming data. I was not able to find this in the…

      james_woodsView on HNParent context

  4. Show HN: Jet – in-memory, fault-tolerant, distributed stream processing 2 comments

    • It's not very different than normal SQL. Imagine that instead of a finite result set, you instead have a never ending result set. You can also roughly map operations like windowed aggregation into SQL with some additional syntax. This paper gives a pretty good overview, even…

      cangencerView on HNParent context

    • Are you referring to this? https://arxiv.org/abs/1905.12133 HN Link: https://news.ycombinator.com/item?id=20059006

      chrisjcView on HNParent context

  5. One SQL to rule them all 1 comment

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.