HackerPapers

arXiv:2210.03718cs.DB

Integration of Skyline Queries into Spark SQL

Lukas Grasmann, Reinhard Pichler, Alexander Selzer

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

Skyline queries are frequently used in data analytics and multi-criteria decision support applications to filter relevant information from big amounts of data. Apache Spark is a popular framework for processing big, distributed data. The framework even provides a convenient SQL-like interface via the Spark SQL module. However, skyline queries are not natively supported and require tedious rewriting to fit the SQL standard or Spark's SQL-like language. The goal of our work is to fill this gap. We thus provide a full-fledged integration of the skyline operator into Spark SQL. This allows for a simple and easy to use syntax to input skyline queries. Moreover, our empirical results show that this integrated solution of skyline queries by far outperforms a solution based on rewriting into standard SQL.

arXiv comments
Journal reference
DOI
Subjects
cs.DBDatabases

On Hacker News

Linked by 1 identifiable commenter in 1 comment across 1 thread, from Oct 2022 to Oct 2022. Found in 1 HN submission from Oct 2022 to Oct 2022. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. Integration of Skyline Queries into Spark SQL 1 comment

    • "...skyline queries are not natively supported and require tedious rewriting to fit the SQL standard or Spark’s SQLlike language... ...The goal of our work is to fill this gap. We thus provide a full-fledged integration of the skyline operator into Spark SQL. This allows for a…

      belterView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.