Skew Strikes Back: New Developments in the Theory of Join Algorithms · HackerPapers discussions

Skew Strikes Back: New Developments in the Theory of Join Algorithms — mentioned by j-pb

Thu, 28 Jul 2022 11:47:34 GMT

That blogpost is a godsend, it provides a great explanation and intuition, and resolved a few questions that I was left with after reading a lot of WCO join papers. Good nitpick, I think you can construct larger rings where the difference becomes larger than one, but I might be wrong. The dynamic variable ordering trick makes a huge difference in practice, especially when skew is in play. (https://arxiv.org/pdf/1310.3314.pdf) In general you're right, WCO joins are a relatively young field of study and sometimes struggle with large constant factors, but they are maturing quickly and in a…

Paper and discussion history. A mention is not an endorsement.

Skew Strikes Back: New Developments in the Theory of Join Algorithms — mentioned by j-pb

Sun, 03 Jan 2021 10:24:49 GMT

It's all about efficient range reads and seek operations. You can compute set operations pretty efficiently using sorted indices that support a "find next or greater" operation. For intersection, take the first value from the first set as the smallest candidate of the output set, then call call candidate := next(candidate) on each set, updating the candidate with the next smallest possible value, and adding it to the output set when a candidate was returned by all sets. The algorithm is worst case optimal up to a log factor, and can even be extended to perform generic constraint programming…

Paper and discussion history. A mention is not an endorsement.

Skew Strikes Back: New Developments in the Theory of Join Algorithms — mentioned by j-pb

Mon, 09 Dec 2019 14:09:04 GMT

Out of the examples I would take a look at the LogicBlox Stuff, they gave join algorithm research a big push with their Leapfrog-Triejoin. The following papers ascend in difficulty, and are based on each other, but offer a fascinating glimpse into the connection of DPLL SAT solving and Conjunctive Query evaluation. https://openproceedings.org/2014/conf/icdt/Veldhuizen14.pdf https://arxiv.org/abs/1310.3314 https://arxiv.org/abs/1404.0703 As for reasoning and decidability, I think we need to clarify the decidability we're talking about. Decision problems are decidable or undecidable, and when…

Paper and discussion history. A mention is not an endorsement.

Skew Strikes Back: New Developments in the Theory of Join Algorithms — mentioned by jamii

Sat, 24 Jan 2015 00:32:28 GMT

Some of the more exciting ideas I've seen recently: http://arxiv.org/abs/1310.3314 - Understanding the core difficulty in answering relational queries and examples of problems for which current query optimisers always produce plans which are asymptotically suboptimal http://arxiv.org/abs/1404.0703 - The first theoretical analysis which can relate the choice of indexes to worst-case bounds. Presents a single join algorithm which is asymptotically optimal on every problem without even using cardinality estimates. http://arxiv.org/abs/1210.0481 - A join algorithm that meets some of the bounds of…

Paper and discussion history. A mention is not an endorsement.