Building a serverless Data Lakehouse from spare parts · HackerPapers discussions

Building a serverless Data Lakehouse from spare parts — mentioned by jtagliabuetooso

Wed, 29 Apr 2026 17:35:41 GMT

Cool release. IMO, "Why it's distinctive" is a bit misleading on a few points: certainly the dbt and DX folks can add their POV, but even considering stuff I know / authored ;-), https://arxiv.org/pdf/2308.05368 from 2023 (and following releases) cover branches in a native way (no clone), immutability (re-run), and lineage. Extensions to be considered are different languages (what about Python), and branch semantics. Two immediate questions would be: can you nest branches? How does merge works across systems if you don't control compute?

Paper and discussion history. A mention is not an endorsement.

Building a serverless Data Lakehouse from spare parts — mentioned by jtagliabuetooso

Sat, 21 Jun 2025 00:51:51 GMT

Yeah, terms are confusing sometimes! "Data lakehouse" is weirdly enough a "technical term". The canonical reference is from CIDR https://www.cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf, but we have our own version from VLDB https://arxiv.org/pdf/2308.05368

Paper and discussion history. A mention is not an endorsement.

Building a serverless Data Lakehouse from spare parts — mentioned by jtagliabuetooso

Fri, 17 Nov 2023 17:21:04 GMT

Quack-reduce author here. I think the most interesting thing about duckDB (for us) is that you can "take out the DB" from duckDB and use the built-in engine over a (stateless, ephemeral) table you got from wherever. We didn't do much map-reduce aside some quick tests (i.e. the ones in that repo), but we did pursue a larger vision for serverless lake-house in which SQL and Python co-exist and data flow seamlessly between functions and across languages. In case you're curious, we have a vision paper out and always happy to chat: https://arxiv.org/pdf/2308.05368.pdf

Paper and discussion history. A mention is not an endorsement.