Reordering Rows for Better Compression: Beyond the Lexicographic Order — mentioned by gopalv

Thu, 22 Oct 2026 02:10:18 GMT

> does duckdb support querying custom storage formats? No, this is still parquet being loaded in. The "trick" is running through sort order combinations which result in maximum packing in a Dremel style data format (like Parquet). Here's the trick explained in enough detail to understand, but not enough to copy as-is https://cloud.google.com/blog/products/bigquery/inside-capac... Search for the section with "Capacitor builds an approximation model that takes into account all relevant factors and comes up with a reasonable solution." or a more detailed dive in the paper (by the favourite…

Paper and discussion history. A mention is not an endorsement.

Reordering Rows for Better Compression: Beyond the Lexicographic Order · HackerPapers discussions

Reordering Rows for Better Compression: Beyond the Lexicographic Order — mentioned by gopalv