Menu

#Parquet

14 posts

Feed·
13 of 14 posts
0Read More

Tparquet - Parquet for time series. Tick data at about half their ZSTD size, lossless, and still Parquet.

If you want to save some space on your market data hard drive vs parquet, we have a very small fork of parquet libraries/loaders that'll give you ~50% better compression (with pcodec), and we're also backing initiative to make it an official parquet…

15s
Parquet vs Avro vs ORC for Data Engineering
📰
0Read More

Parquet vs Avro vs ORC for Data Engineering

DEV Community: database·Keshav·6 days ago
#SsZAdl7k
#devtodatabase#devto#dev#parquet#avro

Compare Parquet, ORC and Avro by layout, compression, schema evolution and use case, with a measured size comparison and how column pruning works in Spark.

15s
0Read More

[self-promotion] HNMovies: 2,281 films & documentaries extracted from 49.3M Hacker News comments (2006–2026, MIT Parquet/JSONL, updated weekly)

Datasets·init0·12 days ago
#P51HO2uQ
#datasets#redditdatasets#title#doesn#match

Hugging Face dataset (MIT, Parquet + JSONL, updated weekly): https://huggingface.co/datasets/gnumanth/hnmovies Interactive explorer: https://hnmovies.org I processed all 239 monthly Parquet shards of the open-index/hacker-news archive (49,324,288 stories…

15s
Asynchronous I/O in DuckDB: Work, Thread, Work
🖼️
0Read More

Asynchronous I/O in DuckDB: Work, Thread, Work

Hacker News·3 months ago
#xv7o25Xd
#duckdb#parquet#csv#photo#article

Starting with v2.0, scheduled for fall 2026, DuckDB will support asynchronous reads of Parquet and CSV files. This can significantly speed up queries when synchronous I/O does not saturate the available bandwidth, as is typical in EC2/S3 compute-storage…

15s
Three TODOs, three weeks, one weekend: finishing pq v0.14
🖼️
0Read More

Three TODOs, three weeks, one weekend: finishing pq v0.14

DEV Community·thehwang·6 months ago
#dP28fbFF
#dev#json#copilot#parquet#duckdb

I built jq for Parquet, then ghosted three v0.14 issues for 21 days. Here's how a hashtag, Copilot, and Cursor finally shipped them — and what I learned when DuckDB lied to me on the cover image.

15s
A Quick Guide to Reading and Writing Files in PySpark
🖼️
0Read More

A Quick Guide to Reading and Writing Files in PySpark

Medium·Arpita Ghosh·7 months ago
#xQTYh2oi
#clip0_4116_10682#pyspark#blogs#write#parquet

A Quick Guide to Reading and Writing Files in PySpark How to Read and Write CSV, Parquet, and JSON Files in PySpark. From the previous blogs about PySpark, you had some idea of why you need to …

15s