blog · author
Head of Engineering
10 posts
We're introducing Zax, a new query and compute engine for multidimensional scientific data, and Zax-SQL, the first interface to it. Zax plans your computation for you, so you are no longer responsible for tuning chunk sizes, worker counts, and memory limits.
CEO & Co-founder
Head of Product
CTO & Co-founder
Principal Research Engineer
Head of Engineering
Delivering enterprise-grade access control for virtual chunks - surprisingly subtle!
Software Engineer
Head of Engineering
How we engineer rigor into Icechunk and its upstream brethren using property and stateful testing techniques. Also, a cautionary tale.
Principal Research Engineer
Head of Engineering
Earthmover's filtered subscriptions allow data providers to create secure, read-only views into multidimensional data cubes, enabling more granular cloud-native data exchange between provider and consumer.
CEO & Co-founder
Head of Engineering
Senior Engineer
When we released Icechunk 1.0 last July, we declared it production-ready and committed to format stability. Since then, adoption has exceeded our expectations. Teams across weather forecasting, climate science, neuroscience, and AI/ML have pushed Icechunk into scenarios we didn't fully anticipate--r
CEO & Co-founder
Head of Engineering
A preview of Icechunk 2, featuring node rename, chunk reindexing, rectilinear grids, repository-level metadata, and significant performance improvements with a smooth migration path from Icechunk 1.
Head of Engineering
A practical guide to Icechunk's garbage collection and expiration operations, explaining when and how to safely reclaim storage from unused snapshots and dangling objects.
Head of Engineering
Icechunk stores versioned array data efficiently by never copying or rewriting existing chunks, so each new version only consumes storage for the data that actually changed. Older versions can be expired and garbage-collected when they are no longer needed.
Head of Engineering
A practical walkthrough of how Icechunk uses transactions and conflict detection to guarantee data consistency when multiple processes write concurrently. The post demonstrates optimistic concurrency control and the rebase workflow using a bank-account transfer example.
Head of Engineering
Demystifying how S3 prefix sharding actually works and demonstrating that Icechunk can scale to hundreds of thousands of requests per second, far beyond the single-prefix limit.
Head of Engineering
