Convert Zarr to Icechunk with just a few clicks!
Arraylake makes it trivial to ingest and view existing Zarr stores in the cloud.
Tom Nicholas
Software Engineer
blog · author
Tom Nicholas
Software Engineer
7 posts
How to make scientific data archives AI-ready
Unlocking scientific data archives for human, agentic and AI model usage requires a truly scalable data storage architecture. We show from first principles what architecture is necessary.
Tom Nicholas
Software Engineer
Virtually Gribberish - Bringing Icechunk clarity to GRIB archives
GRIB data is notoriously hard to work with, so we made it easy to create virtual Icechunk stores from GRIB archives such as the NOAA National Blend of Models, allowing cloud-optimized access to GRIB data archives without copying the data.
Tom Nicholas
Software Engineer
Matt Iannucci
Staff Engineer
Virtual chunks that just work — but securely
Delivering enterprise-grade access control for virtual chunks - surprisingly subtle!
Tom Nicholas
Software Engineer
Sebastian Galkin
Head of Engineering
Old format, no problem!: Cloud-optimizing the GOES-16 archive as Virtual Zarr
Access billions of chunks of satellite imagery as a single Zarr store, without copying any data!
Tom Nicholas
Software Engineer
Plotting NYC heatwaves during NYC Climate Week
A hands-on walkthrough of calculating historical heatwave frequency over NYC using ERA5 reanalysis data on the Earthmover platform with Arraylake, Icechunk, Xarray, and open-source climate tools.
Tom Nicholas
Software Engineer
Fundamentals: What is Cloud-Optimized Scientific Data?
Why traditional scientific file formats like NetCDF perform poorly on cloud object storage, and how cloud-optimized formats like Zarr and Icechunk solve the problem by separating metadata and chunking data.
Tom Nicholas
Software Engineer
Related Posts
Blog Tags
access controls access-control agents ai AI weather AIFS announcement architecture arraylake Arraylake aurora Big Data biology blog post blog-post case study case-study chat cirrus climate risk cloud cloud native compliance compute conference data archives data cube data infrastructure data workflows dev note dynamical earth observation earth-2 earth-observation energy engineering enterprise era5 event FAIR feature release financial-services flux forecast fundamentals garbage collection geospatial GOES-R governance grib guest-post hrrr hugging-face hypothesis iceberg icechunk Icechunk IFS LangChain LangGraph marketplace mcp NASA natural language processing NCEI netCDF NetCDF noaa NOAA nowcasting nvidia nws oceanography open data open science open source performance product property-testing public-sector python Python Radar rust Rust scverse security serverless shuttle single-cell soc2 sql tables tabular data teams technical tensorops tensors testing third-wheel toxiproxy trust vector data virtual-zarr virtualizarr virtualization weather wildfire WOD xarray Xarray zarr Zarr zax zero-copy-ingest