Virtual chunks that just work — but securely
Delivering enterprise-grade access control for virtual chunks - surprisingly subtle!
Tom Nicholas
Software Engineer
Sebastian Galkin
Staff Engineer
Tom Nicholas
Software Engineer
4 posts
Old format, no problem!: Cloud-optimizing the GOES-16 archive as Virtual Zarr
Access billions of chunks of satellite imagery as a single Zarr store, without copying any data!
Tom Nicholas
Software Engineer
Plotting NYC heatwaves during NYC Climate Week
A hands-on walkthrough of calculating historical heatwave frequency over NYC using ERA5 reanalysis data on the Earthmover platform with Arraylake, Icechunk, Xarray, and open-source climate tools.
Tom Nicholas
Software Engineer
Fundamentals: What is Cloud-Optimized Scientific Data?
Why traditional scientific file formats like NetCDF perform poorly on cloud object storage, and how cloud-optimized formats like Zarr and Icechunk solve the problem by separating metadata and chunking data.
Tom Nicholas
Software Engineer
Related Posts
-
Multiscale overviews in Arraylake: GeoZarr-Powered Pyramids in Flux - and How to Generate Them in Icechunk Jun 16, 2026 -
The University of Southern Mississippi makes 90 million ocean profiles conversational: AQUAVIEW's approach to the World Ocean Database Jun 15, 2026 - Virtual chunks that just work — but securely Jun 10, 2026
-
Icechunk Adopted by the National Weather Service: Earthmover Joins Booz Allen on NWS CIRRUS Jun 4, 2026
Blog Tags
access controls ai AI weather AIFS announcement arraylake Arraylake Big Data blog post blog-post case study case-study chat cirrus climate risk cloud cloud native conference data cube data infrastructure data workflows dev note earth observation earth-observation energy engineering enterprise era5 event FAIR feature release flux fundamentals garbage collection geospatial GOES-R hypothesis icechunk Icechunk IFS LangChain LangGraph marketplace mcp NASA natural language processing NCEI netCDF NetCDF noaa NOAA nws oceanography open data open science open source performance property-testing python Python Radar rust Rust security serverless shuttle tabular data technical tensorops tensors testing third-wheel toxiproxy vector data virtual-zarr virtualizarr weather wildfire WOD xarray Xarray zarr Zarr