Ship fast, break nothing: Engineering rigor in Icechunk with concurrency & fault injection testing
We describe our approach to rigor in Icechunk 2 including concurrency permutation testing and more!
Deepak Cherian
Principal Research Engineer
blog · author
Deepak Cherian
Principal Research Engineer
5 posts
Ship fast, break nothing: Engineering rigor in Icechunk with property and stateful testing
How we engineer rigor into Icechunk and its upstream brethren using property and stateful testing techniques. Also, a cautionary tale.
Deepak Cherian
Principal Research Engineer
Sebastian Galkin
Head of Engineering
Wicked smaht dynamic map tile rendering of Icechunk/Zarr data with xpublish-tiles
Earthmover is launching a new open-source library xpublish-tiles that powers our new Flux Tiles service, which allows Earthmover Platform users to view their data on a slippy map with dynamically rendered tiles at lower zoom levels than was possible previously.
Deepak Cherian
Principal Research Engineer
Matt Iannucci
Staff Engineer
Ergonomic seasonal grouping and resampling in Xarray
Xarray introduces SeasonGrouper and SeasonResampler, two new Grouper objects that enable custom, overlapping, and variable-length seasonal aggregations without workarounds.
Deepak Cherian
Principal Research Engineer
Cloud-native platforms are the natural evolution of atmosphere-ocean open-data practice
Open-data practice in ocean/atmosphere sciences is approximately 170 years old! While it is easy to exclaim, "weather/climate are global, of course we must share data", the actual story is anything but. That story holds valuable inspiration that we can draw from as we face a significant reduction in US climate science research.
Deepak Cherian
Principal Research Engineer
Related Posts
Blog Tags
access controls access-control ai AI weather AIFS announcement arraylake Arraylake Big Data blog post blog-post case study case-study chat cirrus climate risk cloud cloud native compliance conference data cube data infrastructure data workflows dev note dynamical earth observation earth-observation energy engineering enterprise era5 event FAIR feature release financial-services flux fundamentals garbage collection geospatial GOES-R governance grib guest-post hrrr hypothesis icechunk Icechunk IFS LangChain LangGraph marketplace mcp NASA natural language processing NCEI netCDF NetCDF noaa NOAA nws oceanography open data open science open source performance product property-testing public-sector python Python Radar rust Rust security serverless shuttle soc2 tabular data teams technical tensorops tensors testing third-wheel toxiproxy trust vector data virtual-zarr virtualizarr weather wildfire WOD xarray Xarray zarr Zarr