Tag

#performance

2 posts

I/O-Maxing Tensors in the Cloud

November 25, 2025

The critical role of I/O in data science and AI/ML For both analytics and AI workloads, fast I/O is the foundation of good performance. Most of these workloads involve fluxing a large amount of data from storage into RAM, and then to the CPU or GPU. In the cloud, where data reside on object storage,

Ryan Abernathey

CEO & Co-founder

Blog Post

Exploring Icechunk scalability: untangling S3's prefix story

April 10, 2025

We at Earthmover recently released the Icechunk tensor storage engine, a novel cloud-optimized storage format and library for large-scale array data. Built on Rust’s tokio async runtime, Icechunk delivers impressive gains in performance over today’s array storage engines (e.g. Zarr V2, netCDF). The