Science data comes in tables and tensors
Apache Iceberg comes to Arraylake. Tables now live alongside Icechunk arrays in the same catalog, under the same permissions, in your own buckets — queryable from DuckDB, Spark, Trino, Snowflake, and Zax-SQL.
product · arraylake
A multi-cloud data catalog for scientific data: Icechunk repositories and Iceberg tables side by side, in your own buckets, under one set of permissions. Built for humans and agents.
the problem
At most organizations, scientific data lives in random buckets with no discoverability or governance. Nobody can say what exists, who may use it, or which version an analysis ran on.
Lakehouse catalogs understand tables, not the multidimensional arrays most scientific data comes in — so teams with both end up running two systems, or building their own. Each one pays the same costs:
No catalog — nobody can say what data exists, where it lives, or whether it is current.
Credentials everywhere — cloud storage keys spread across laptops, notebooks, and CI jobs.
No version history — datasets change silently underneath the analyses that depend on them.
DevOps drag — scientists spend their time operating infrastructure instead of doing science.
what arraylake does
Your data lives in open formats in your own bucket — arrays in Icechunk repositories, tables in Apache Iceberg. Arraylake holds the metadata about them and the record of who may see and change them.
One catalog for every dataset in your organization: Icechunk repositories and Iceberg tables side by side, with a native understanding of arrays. Browse groups, arrays, dimensions, and attributes in the web app, or let an agent explore them over MCP.
A standard Iceberg REST catalog for the data that belongs in rows. DuckDB, Spark, Polars, pandas, Trino, Snowflake, and ClickHouse read and write your tables with no Earthmover-specific connector. In public beta.
Sign in with Google, GitHub, Microsoft, or your Active Directory. Grant roles on the organization or a single repository, group users and API keys into teams that inherit them, and give automation its own service accounts.
Arraylake brokers scoped, short-lived cloud credentials on every request — to the Python client and to Iceberg engines alike — so users and pipelines never hold long-lived storage keys.
Configure S3, Google Cloud Storage, Azure, Cloudflare R2, or any S3-compatible endpoint. Your bytes stay in your account under your policies — or let us manage storage for you.
Every Icechunk repository has one shared version history: branches for staging changes, tags for versions that will never move, and time travel to any prior snapshot.
Catalog existing NetCDF, HDF5, GRIB, and TIFF archives as virtual datasets. The bytes stay where they are; Arraylake presents them as analysis-ready Zarr.
Trigger downstream work on every commit: send an HTTPS webhook to your own service, or a notification to a Slack channel, configured per repository.
Scheduled jobs expire old snapshots and garbage-collect data nothing references any more, in an optimization window you choose, at no extra compute cost.
credential vending
Your data can live in AWS, Google Cloud, and Azure at once, and your people and pipelines still sign in once. Arraylake checks who is asking, then vends short-lived credentials scoped to exactly the dataset they asked for — so nobody ever holds a long-lived storage key, and access is revoked in one place.
Now: AWS
in practice
Arraylake meets you in the tools you already use. Open a repository by name and read it with Xarray, writing inside a transaction that either lands completely or not at all — or attach the Iceberg catalog to DuckDB and query your tables with SQL.
arrays · Python
import arraylake as al
import xarray as xr
client = al.Client()
repo = client.get_repo("my-org/weather-forecast")
# Read the current state of the data — credentials vended by Arraylake,
# no cloud keys in your environment
session = repo.readonly_session("main")
ds = xr.open_zarr(session.store, group="forecast", consolidated=False)
# Write a new forecast cycle as one atomic transaction
with repo.transaction("main", message="2026-08-23 00Z cycle") as store:
ds_new.to_zarr(store, group="forecast", mode="a", append_dim="time") tables · DuckDB
-- Point DuckDB at Arraylake's Iceberg REST catalog
INSTALL iceberg; LOAD iceberg;
CREATE SECRET al (
TYPE ICEBERG,
TOKEN 'ema_...',
ENDPOINT 'https://api.earthmover.io/iceberg');
ATTACH 'my-org' AS warehouse (TYPE iceberg, SECRET al);
SELECT station, avg(temp_c) AS mean_temp
FROM warehouse."my-tables".observations
GROUP BY station
ORDER BY mean_temp DESC; More in the quickstart guide and the Iceberg engines guide.
from the blog
Apache Iceberg comes to Arraylake. Tables now live alongside Icechunk arrays in the same catalog, under the same permissions, in your own buckets — queryable from DuckDB, Spark, Trino, Snowflake, and Zax-SQL.
Arraylake's refreshed Community Tier unlocks powerful developer capabilities and open-science workflows for free: create your own repos, bring your own object store or use 10 GB of managed storage, and serve data through Earthmover Compute with 50 compute credits per month.
Earthmover has completed its SOC 2 Type 2 audit with Insight Assurance: independent, third-party verification of the security controls we've built the platform around from day one.
Teams is here for Arraylake paid accounts: group users and API keys, grant roles once, and every member inherits them. Group-level access management for growing data teams.
The Arraylake MCP server is now publicly available. Connect Claude, ChatGPT, Cursor, or any MCP-compatible client to your Arraylake data and let an AI assistant discover repositories, inspect schemas, query Flux, and render interactive maps.
A practical guide to Icechunk's garbage collection and expiration operations, explaining when and how to safely reclaim storage from unused snapshots and dangling objects.
where arraylake sits
Humans and agents
Same interfaces, same permissions
Powered by Zax tensor query engine
Bring Your Own Compute
Read and write the open formats directly
Data catalog and governance
Exchange for scientific data
Zarr-compatible open tensor format
Apache Iceberg
Industry-standard open table format
Open source, in your own bucket — readable with or without Earthmover.
Bring your own bucket
Or use storage we manage for you