Throughout these notes we make extensive use of a cloud-hosted version of the ERA5 Reanalysis data Hersbach et al. (2020) made available by Earthmover.
The dataset is available from from Earthmover’s Arraylake Marketplace. We are using this version of ERA5 because it is in what’s known as “Analysis-Ready, Cloud-Optimized” (ARCO) format. As we’ll show, this means we can be very efficient in accessing and working with amounts of data much larger than will fit into memory on a single computer.
There is no charge to access the public version of the dataset, but login credentials are required. You need to sign up for a free Earthmover account before running any of this code. See https://
import numpy as np
import matplotlib.pyplot as plt
import xarray as xr
import cartopy.crs as ccrs
from arraylake import Client as alClientalclient = alClient()# This step requires a free Earthmover account
alclient.login()repo = alclient.get_repo("earthmover-public/era5")
session = repo.readonly_session("main")The ERA5 dataset is distributed in several different stores. The “single” store contains single-level variables such as sea level pressure.
single_spatial = xr.open_zarr(session.store, group="single/spatial", chunks={})single_spatialThe total size of the data store in Terabytes:
single_spatial.nbytes / 1E12122.453669658632Despite this vast size, we can quickly and easily access a single time step:
single_spatial.msl.sel(valid_time="2024-01-01T00").plot()
There are two independent copies of the “single” data store, and they differ in “chunking” which refers to how the data arrays are broken up into small elements.
The “single/spatial” store is optimized for making maps, while “single/temporal” is optimized for timeseries analysis. The data themselves should be identical.
single_temporal = xr.open_zarr(session.store, group="single/temporal", chunks={})single_temporalPressure-level data¶
pressure_temporal = xr.open_zarr(session.store, group="pressure/temporal", chunks={})pressure_temporalpressure_temporal.nbytes / 1E12326.543081293936pressure_temporal.pressure_levelpressure_temporal.tA derived quadratic field¶
Compute the product
vT_temporal = pressure_temporal.v * pressure_temporal.t
vT_temporalpressure_spatial = xr.open_zarr(session.store, group="pressure/spatial", chunks={})vT_spatial = pressure_spatial.v * pressure_spatial.t
vT_spatialvT_spatial.sel(pressure_level = 500.)- Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz‐Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., Simmons, A., Soci, C., Abdalla, S., Abellan, X., Balsamo, G., Bechtold, P., Biavati, G., Bidlot, J., Bonavita, M., … Thépaut, J. (2020). The ERA5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730), 1999–2049. 10.1002/qj.3803