Welcome to Data Cache Management Service documentation!

Cache-B

Credentials and tools configuration

In order to access the datasets in CacheB cache you need to obtain CacheB credentials and instruct your tools to use them when downloading the data.

With your DestinE platform credentials at hand, download the cacheb-authentication.py script.

You first need a Python interpreter (>= 3.9) and the installation of some dependencies:

pip install requests conflator lxml

Then run the script:

python cacheb-authentication.py
Username: <Enter your DestinE username here>
Password: <Type your DestinE password here>

The output will be a configuration snippet which follows the netrc format.

It will be similar to this:

machine cacheb.dcms.destine.eu
    login anonymous
    password <…a-very-long-string…>

Copy the configuration in the .netrc file inside your home directory (~/.netrc on Linux/MacOS, on Windows the file must be named _netrc). Create the file if doesn’t exists yet.

Now you should be able to access CacheB datasets with Xarray / Zarr.

Note

CacheB credentials are valid for a limited period of time and need to be regenerated and reconfigured periodically by running the cacheb-authentication.py script.

Access the Data

After setting up your CacheB credentials, you can access CacheB datasets with Xarray / Zarr.

Here is a list of available datasets:

datasets = [
 "https://cacheb.dcms.destine.eu/d1-climate-dt/ScenarioMIP-SSP3-7.0-IFS-NEMO-0001-high-sfc-v0.zarr",
 "https://cacheb.dcms.destine.eu/d1-climate-dt/ScenarioMIP-SSP3-7.0-IFS-NEMO-0001-high-o2d-v0.zarr",
 "https://cacheb.dcms.destine.eu/d1-climate-dt/ScenarioMIP-SSP3-7.0-IFS-NEMO-0001-high-pl-v0.zarr",
 "https://cacheb.dcms.destine.eu/d1-climate-dt/ScenarioMIP-SSP3-7.0-ICON-0001-high-sfc-v0.zarr",
 "https://cacheb.dcms.destine.eu/era5/reanalysis-era5-land-no-antartica-v0.zarr",
 "https://cacheb.dcms.destine.eu/era5/reanalysis-era5-single-levels-v0.zarr",
]

Each dataset can be reached via its dedicated URL.

You can use the following code snippet to access the data:

# pip install xarray zarr fsspec requests aiohttp dask
import xarray as xr
data = xr.open_dataset(
    "https://cacheb.dcms.destine.eu/d1-climate-dt/ScenarioMIP-SSP3-7.0-IFS-NEMO-0001-high-sfc-v0.zarr",
    engine="zarr",
    storage_options={"client_kwargs": {"trust_env": "true"}},
    chunks={},
)

Quota limits

Request Limit: 500000 requests per user per month

Authentication: Required for all data access

Recommended Approach: we recommend carefully planning your data retrieval strategy, as downloading entire datasets can quickly consume your quota. Our tutorials provide guidance on how to efficiently access and work with large datasets while managing your request limits effectively.