Resources

Resources

This page provides a list of resources related to working with large and challenging climate datasets. Brief descriptions and links to the resources are provided.

Documentation

Name

Description

CLEX-CMS wiki

Lots of info/links from Computation Modelling Systems team at the Climate Extremes ARC Centre of Excellence

Pangeo Cloud

Experimental service providing cloud-based data science environments

Pangeo Data Catalog

List of all datasets available on Pangeo Cloud

Pangeo-Forge

Tool for extracting data from traditional repositories and adding them to the cloud

Intake-ESM

Data cataloging utility for Earth System Models (ESMs)

CMS parallel tutorial

Tips for xarray + dask analysis

CMS youtube

Recorded presentations on climate analysis / model runs

Xarray docs

Python package for working with labelled, multi-dimensional arrays

Dask docs

Distributed computing

Training

Name

Description

Type

CLEX Training

ARC Centre of Excellence computing training sessions

Videos, interactive notebooks, youtube channel

Research Software Engineering with Python

By Damien Irving et al

eBook

Project Pythia

An education and training hub for the geoscientific Python community

Website

PyAOS Training Resource Guide

Python for Atmosphere and Ocean Scientists - training materials

Website with links

An Introduction to Earth and Environmental Data Science

Course materials from Ryan Abernathey

Jupyter Book

Xarray tutorial

90-minute course introducing data scientists already familiar with Numpy to Xarray

Video

Intro to Dask

In Pangeo Gallery

Notebook (interactive via Binder or noninteractive on website)

Intake-ESM – Making It Easier To Consume Climate Data

By Anderson Banihirwe (NCAR)

Video screencast

Example workflows

Name

Description

Type

CLEX-CMS Blog

Blog posts (usually as non-interactive Jupyter notebooks)

Pangeo Gallery

Collections of notebooks with workflow examples; notebooks can be run interactively on Binder or viewed directly on the website

Website with links to notebooks

Other resources

Name

Description

Type

Tips for running Pangeo workflows on Australian HPC

Github repo with scripts and written advice