Working Group Goals
Contents
Working Group Goals¶
Goal:¶
Create a best practices framework for climate-related scientists in Australia working with big/challenging datasets.
Definition of terms:¶
“Big/challenging data”: when the size and complexity of the dataset is such that “traditional data-processing application software” is inadequate to handle the analysis or management of the dataset (Wikipedia page for “Big data”; more specifics can also be found at Pangeo FAQ: https://pangeo.io/faq.html)
Steps to achieve our goal:¶
Consolidate a list of software, tools, learning/training modules, relevant documentation, and other resources that currently exist for dealing with, specifically for storing, accessing, and analyzing, big data.
Identify weak points/missing gaps in existing resources
Expand documentation, create learning modules, etc. to fill the identified gaps in currently available resources
Communicate with other working groups (e.g. on data management guidelines) on issues related to improving data organization
Working group full name:¶
“Working with big/challenging data collections”