Beginning A Project

Session Overview

  • Ingredients
  • Defining a project
  • Finding Data
  • Patterns of Analysis
  • Working in Teams

Ingredients for a successful project

  • Research Question!
  • Data
  • Analysis, AI/ML, etc.
  • Products

Thinking about Products:

  • Publication
  • Presentations
  • Model
  • Pipeline

Ingredients for a successful project

Cloud:

  • Scaling
  • Event Driven
  • Sharing

Machine Learning:

  • Data-Driven
  • Prediction
  • Method Change

Defining A Project

Research Question: What makes you curious, both in your Domain and about the material in this Class.


Think about:

  • Scope
  • Takeawayability*
  • What you want to learn individually
  • What your Team wants to learn collectively
  • Technical Term

Defining A Project

Begin discussing team project ideas, collectively, in your teams over the next few days.


  • Project themes will be firmed up beginning of Next Module
  • Projects will be defined (and scoped) by end of Next Module

Finding Data

Deliberately consider cloud based versus on-premise data

NOAA Open Data Dissemination

Other Sources

  • USGS, DOC, Census, USDA, NASA, data.gov, etc.

Patterns of Analysis

On-Premise

  • Download Data
  • Load into memory
  • Conduct Analysis
    • Single Threaded
    • Multi Threaded
    • Scheduling on HPC
  • Write new data/figure to disk

Cloud

  • Connect to data
  • Write/Map Analysis
  • Run Analysis
    • Lazy Loading
    • Natively and Massively Parallel
    • Portable Pipeline
  • New data/figure stays on cloud

Working in Teams

Communication with your team!


You are brainstorming

  • Idea Generation (Not Curation)
  • Yes, and…
  • Add constraints
  • Capture your Ideas

Earth System Data Science in the Cloud