Overview
Why move to the cloud?
Remove technical barriers
Software
Compute
Data
Open new research opportunities
Massive Earth observation data
External resources on cloud computing for science
Beginner tutorials
Quick start guide for new users
Step 1: Sign up an Amazon Web Service (AWS) account
Step 2: Launch a server with GEOS-Chem pre-installed
Step 3: Log into the server and run GEOS-Chem
Step 4: Analyze output data with Python
Step 5: Shut down the server (Very important!!)
Overview of basic AWS compute and storage services
Core AWS concepts for scientific computing
Set up AWS Command Line Interface (AWS-CLI)
Install AWSCLI
Try to configure AWSCLI
Obtaining security credentials
With root account
With IAM user account
Finish AWSCLI configuration
Additional notes
Use S3 as major storage
Use S3 from the console
Working with S3 using AWSCLI
Access NASA-NEX data in S3 (Optional but recommended)
Access GEOS-Chem input data repository in S3
Use EBS volumes as temporary disk storage
Viewing existing EBS volumes
Choose volume size at launch time
Attach new volumes after EC2 launch (Optional)
Launch and attach a volume
Make that volume usable
Save volumes into snapshots (Optional)
Use Spot Instances to reduce EC2 cost
What are spot instances and why they exist
Use spot instances for big computing
Deal with spot instance interruptions
Notes on security groups (EC2 firewall)
Put everything together: a complete workflow
General comments on cloud vs local computer
A reference workflow
Launch EC2 instance with custom configuration
Set up your own model configuration
Get source code and checkout model version
Generate run directory
Tweak run-time configurations as needed
Get more input data from S3
Perform long-term simulation
Analyze output data
Save your files to S3
Terminate server, start over whenever needed
Advanced tutorials
Enable S3 access from EC2 by IAM role
What is Identity and Access Management (IAM)
Grant S3 permission to EC2
Create a new IAM role
Assign that role to EC2
More advanced usages of AWSCLI
Launch on-demand instances
Request spot instances
Other use cases
Use containers for platform-independent deployment
Containers and scientific reproducibility
The container landscape
Installing the container software
Install on your own machine
Testing pre-installed container on the cloud
Using GEOS-Chem container image
Using Docker
Using Singularity
Using CharlieCloud
Using custom GEOS-Chem source code
Using GCHP container image
Getting GEOS-Chem input data to your local machine
Overview of HPC cluster options on AWS
Why do you need an HPC cluster
HPC cluster management tools
AWS ParallelCluster
AlcesFlight
ElasticCluster
StarCluster
Developer guide
Install compilers and commonly-used libraries
Start with a fresh operating system
C, C++, and Fortran compilers
NetCDF library
Install NetCDF with package manager
Test sample NetCDF code
(Optional) Build NetCDF from source
MPI library
Install MPI with package manager
Test sample MPI code
(Optional) Build MPI from source
Install scientific Python environment
Additional tools
Save your system and share with others
Building an MPI cluster from scratch
Set up GEOS-Chem environment from scratch
Building GEOS-Chem AMI
Building GEOS-Chem Docker image
AWS concepts and services in detail
Common AWS concepts (but often confuse non-IT people)
Regions and Availability Zones
Virtual Private Clouds (VPCs) and Subnets
IAM users and IAM roles
Security groups
Monitoring AWS cost
Appendix
List of public AWS resources for GEOS-Chem
Simplify SSH login with config file
Set up SSH config file
Enable port forwarding
For multiple servers
Additional notes
Keep a program running after logoff
Use nohup command (not recommended)
Use GNU Screen
Use tmux (recommended)
Sample Python code to plot GEOS-Chem data
GEOS-Chem netCDF diagnostics
GEOS-FP met field
Sample Python code to analyze NASA-NEX data
GEOS-Chem on cloud
»
Appendix
Edit on GitHub
Appendix
¶
This chapter provides additional resources that don’t fit into the main tutorials.
List of public AWS resources for GEOS-Chem
Simplify SSH login with config file
Set up SSH config file
Enable port forwarding
For multiple servers
Additional notes
Keep a program running after logoff
Use nohup command (not recommended)
Use GNU Screen
Use tmux (recommended)
Sample Python code to plot GEOS-Chem data
GEOS-Chem netCDF diagnostics
GEOS-FP met field
Sample Python code to analyze NASA-NEX data
Read the Docs
v: latest
Versions
latest
stable
Downloads
pdf
html
epub
On Read the Docs
Project Home
Builds