GrADS:

A Handy Tool for Data Access, Analysis, and Visualization

Jennifer M. Adams and Brian Doty

IGES/COLA

What is GrADS?
GrADS is an interactive tool that integrates data access, analysis, and visualization
it handles gridded and station data
it has a programmable interface for scripting

it runs on Unix, MS Windows, and Apple Mac

it has a busy users forum with 1800+ subscribers

it has been under active development since 1989

and it is open source software under GPL

What Types of Data Can GrADS Read?

GrADS handles the following data formats: Flat Binary NetCDF GRIB (versions 1 and 2) HDF (versions 4 and 5) BUFR (for station data)

Data Formats and Metadata

Data are arrays of numbers
Metadata provides information about the data so we know what the numbers mean Some data formats are self-describing: data and metadata are contained in the same file Accurate metadata is absolutely essential

Metadata and GrADS

Metadata is communicated to GrADS with a separate Data Descriptor File (a.k.a. Control File)
Gives the user more flexibility and control

Aggregates numerous data files into a single data set

Supplements or overrides the metadata in the data file

Descriptor file may not be necessary if the metadata in data file is good enough

GrADS Gridded Data Model

Internal grid structure supports 5 dimensions:
X Y Z T E ~ ~ ~ ~ ~ Longitude Latitude Level Time Ensemble

T and E axes must be linear X, Y, and Z axes may be non-linear Grid coordinate values must be 1-dimensional

GrADS Station Data Model

Used for in situ observational data

Individual reports may be located anywhere in space and time

Required metadata for each report:
Station Identifier (8-character string) Longitude Latitude Level (for level-dependent variables) Time

Linear time axis is used to organize reports

Visualization in GrADS
Display options include: Contour (lines or shaded) Grid fill (colors or numbers) Bar graph (with error bars) Scatter plot Winds: Vectors, Barbs, or Streamlines Output options include: Images Vector Graphics (e.g. Postscript) ASCII Binary or NetCDF file GIS-Compatible Formats

Data Analysis in GrADS

GrADS expressions are formulas that contain:
Operands (variables, functions, or constants) Operators (add, subtract, multiply, divide) Parentheses (to control the order of operation)

Expression handling is recursive, so an expression may be embedded within another. Some examples:
temp*9/5+32 aave((temp*9/5+32),global) hgt(lev=500)-hgt(lev=1000) ave(uwnd,lon=0,lon=360) sum(precip,t+0,t+24)

GOES VIS Image with Radar, 500mb Heights, and SLP

18z 4 June 2009

Another Example of GrADS Graphics Output

QuikSCAT Winds (HDF), Surface Obs (BUFR), and ETA Model SLP (GRIB)

00z 6 Dec 2003

Analysis of Surface METAR Observations with Radar

18z 29 May 2009

GrADS Output in Google Earth

Palmer Drought Severity Index July 2009

Ensemble Forecast Time Series

(Longitude, Latitude, and Level are fixed)

Forecast Time --->

Ensemble Forecast Grid

(Longitude, Latitude, and Level are fixed)

Ensemble Member

Forecast Time --->

Ten Ensemble Forecasts

(Longitude, Latitude, and Level are fixed)

Ensemble Member

Forecast Time --->

Ensemble Forecast Time Series

(Longitude, Latitude, and Level are fixed)

Forecast Time --->

Ensemble Mean, Ensemble Min/Max, +/- StdDev of Ensemble Mean

Forecast Time --->

What is OPeNDAP?
OPeNDAP (formerly known as DODS) is an Open-source Project for a Network Data Access Protocol OPeNDAP servers make local data files accessible from remote locations

OPeNDAP clients (such as GrADS) can access remote data files

Open a data set with http://server/filename instead of /disk/filename

OPeNDAP Server-Client Interaction

Data
Observational data Model-based analyses and forecasts

Server
Handles Metadata Requests

Returns Data Subsets

Evaluates Analysis Expressions

Analysis Result Cache

Internet

Client (Users Program)

Local or Remote Data Analysis Visualization Web-Based Interface

What is the GrADS Data Server?

The GrADS Data Server (GDS) is a type of OPeNDAP server it can serve any GrADS-readable data set it translates all data formats into NetCDF it offers subsetting and server-side analysis it has a user-friendly browser interface and it is running operationally at NCEP, NCDC, NASA, and many other locations

Why is Server-Side Analysis Important?

Reading data over the internet is slower than reading data from a local file

Some analysis tasks need a lot of data

Compute the monthly mean annual cycle from a 100-year climate model run

Analysis expressions are evaluated at the server, where the data reside on disk
Only the analysis result is delivered to the client Server-side analysis saves a lot of time when: (size of result) << (size of data operated on)

THORPEX Interactive Grand Global Ensemble

TIGGE is an international collaboration between operational centers and universities for the development of ensemble prediction
Ten participating centers provide their ensemble forecasts in near real time:
BoM (Australia) CMC (Canada) ECMWF (Europe) KMA (Korea) NCEP (USA) CMA (China) CPTEC (Brazil) JMA (Japan) MeteoFrance (France) UKMO (England)

TIGGE data portals are at ECMWF, NCAR, and CMA

TIGGE Data Behind GDS at NCAR

+ + + + + Data neatly handled by GrADS Boost to usage of TIGGE data Forecasts organized by date and by provider Time series of initialization analyses Server-side analysis capability

- 48-hour data embargo - Only a 3-week window of data is online - Intl agreement requires password protection

TIGGE Multi-Member Multi-Model Ensemble

500mb Geopotential Height valid August 30, 2008
7-day Lead 5-day Lead

3-day Lead

1-day Lead

Washington Post Weather Page

Extreme Makeover: Weather Graphics Edition

Step 1: Get the Daily Normals and Extremes

Check the web site of the local Weather Service Forecast Office for local climate data: http://www.weather.gov/climate/local_data.php?wfo=lwx
Grab ASCII files, which will require some manual editing:
DY NMX NMN NPCP NS RMX /YEAR LOMX/YEAR RMN /YEAR HIMN/YEAR MXPCP YEAR MXSN YEAR 01 43 29 0.10 0.1 69 /2005 17 /1918 -14*/1881 51 /1876 1.52 2003 4.5 1899 02 43 28 0.10 0.2 71 /1876 15 /1918 -1 /1899 52 /1876 1.44 1979 4.5 1925 03 43 28 0.10 0.2 68 /2004+ 10 /1879 -3 /1877 52 /2000 1.87 1914 2.4 1988 04 43 28 0.10 0.2 73 /1997 16 /1879 -3 /1877 60*/1950 2.28 1886 2.1 1980 05 43 28 0.10 0.2 71 /1997 18 /1896 -3 /1877 56 /1950 1.44 1949 4.8 1980

Convert ASCII to GrADS station data format (txt2stn.c)

Daily Normal and Extreme Temperatures Observed at Reagan National Airport

Step 2: Get Forecast Data for Next 10 Days

Browse through available TIGGE forecasts and select which initial date and model to use: http://vetscomm.ucar.edu:9090/dods/tigge/ http://vetscomm.ucar.edu:9090/dods/tigge
Run the GrADS script: ga-> get_fcst yyyymmddhh model The script creates a local data set (binary data and a descriptor file) containing daily min and max 2-meter temperatures for the first 10 days of the given forecast

Step 3: Get Observed Data for Past 10 Days

Recent hourly observations (past 15 days) are here: http://monsoondata.org:9090/dods/stn/metar/past360
Run the GrADS script (note date format is different):

ga-> get_past get_past ddMONyyyy

The script creates a local data set (binary data and a descriptor file) containing daily min and max surface temperatures observed over the past 10 days

Combine Climate, Forecast, and Observed Data in a New and Improved Graphic

Some URLs to Remember http://iges.org/grads

GrADS Home Page

http://iges.org/jma/AMS2010
This Presentation (plus all scripts and programs)

http://vetscomm.ucar.edu:9090
TIGGE GDS

Jennifer Miletta Adams jma@iges.org