CDIP THREDDS

The CDIP THREDDS server provides access to the wave and temperature datasets collected by each CDIP buoy. The THREDDS NetCDF format is an easily-accessible, powerful tool to bring buoy data directly into a programming language (e.g. MATLAB, Python) for plotting and manipulation. If you are interested in working with CDIP data via one of these programs, see below for further instruction about data access.

To view or download data as a table (e.g. to work with in Excel or a text-editor), you can visit a CDIP station’s Parameter Download page (available as a left-hand menu link on the station’s Data Access page, e.g. https://cdip.ucsd.edu/themes/?d2=p70&u2=s:191:st:1:v:pm_download_table) and click on the desired month-year combination.

THREDDS Data Access Instructions

CDIP THREDDS data are organized into Archive and Realtime folders:

• Archive - contains individual folders for all CDIP stations, both active and decommissioned. Each station’s individual Archive folder contains NetCDF files for each separate deployment (e.g. ‘d17.nc’) and an aggregate file (‘historic.nc’) of the full time-span of data for a buoy.
• Realtime - contains single NetCDF files (‘rt.nc’) for CDIP stations that are currently active and transmitting data.

• NetCDF files for Archived and Realtime data contain identical buoy parameters and variables, with the exception that the ‘historic.nc’ Archived file and the ‘rt.nc’ Realtime file do not contain Directional Displacement (xyz) data.

• Click here for a list of variables available for each CDIP buoy. The time period of available data varies by station.

The CDIP website offers graphical displays of buoy data at user-specified time periods. Click here for example plots for visualizing data, with corresponding links to the appropriate CDIP plots, and sample Python-based code that will reproduce the plots.

DATA ACCESS OPTIONS

1. OPENDAP - provides URL that can be used in Python/Matlab to automatically grab NetCDF file of data from server. Also provides option to download user-specified variables/timeperiods as ASCII or Binary file.
2. HTTPServer - option to download the whole NetCDF file.
3. NCML (NetCDF Markup Language) - XML document used to define a CDM dataset, and to allow user to add/delete/change metadata and variables, or combine data from multiple CDM files.
4. ISO - XML metadata record for each station.
5. UDDC (Unidata Data Discovery Convention) - tool to determine how well file metadata conforms to list of recommended metadata attributes.
6. SOS - web service interface which allows querying observations, sensor metadata, and representations of observed features. Defines means to register/remove sensors and insert new sensor observations.

CDIP Plot Examples

Caveats and Disclaimers

The following code is meant for illustrative purposes only. Most of the examples will only work for Datawell mark 3 buoys. For mark 4 buoys, make sure you use the data type time parameter that corresponds to the variable. For example, waveTime for Hs, Tp, Dp and  sstTime for seaSurfaceTemperature. Also, plots using real-time data files (e.g. *_rt.nc) may include bad records. To filter out bad records use this python code. Finally, these examples were developed using python and matlab libraries that are several years old. They may not work with the latest library versions.

Descriptions of wave- and temperature-related plots that CDIP generates, and Python-based examples of how to reproduce these plots.

1. Wave Height Boxplot

1. Description: A box is plotted for each month of the year that has data. The top and bottom of the black box mark the lower quartile (25% of data values are less than this value) and upper quartile (75% of data values are less than this value), a red line indicates the median, and a green line and green numeric value indicate the mean. The black lines above and below the box (the "whiskers") show the extent of the remaining data within a maximum range of 1.5L, where L is the length in meters from the lower quartile to the upper quartile. Data values that extend beyond the whiskers (the "outliers") are indicated by red crosses.
2. CDIP example plot: https://cdip.ucsd.edu/themes/?d2=p70&u2=s:093:st:1:v:hs_box_plot:dt:2015
3. Python-based example code: https://cdip.ucsd.edu/themes/media/docs/documents/html_pages/annualHs_plot.html

2. Compendium

1. Description: Plots of significant wave height (Hs), peak period (Tp) and peak direction (Dp) for a single station for a given month on 3 separate plots. The format of this plot is consistent with the original USACE wave parameters plot known as the wave compendium plot. Plotting all 3 parameters simultaneously gives a concise display of the general wave environment for the month.
2. CDIP example plot: https://cdip.ucsd.edu/themes/?d2=p70&u2=s:093:st:1:v:compendium:dt:201501
3. Python-based example code: https://cdip.ucsd.edu/themes/media/docs/documents/html_pages/compendium.html
4. MATLAB-based example code: https://cdip.ucsd.edu/themes/media/docs/documents/html_pages/compendium_matlab.txt

3. Monthly Temperature (Sea Surface Temperature)

1. Description: The sea temperature plot shows SST data (measured by Datawell Waverider thermistors at the sea surface, near the mooring eye of the buoy) over the entire month. Buoys return one SST value each half hour.
2. CDIP example plot: https://cdip.ucsd.edu/themes/?d2=p70&u2=s:093:st:1:v:temperature:dt:201501
3. Python-based example code: https://cdip.ucsd.edu/themes/media/docs/documents/html_pages/temperature.html
4. MATLAB-based example code: https://cdip.ucsd.edu/themes/media/docs/documents/html_pages/temperature_matlab.txt

4. Polar Spectrum

1. Description: The standard CDIP spectral file is converted into a 2-dimensional frequency/direction spectrum and displayed on a polar coordinate system. The Maximum Entropy Method (MEM) is used to create the 2D spectrum; this gives a narrow directional spectrum that CDIP considers appropriate for swell in the Pacific. The polar plot is useful for identifying individual wave events based on their directions and wave periods.
2. CDIP example plot: https://cdip.ucsd.edu/themes/?d2=p70&u2=s:093:st:1:v:polar_plot:dt:201501
3. Python-based example code: https://cdip.ucsd.edu/themes/media/docs/documents/html_pages/polar.html

5. Spectrum

1. Description: A single spectral wave record contains energy density and mean wave direction for each frequency bin (click here for an explanation of a CDIP spectral file). The top frame plots energy density vs. frequency, and the second plots mean wave direction vs. frequency. Note that the x-axis includes a scale for both frequency in Hz (bottom) and period in seconds (top).
2. CDIP example plot: https://cdip.ucsd.edu/themes/?d2=p70&pb=1&u2=s:100:st:1:v:spectral_plot:dt:202112222300^UTC
3. Python-based example code: https://cdip.ucsd.edu/themes/media/docs/documents/html_pages/spectrum_plot.html
4. MATLAB-based example code: https://cdip.ucsd.edu/themes/media/docs/documents/html_pages/spectrum_plot_matlab.txt

6. Timeseries (xyz direction)

1. Description: Plots the x- (North-South), y- (East-West) and z- (vertical) displacements as calculated by a Datawell Waverider. These timeseries are unedited and may contain spikes and other anomalies. Note: data for xyz directional displacements are only available in individual deployment NetCDF files for each station (not the aggregate ‘historic’ .nc file).
2. CDIP example plot: https://cdip.ucsd.edu/themes/cdip?d2=p70&pb=1&u2=s:093:st:1:v:xz_plot:dt:201501310627^UTC
3. Python-based example code: https://cdip.ucsd.edu/themes/media/docs/documents/html_pages/dw_timeseries.html

7. Wave Histogram

1. Description: Plots a histogram of the heights of individual waves derived from the vertical (z) displacement timeseries. Individual waves are identified using the crest-to-trough method, i.e. by calculating the crest-to-trough difference (max-min z-displacement) between consecutive zero up-crossings.

1. For buoys operated by the Navy (e.g. 067, San Nicolas Island), the user can specify a height threshold and the plot will show the percentage of waves that exceed the given threshold (default threshold is 4 m/13 ft).

2. CDIP example plot: https://cdip.ucsd.edu/themes/cdip?pb=1&bl=cdip?pb=1&d2=p1&d2=p70&u2=s:093:st:1:v:wave_histogram:dt:201501310627^UTC

8. Period Rose

1. Description: The data of the selected time period are categorized by peak wave direction and significant wave height. The length of a petal (or a portion of the petal), corresponds to the frequency of occurrence.
2. CDIP example plot - Period Rose:  https://cdip.ucsd.edu/themes/cdip?d2=p70&pb=1&u2=s:093:st:1:v:period_rose:dt:201501310627^UTC

9. Wave Rose

1. Description: The data of the selected time period are categorized by peak wave direction and significant wave height. The length of a petal (or a portion of the petal), corresponds to the frequency of occurrence.
2. CDIP example plot - Wave Rose: https://cdip.ucsd.edu/themes/cdip?d2=p70&pb=1&u2=s:093:st:1:v:wave_rose:dt:201501310627^UTC

BUOY VARIABLES

• sourceFilename - Source File Name (unitless). 2D variable based on: (sourceCount, maxStrlen64).
• waveTime - UTC-based Time variable (UNIX - seconds since 1970-01-01 00:00:00 UTC). 1D variable based on: (waveTime). Spectral processing performed on 1600-second samples specified by waveTimeBounds. Range: start time varies by buoy.
• waveTimeBounds - Time Cell Bounds (UNIX - seconds since 1970-01-01 00:00:00 UTC). 2D variable based on: (waveTime, metaBoundsCount). Range: start time varies by buoy.
• waveFlagPrimary - Primary wave QC flag (unitless). 1D variable based on: (waveTime). Flag values: [1, 2, 3, 4, 9]. Flag_meanings: 1)good, 2) not_evaluated, 3) questionable, 4) bad, 9) missing.
• waveFlagSecondary - Secondary wave QC flag (unitless). 1D variable based on: (waveTime). Range: 0-12. Flag values: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]. Flag_meanings: 0) unspecified sensor_issues, 1) Hs_out_of_range, 2) Tp_out_of_range, 3) Ta_out_of_range, 4) elevated_check_factors, 5) Hs_spike, 6) Ta_spike, 7) low_freq_energy_spike, 8) excessive_low_freq_energy,  9) hf_transmission_errors_fixed, 10) hf_transmission_errors_present, 11) directional_coeffs_out_of_range.
• waveHs - Significant Wave Height (meters). 1D variable based on: (waveTime). Range: 0-20.
• waveTp - Peak Wave Period (seconds). 1D variable based on: (waveTime). Range: 0-40.
• waveDp - Peak Wave Direction (degrees True). 1D variable based on: (waveTime). Includes compensation via (magnetic declination - INT(metaDeclination) to shift to degrees true. Range: 0-360.
• waveTa - Average Wave Period (seconds). 1D visualization based on: (waveTime). Calculated from full-spectrum, buoy-returned energy density values. Range: 0-40.
• wavePeakPSD - Peak Wave Power Spectral Density (m*m/Hz). 1D variable based on (waveTime). Range: 0-5000.
• waveTz - Mean Zero-Upcross Wave Period (seconds). 1D variable based on (waveTime). Range: 0-40.
• waveSourceIndex - Source File Index (unitless). 1D variable based on: (waveTime).
• waveFrequency - Band Center Frequency (Hz). 1D variable based on (waveFrequency). Range: 64 standard bands.
• waveFrequencyBounds - Frequency Cell Bounds (Hz). 2D variable based on (waveFrequency, metaBoundsCount). Range: 64 standard bands.
• waveFrequencyFlagPrimary - Primary waveFrequency QC Flag (unitless). 1D variable based on (waveFrequency). Flag values: [1, 2, 3, 4, 9]. Flag_meanings: good, not_evaluated, questionable, bad, missing.
• waveFrequencyFlagSecondary - Primary waveFrequency QC Flag (unitless). 1D variable based on (waveFrequency). Range: 0-1. Flag values: [0, 1].
• waveBandwidth - Bandwidth (Hz). 1D variable based on (waveFrequncy). Width of spectral bands for wave energy.
• sstTime  - UTC-based Time (UNIX - seconds since 1970-01-01 00:00:00 UTC). 1D variable based on: (sstTime). Range: start time varies by buoy.
• sstTimeBounds - Time Cell Bounds (UNIX - seconds since 1970-01-01 00:00:00 UTC). 2D variable based on: (sstTime, metaBoundsCount). Range: start time varies by buoy.
• sstFlagPrimary - Primary sst QC Flag (unitless). 1D variable based on (sstTime). Flag values: [1, 2, 3, 4, 9]. Flag_meanings: good, not_evaluated, questionable, bad, missing.
• sstFlagSecondary - Secondary sst QC Flag (unitless). 1D variable based on (sstTime). Range: 0-6. Flag values: [0, 1, 2, 3, 4, 5, 6]. Flag_meanings: unspecified sensor_issues SST_out_of_range SST_spike SST_max_change_exceeded hf_transmission_errors_fixed hf_transmission_errors_present.
• sstSeaSurfaceTemperature - Sea Surface Temperature (Celsius). 1D variable based on: (sstTime). Range: -5.0 - 46.15.
• sstSourceIndex - Source File Index (unitless). 1D variable based on (sstTime).
• sstReferenceTemp - Reference Temperature (Celsius). 1D variable based on: (sstTime).
• gpsTime  - UTC-based Time (UNIX - seconds since 1970-01-01 00:00:00 UTC). 1D variable based on: (gpsTime). Range: start time varies by buoy.
• gpsTimeBounds - Time Cell Bounds (UNIX - seconds since 1970-01-01 00:00:00 UTC). 2D variable based on: (gpsTime, metaBoundsCount). Range: start time varies by buoy.
• gpsStatusFlags - GPS Data Flags (unitless). 1D variable based on: (gpsTime). Flag values: [1, 2, 4, 8]. Flag_meanings: module_ok new_fix figure_of_merit hf_transmission_error
• gpsLatitude  - Buoy Latitude (degrees North). 1D variable based on: (gpsTime). Range: -90.0 - 90.0.
• gpsLongitude - Buoy Longitude (degrees East). 1D variable based on: (gpsTime). Range: -180.0 - 180.0.
• gpsSourceIndex - Source File Index (unitless). 1D variable based on (gpsTime).
• dwrTime  - UTC-based Time (UNIX - seconds since 1970-01-01 00:00:00 UTC). 1D variable based on: (dwrTime). Range: start time varies by buoy.
• dwrTimeBounds - Time Cell Bounds (UNIX - seconds since 1970-01-01 00:00:00 UTC). 2D variable based on: (dwrTime, metaBoundsCount). Range: start time varies by buoy.
• dwrSourceIndex - Source File Index (unitless). 1D variable based on (dwrTime).
• dwrBatteryLevel - Battery Level (unitless). 1D variable based on: (dwrTime). Range: 0-7.
• dwrZAccelerometerOffset - Vertical accelerometer offset (m/s^2). 1D variable based on (dwrTime).
• dwrXAccelerometerOffset - x-axis accelerometer offset (m/s^2). 1D variable based on (dwrTime).
• dwrYAccelerometerOffset - y-axis accelerometer offset (m/s^2). 1D variable based on (dwrTime).
• dwrOrientation - Buoy Orientation (degrees). 1D variable based on: (dwrTime).
• dwrInclination - Magnetic Inclinataion (degrees). 1D variable based on: (dwrTime).
• dwrBatteryWeeksOfLife - Battery Weeks of Life (weeks). 1D variable based on: (dwrTime).
• metaDeployLatitude - Deployment Latitude (degrees North). 1D variable based on: (metaDeployCount).
• metaDeployLongitude - Deployment Longitude (degrees East). 1D variable based on: (metaDeployCount).
• metaWaterDepth - Water Depth (meters). 1D variable based on (metaDeployCount).
• metaDeclination - Magnetic Declination (degrees). 1D variable based on: (metaDeployCount).
• metaStationName - Station Name (unitless). 1D variable based on: (maxStrlen64).
• metaStationLatitude - Station Average Latitude (degrees North). 0D variable. Notes: Averaged over all records. Record-specific deployment positions can be obtained from metaDeployLatitude and metaDeployLongitude as indexed by waveSourceIndex, sstSourceIndex, etc.
• metaStationLongitude - Station Average Longitude (degrees East). 0D variable. Notes: Averaged over all records. Record-specific deployment positions can be obtained from metaDeployLatitude and metaDeployLongitude as indexed by waveSourceIndex, sstSourceIndex, etc.
• metaPlatform - Platform Attributes (unitless). 1D variable based on: (maxStrlen64). Additional information: make_model: Multiple sensors possibly used. Please refer to per-deployment datasets. Documentation URL: https://datawell.nl. Mooring: Datawell-compliant.
• metaInstrumentation - Instrumentation Attributes (unitless). 1D variable based on: (maxStrlen64). Additional information: make_model: Multiple sensors possibly used. Please refer to per-deployment datasets. Documentation URL: https://datawell.nl. Mooring: Datawell-compliant.
• waveEnergyDensity - Band Energy Density (m^2 second). 2D variable based on: (waveTime, waveFrequency). Min: 0.
• waveMeanDirection - Band Mean Direction (degrees True). 2D variable based on: (waveTime, waveFrequency). Range: 0-360. Notes: Average Wave Direction for each frequency band (64 total).
• waveA1Value - Band a1 Fourier coefficient (metaStationLatitude, metaStationLongitude). 2D variable based on: (waveTime, waveFrequency). Range: -1.0 - 1.0. Calculated relative to true north from buoy-returned mean dir, spread, m2, n2.
• waveB1Value - Band b1 Fourier coefficient (metaStationLatitude, metaStationLongitude). 2D variable based on: (waveTime, waveFrequency). Range: -1.0 - 1.0. Calculated relative to true north from buoy-returned mean dir, spread, m2, n2.
• waveA2Value - Band a2 Fourier coefficient (metaStationLatitude, metaStationLongitude). 2D variable based on: (waveTime, waveFrequency). Range: -1.0 - 1.0. Calculated relative to true north from buoy-returned mean dir, spread, m2, n2.
• waveB2Value - Band b2 Fourier coefficient (metaStationLatitude, metaStationLongitude). 2D variable based on: (waveTime, waveFrequency). Range: -1.0 - 1.0. Calculated relative to true north from buoy-returned mean dir, spread, m2, n2.
• waveCheckFactor - Band Check Factor (inverse of wave ellipticity). 2D variable based on: (waveTime, waveFrequency). Range: 0-2.55.
• waveSpread - Band Directional Spread (radians). 2D variable based on: (waveTime, waveFrequency). Range: 0-90.
• waveM2Value - Band-centered sine Fourier coefficient. 2D variable based on: (waveTime, waveFrequency). Range: -1.0 - 1.0.
• waveN2Value - Band-centered cosine Fourier coefficient. 2D variable based on: (waveTime, waveFrequency). Range: -1.0 - 1.0.

How to select the good records

The following python code will select all the good records (FlagPrimary=1) for variable names beginning with “wave”. The same can be done for other variable names beginning with “sst”, “acm” and “cat4” using sstFlagPrimary, acmFlagPrimary and cat4FlagPrimary respectively.

good_waves = ( nc.variables['waveFlagPrimary'][:] == 1 )

Method 1: Produce an array of Hs records whose length equals the number of good records.

good_times = nc.variables['waveTime'][good_waves]

good_hs = nc.variables['waveHs'][good_waves]

Method 2: Set the mask on the original array of Hs. (mask=False indicates a good record).

hs = nc.variables['waveHs'][:]

hs.mask = ~good_waves

This method is appropriate for statistical work when you need to account for missing values.