Last modified 15 months ago Last modified on 2017-03-17 13:49:24

The general steps for taking processed data to publication on stellwagen:

Stellwagen is the group's location for distribution of published data. Our primary data produce is netCDF files, that follow the EPIC convention. They are later converted to CF-1.6 convention and served elsewhere. Data is allowed to be released on stellwagen as long as it has been through the process described in | OFR2007-1194. This page provides more details about how the Best Basic Version review and subsequent publication is accomplished.

  • get files for BBV approval from stg-proc in some_experiment_name/candidate_final_files
  • evaluate and fix issues found in files
  • save BBV approved files to some_experiment_name/BBV_reviewed
  • make html experiment and catalog pages:
    • make maps (currently use GMT) to go into graphics directory
  • make KML files for each experiment
    • data from BBV_reviewed is used to generate the html, maps, and kml, which all go on to the stellwagen staging area
  • get html experiment and catalog pages reviewed
  • move data files, html and kml files to stellwagen server (VPN required)
  • get dataset DOI for new experiment page
  • add citation info to the bottom of experiment page
  • add the new experiment to the index page to make "live"

I get the files to review from stg-proc/data/experiment_name/candidate_final_files, and return the reviewed files to stg-proc/data/experiment_name/BBV_reviewed.

The stuff the reviewer looks at in the review is a whole 'nother thing. Basically I look at "does the metadata match the mooring log?" so that the data will be located correctly in 3-d space and time, and "is the data reasonable, with bad data replaced by the appropriate FillValue??". I try to make overplots to check reasonableness, since if the wiggles look the same in ncBrowse, I don't always pick up on order of magnitude differences that sometimes occur. We can go over this in greater detail. I have some scripts that check, but a lot still requires human eyeballs to verify. I keep 2 BBV*.xls files to track the progress of which files have been checked in the review, and if any fixes are needed. Those files get uploaded to stg-proc/data/experiment as the process progresses.

Should you need to prepare the web pages and maps to go with a data release after the BBV is done, here's what to do.