Last modified 3 years ago Last modified on 2015-08-03 11:52:52

Data processing workflow

This describes the current way data is treated from Instrument to publishable form (Best Basic Version (BBV))

  • files are downloaded from the instrument
  • some or all the data from each instrument is processed by Marinna or Jon to assess instrument function. This is done primarily to discover whether the instrument worked, and if repair is needed prior to the next deployment. However, some problems that might be encountered in data processing may be identified.
  • data files are processed for real, paying attention to metadata correctness, removing pre-and post deployment transitions, and creating EPIC compliant files. Not necessarily all editing is accomplished in this step.
  • data is review by the processor, and hopefully looked at with regard to whether it makes sense in context, this is when the real scope of problems comes out as time series are compared across instrument and platforms.
  • once the data are thought to be OK, the BBV evaluation is done, and any issues found are addressed (usually all the files from one experiment are submitted together for approval)

After BBV approval, files may be put on Stellwagen for distribution, or included in an Open File Report or other publication. nb: BBV is for getting the parameters measured by each instrument or logger reviewed and published. Water depth adjustments don’t have to be done for the BBV stage, although it makes sense for them to be done as part of a integrated tripod-centric system that’s probably needed to do the analysis.