DataPro and related utilities - Revision history

imported>Bob at 18:49, 1 June 2016

2016-06-01T18:49:53Z

137.229.92.230: /* Where to find it */

2015-11-06T21:00:35Z

Where to find it

137.229.92.230 at 20:59, 6 November 2015

2015-11-06T20:59:40Z

137.229.92.230 at 20:57, 6 November 2015

2015-11-06T20:57:32Z

137.229.29.173: Created page with " ===Background=== This program and related utilities started out in 2006 or 2007 as a way to leverage the gp.py program for web page creation. Basically, as a way to allow we..."

2015-11-04T02:50:55Z

Created page with " ===Background=== This program and related utilities started out in 2006 or 2007 as a way to leverage the gp.py program for web page creation. Basically, as a way to allow we..."

New page

===Background===
This program and related utilities started out in 2006 or 2007 as a way to leverage the gp.py program for web page creation. Basically, as a way to allow web pages generated by that tool to display data coming from Array styled outputting data loggers (Campbell Scientific cr10x & cr23x etc). Along the way it turned into a more full featured data processing system written in python with a host of support utilities. Bob Busey wrote the original and then student (now staff member) Ross Spicer did a nice rewrite and organized it like a proper computer scientist might.<br.
===Where to find it===
The current release can be found on Ross' github page:<br>
https://github.com/rwspicer/csv_utilities
(example starter files here:)<b>
https://github.com/frankohanlon/DataPro_Sample_Files

There are several ways to get it. Easy way is just to download the package as a zip file. Proper way is to use open source tool Git (from command line or graphical interface):<br>
<pre>git clone https://github.com/frankohanlon/DataPro_Sample_Files</pre>

Also useful if you decide to use it is this is a windows / linux graphical utility which helps generate the configuration files:<br>
https://github.com/rwspicer/sc_apps

===What is included===
<pre>
datapro.py -- Datapro 3.0 the data processing utility
step_function_utility.py -- applies a correction to data values over given
time periods
rh_calculator.py -- given data on precipitation and dew point caclualtes the
relative humidity
precip_utility.py -- allows for removal of bad pecipatation data
plotter.py -- used to plot csv files
PNGen.sh -- generates the plots for each csv file in a directory
get_ip -- a utility to get the ip adderess of a machine
rad_decoder.py -- takes a cdf File and converts to csv file
tz_shift.py -- for shifting between utc - 0 and akst timezones
wunder_formatter.py -- takes csv file data and converts it to a url that can
be sent to weather under ground
LoggerLink -- a porgram for uploading/ downloading data and programs from
cr1000 data loffers
noaa_data.py -- utility for retriving data form the noaa website

barrow_monthly.py -- a utility for getting the monthly data from a site in
barrow
tutorial.py -- the utility that goes with the tutorial
</pre>

@@ Line 1: / Line 1: @@
 ===Background===
-This program and related utilities started out in 2006 or 2007 as a way to leverage the gp.py program for web page creation.  Basically, as a way to allow web pages generated by that tool to display data coming from Array styled outputting data loggers (Campbell Scientific cr10x & cr23x etc).  Along the way it turned into a more full featured data processing system written in python with a host of support utilities.  Bob Busey wrote the original and then student (now staff member) Ross Spicer did a nice rewrite and organized it like a proper computer scientist might.<br.
+This program and related utilities started out in 2006 or 2007 as a way to leverage the gp.py program for web page creation.  Basically, as a way to allow web pages generated by that tool to display data coming from Array styled outputting data loggers (Campbell Scientific cr10x & cr23x etc).  Along the way it turned into a more full featured data processing system written in python with a host of support utilities.  Bob Busey wrote the original and then student (now staff member) Ross Spicer did a nice rewrite and organized it like a proper computer scientist might.<br>
 ===Where to find it===
 The current release can be found on Ross' github page:<br>
@@ Line 44: / Line 44: @@
 </pre>

@@ Line 5: / Line 5: @@
 The current release can be found on Ross' github page:<br>
 https://github.com/rwspicer/csv_utilities
-(example starter files here:)<b>
+(example starter files here:)<br>
 https://github.com/frankohanlon/DataPro_Sample_Files
-Finally, see a semi-real workflow for it here:
+Finally, see a semi-real workflow for it here:<br>
-[[DataPro setup and workflow]]
+'''[[DataPro setup and workflow]]'''
+<br>
 There are several ways to get it.  Easy way is just to download the package as a zip file.  Proper way is to use open source tool Git (from command line or graphical interface):<br>
 <pre>git clone https://github.com/frankohanlon/DataPro_Sample_Files</pre>

@@ Line 18: / Line 18: @@
 datapro.py -- Datapro 3.0 the data processing utility
 step_function_utility.py -- applies a correction to data values over given
-                             time periods
+                             time periods. Helps with applying temperature offsets to soil temperature data.
 rh_calculator.py -- given data on precipitation and dew point caclualtes the
                      relative humidity
-precip_utility.py -- allows for removal of bad pecipatation data
+precip_utility.py -- an additional level of qa that gets applied to the summer precipitation data
-plotter.py -- used to plot csv files
+plotter.py -- a plotting utility for making png charts
 PNGen.sh -- generates the plots for each csv file in a directory
 get_ip -- a utility to get the ip adderess of a machine
 rad_decoder.py -- takes a cdf File and converts to csv file
 tz_shift.py -- for shifting between utc - 0 and akst timezones
 wunder_formatter.py -- takes csv file data and converts it to a url that can
@@ Line 31: / Line 32: @@
 LoggerLink -- a porgram for uploading/ downloading data and programs from
                  cr1000 data loffers
-noaa_data.py -- utility for retriving data form the noaa website
+tutorial.py -- should show how to use all of these things.
 </pre>