IARC 207 Wiki - User contributions [en]

K1 Burn Notes

2016-12-16T20:55:34Z

69.161.4.229:

===Met Review===
* 1999 All good
** Precip needs comparing to another site as it doesn't seem to record until end of summer.
* 2000 All Good
* 2001 All Good
** No precip for this year though (in NSIDC data set)
* 2002
** Missing data 1/1/ to 2/4
** AT/RH bad one @ 6/10/2002 15:00
* 2003 All Good
* 2004 All Good
* 2005 All Good
* 2006 Mostly Good
** Bear damage of 3m WS/WD?
* 2007 All Good
* 2008 All Good
* 2009 Problems... need to follow up with data from other sites.
AT/RH 1m: 6/4/2009 0:00 - 7/31/2009 16:00 (Bear Damage)
AT/RH 3m: 6/4/2009 0:00 - 9/25/2009 16:00
WS/WD need to compare to other Kougarok sites. I don't trust specifically the anemometer being pointed north much of the time. Perhaps fixed on 9/25/2009 visit.
Precip -- Broken all year (tipping bucket removed by animal) but bucket replaced on 9/25/2009 trip.

===Soil Information:===

D:\projects_usb_drive\UAF\From Crane\archived files from past workers\ANDYS\ANDYWORK\SEWARD_PEN\KOUGAROK\K1_3MTOWERNIAGARA_\WORKING\2000K1.XLS
=IF(F3>60,6999,IF(F3<3,6999,IF(F3=0,6999,IF(F3=-6999,-6999,IF(F3=6999,6999,1/(0.00128+0.000237*(LN(F3*1000))+0.0000000906*((LN(F3*1000))^3))-273.15)))))
Thermistor String 1 -- 5, 10, 15, 20, 30, 40
Thermistor String 2 -- 25, 40, 55, 75, 100, 125
Soil Moisture Pit -- 5, 10, 15, 20, 30, 40

K1 Burn Notes

2016-12-15T23:25:58Z

69.161.4.229:

Datapro Parameter Options

2016-12-15T19:35:02Z

69.161.4.229: Created page with "===Date / Time Related=== <pre> datey -- designates the year column for array based data loggers dated -- designates the julian day column for array based data loggers dateh -..."

===Date / Time Related===
<pre>
datey -- designates the year column for array based data loggers
dated -- designates the julian day column for array based data loggers
dateh -- designates the time column for array based data loggers

tmstmpcol -- Identifies the ISO format date/time found in table based data loggers.
</pre>

===Numerical Functions===
<pre>
num -- assumes the value coming from the logger already has proper units etc and any data checking is of a qc nature.
net -- assumes the value coming from the logger already has proper units etc and any data checking is of a qc nature.
precip -- assumes the value coming from the logger already has proper units etc and any data checking is of a qc nature. (should augment this 12/2016)
therm -- assumes the value is a thermistor resistance with units of kilo-ohms.
coef_1 = Steinhart-Hart A value
coef_2 = Steinhart-Hart B value
coef_3 = Steinhart-Hart C value
coef_4 = Freezing Point Offset (degrees Celsius)
returns a value in degrees Celsius
thermF -- same options as therm but returns degrees Fahrenheit
therm1 -- similar to therm but rather than coefficients being specified in coef_1 - coef_3, a text file with a look up table of resistance vs coefficients is used.
therm2 -- same as therm1 but for a second thermistor type
therm3 -- same as therm1 but for a third thermistor type
poly -- applies an up to 6th order polynomial to the value from the data logger. Use coef_1 - coef_7 to set coefficient values.
rh -- identifies variable as a relative humidity sensor. Additional relative humidity checks are applied in QC. (values over 100%)
swrad -- identifies variable as a shortwave radiometer. Additional checks are applied (night time values below 0 W/m^2)
rt_sensor -- another type of radiation calibration.
coef_1 = divisor value
coef_2 = offset value
coef_3 = multiplier value
equation = self.result = ((self.variable / self.div) + self.offset) / self.mult
sm -- same as poly but then multiplies the result by 100 to convert to percent.

</pre>
===Functions that use two or more data elements in processing===
<pre>
mrctherm -- converts millivolt output to temperature in degrees Celsius for MRC type thermistor strings.
coef_3 = specify the data column with reference voltage here.
netrad -- converts millivolt output for Q7 Rebs Net Radiometer to W/m^2.
coef_1 = positive calibration coefficient
coef_2 = negative calibration coefficient
coef_3 = specify the data column with reference wind speed
</pre>

Software

2016-12-15T19:12:00Z

69.161.4.229: /* In-house software development */

Here's a list of software commonly used in the department. You may wish to add tips or questions to the program's wiki page.
New to programming? This may be a good resource (in addition to web searches of course): http://software-carpentry.org/

== Mac ==
* [[Dreamweaver]] -- HTML/CSS editing program
* [[Aptana]] -- HTML / CSS editing program with a different IDE from dreamweaver (some advantages for each)
* [[MS Office]] -- Excel, Powerpoint etc.
* [[fugu]] -- a nice GUI version of SFTP/SCP for the Mac

== Linux ==
* [[Loggernet]] -- Campbell Scientific's data retrieval program
* [[Perl]] -- Handy programming/scripting language for all sorts of things
* [[Ruby]] -- A '''very''' ''object-oriented'' language we're just beginning to use
* [[Python]] -- Another programming/scripting language
* [[TopoFlow V3]] -- Python version of TopoFlow
* [[Initial Debian Server Setup]]

== Windows ==
* [[Dreamweaver]] -- HTML/CSS editing program
* [[Aptana]] -- HTML / CSS editing program with a different IDE from dreamweaver (some advantages for each)
* [[MS Office]] -- Excel, Powerpoint etc.
* [[Adobe PDF tools]] -- Making PDFs (free alternatives also available)
* [[Adobe Photoshop]] -- Image Manipulation
* [[Adobe Illustrator]] -- Graphics creation
* [[Irfanview]] -- Image viewing program more flexible than the default Microsoft Image viewer
* [[Cygwin]] -- linux utilities for Windows
* [[Python]] -- available either through cygwin or as a standalone program.
* [[ArcGIS]] -- ESRI's GIS program - Visit the GINA page for the UAF license info: http://esrisitelicense.gina.alaska.edu/
* [[Putty]] -- Small suite of tools for doing SSH/SFTP/SCP
* [[SSH Secure Shell]] -- An alternative tool for doing SSH/SFTP
* [[Origin]] -- A fancy plotting & graph program
* [[Surfer]] -- A fancy plotting & graph program
* [[Perl]] -- Handy programming language for all sorts of things
* [[Loggernet]] -- Campbell Scientific's data retrieval program
* [[HBV]] -- Streamflow modeling, a rainfall-runoff program
* [[HBV-light]] -- Streamflow modeling, a monte carlo approach for parameter estimation
* [[RiverTools]] -- A program for getting for DEM and related files ready for input into TopoFlow
* [[TopoFlow]] -- Streamflow modeling, a physically based model originally written in IDL and currently being redone with python
* [[WinRiver II]] -- RDI discharge measurement software for use with RDI products
* [[Topcon ImageMaster Photo]] -- DEM Creation tool
==In-house software development==
* [[src-perl-Campbell::Comm-overview|data-gathering system]], written in Perl:
** [[src-perl-Campbell::Comm|Campbell::Comm]] -- perl module to interface Campbell CR10 loggers
** [[src-perl-querylogger|querylogger.pl]] -- query remote loggers
** [[src-perl-loggerdata|loggerdata.pl]] -- automatic downloads from remote loggers
** [[WERC datasite architecture]] -- setup & conventions assumed by the above
* [[src-datasite-commserver|commserver]] -- provide connections to radio telemetry sites
* [[src-perl-lab-scale|laboratory scale logger]]
* [[src-datasite|datasite -- object-oriented site and data management]]
* [[Excel Macros]] -- Handy Excel Macros useful for data processing etc.
* [[GP]] -- GWS/EEI developed html page generating program from DOT/DNR project
* [[Graphing]] -- GWS/EEI developed graph generating program from DOT/DNR project
* [[DataPro and related utilities]] -- Data processing engine useful
** [[DataPro setup and workflow]] (related)
** [[Datapro Parameter Options]] -- Definitions for the variety of data types for processing
** [[QC.py]] -- utility for merging manual corrections back into the datapro data record.
* [[Site Creator]] -- program to set up dirctories and key files for datapro processing

==Other Specialized software==
* [[MicroMet]] -- Suite of programs including SnowModel, SnowTran, and some data infilling / assimilating utilities.
* [[TopoFlow]] -- A physically based hydrologic model.
* [[Git Tips]] -- Bryan Johnson Write up

K2 Met Data Downloads

2016-12-15T08:03:01Z

69.161.4.229:

K2 Met Data Data Start - Data Stop Array IDs Present
050599K2.DAT 1999,97,1700 - 1999,125,1700 128,111
051099K2.DAT 1999,97,1700 - 1999,130,1100 128,129
051899K2.DAT 1999,97,1700 - 1999,138,1600 129
052299K2.DAT 1999,97,1700 - 1999,142,1300 129
052499K2.DAT 1999,97,1700 - 1999,144,1400 128,129
K2072199.DAT 1999,97,1700 - 1999,202,1200 128.129,130
K2.DAT 1999,202,1500 - 1999,203,2300 129,226,230
K20CT99.DAT 1999,202,1500 - 1999,278,1200 129,226,230
K2JAN00.DAT 1999,279,1100 - 2000,28,1300 129,263
K2MAY00.DAT 2000,28,1400 - 2000,126,1800 129,263
K2JUN00.DAT 2000,126,1900 - 2000,154,1700 129,267,270
K2JULY2000.DAT 2000,126,1900 - 2000,206,1300 129,267,270,263
K10M2801.DAT 2000,168,100 - 2000,302,1100 129,263
K210M001.DAT 2000,302,1200 - 2001,30,1800 129.263
K20401.DAT 2001,8,100 - 2001,95,900 129,263
K1001.DAT 2001,95,1100 - 2001,122,1600 129,263
K10--2001-06-03.DAT 2001,95,1100 - 2001,154,1400 129,263
K10--2001-07-13.DAT 2001,154,1500 - 2001,194,1000 129,263
K10M--2001-08-21.DAT 2001,95,1100 - 2001,232,2200 129,130,263
K10M--2001-10-11.DAT 2001,95,1100 - 2001,283,1900 129,130,263
K10MOCT01.DAT 2001,95,1100 - 2001,283,1900 129,130,263
KOU10M.DAT 2001,95,1100 - 2001,165,900 129,263
K10MJAN02-2002-02-03.DAT 2001,95,1200 - 2002,22,1600 129,130,263
K10M-2002-04-19.DAT 2001,95,1200 - 2002,22,1600 129,130,263
K10STO01.DAT 2001,347,2000 - 2002,22,1600 130,263 (PLUS REPEATED DATA FROM 2001)
K10STO02.DAT JUST: 2002,34,2100 130,263
K10MSPRING2002.DAT 2002,35,1400 - 2002,132,1300 130,263
K10MJAN02--2002-08-29.DAT 2001,95,1200 - 2002,22,1500 129,130,263
K-MET_FEB_2003.DAT 2002,167,1000 - 2003,58,1600 130,127,263
K10M--2003-10-30.DAT 2003,251,1200 - 2003,282,1700 128,241
K2_2005-06-03a1.dat 2002,285,1800 - 2005,154,1800 127,128,229,241
K2_2005-07-27a1.dat 2005,154,1900 - 2005,208,1500 128,241
SP_K2.DAT 2005,218,2000 - 2006,153,1900 128,241
K2_final_storage_1.dat 2002,297,1200 - 2006,229,1000 128,241
K2_2006-10-07.dat 2006,229,1100 - 2006,281,1100 128,241
K2_2007-06-11.dat 2006,281,1200 - 2007,162,1000 128,241
K2_2007-07-25.dat 2007,162,1200 - 2007,206,800 128,249
K2_2007-09-15.dat 2007,206,900 - 2007,258,1800 128,265
K2_2008-06-14.dat 2007,258,1900 - 2008,166,2000 M128-30, SL269-35, SN280-7
K2_2008-08-28.dat 2008,166,2100 - 2008,241,1000 M128-30, SL272-35, SN280-7, SN280-5
k2-met_2009_07_30.dat 2008,241,1100 - 2009,211,1800 M128-30, SL273-33, SN280-5
k2-met_2009_09_25.dat 2009,211,1900 - 2009,268,1700 M129-30, SL273-33, SN285-4, TC144-27
K2_2010-06-24.dat 2009,211,1900 - 2010,175,1100 M129-30, SL273-33, SN285-4, TC144-27
k2_2010_08_12.dat 2009,268,1800 - 2010,224,1300 M129-30, SL273-33, SN285-4, TC144-27
k2_2011_08_19.dat 2010,224,1500 - 2011,231,1200 M128-30, SL271-34, SN284-4
k2_2012_08_08.dat 2011,231,1300 - 2012,221,1100 M132-30, SL271-26, SN283-4
k2_2014_07_23.dat 2012,296,1200 - 2014,204,1100 M132-30, SL271-26, SN283-4 (I think the missing data between 2012 and 2014 can be found on a storage module)
k2_2015_07_27.dat 2014,204,1200 - 2015,209,1200 132-30 ,271-26, 283-4
k2_2016_09_07.dat 2015,209,1300 - 2016,257,1300 132-30, 271-26, 283-4
Partial List of Array IDs:
128 -- met, 30 elements
129 -- met, 30 elements
132 -- met, 30 elements

271 -- soil, 34 elements (k2_2011_08_19.dat)
271 -- soil, 26 elements
273 -- soil, 33 elements

144 -- snow temperature profile, 27 elements

280 -- snow, 5 elements
283 -- snow, 4 elements
285 -- snow, 4 elements

K1 Burn Data Downloads

2016-12-14T21:35:49Z

69.161.4.229:

See archived Key Files here (mapping array IDs to data point):
http://ngeedata.iarc.uaf.edu/data/sp_datapro/k-burn/config/

K1 Met Data Data Start - Data Stop Array IDs Present & Array Elements
050599K1.dat 1999,97,1700 - 1999,125,1700 128-19,111-14
051099K1.dat 1999,125,1900 - 1999,130,1300 112-15
052399K1.dat 1999,125,1900 - 1999,143,1400 112-15
K17219.dat 1999,125,1900 - 1999,202,1800 112-15
K1OCT99.dat 1999,202,2200 - 1999,278,1800 112-15,220-22
K1Jann00.dat 1999,278,1900 - 2000,28,1600 112-15,220-22
K1MAY00.dat 2000,28,1700 - 2000,126,2200 112-15,220-22
K1JUN00.dat 2000,126,2300 - 2000,155,1300 112-15,220-22
K1JULY2000.dat 2000,126,2300 - 2000,206,1200 112-15,220-22
BURN.DAT 2002,24,1600 - 2002,29,1700 112-15,112-16,113-15,220-22
2001,95,2200 - 2001,260,800
2002,24,1600 - 2002,29,1700
2001,95,2200 - 2001,364,1900
K1_final_storage_1.dat 2002,35,1800 - 2006,229,1300 113,123,124,220 <--------Best record from logger (2005/09 file lost in HD crash, )
K1_2005-06-04a1.dat 2002,35,1800 - 2005,155,2000 113,123,124,220
k1_2005-07-27a1.dat 2005,155,2100 - 2005,208,1300 124,220
K1_206-10-07.dat 2006,229,1400 - 2006,281,1900 124,220
K1_2007-06-11.dat 2006,281,2000 - 2007,162,1600 124,220
K1_2007-07-22.dat 2007,162,1700 - 2007,203,1800 124,220
K1_2007-09-16.dat 2007,203,1900 - 2007,259,1300 124,220
K1_2008-06-14.dat 2007,259,1400 - 2008,166,1600 131,220
K1_2008-08-27.dat 2008,166,1645 - 2008,240,1720 SNOW = 142-5,142-8 MET = 133-18,SOIL = 220-23
k1-burn_2009_07_31.dat 2008,166,1800 - 2009,212,1800 SNOW = 142-5 143-5 133-18,220, ,134,137,230,235 (thermocouples added?)
137-18 = MET
220-23
142-5
143-5 = SNOW
134-18 = MET
137-18
235-28 = THERMOCOUPLES
230-23
k1-burn_2009_09_25.dat 2009,212,1900 - 2009,268,1300 137,235,230,148
K1_2010-06-23.dat 2009,212,1700 - 2010,174,1800 137,230,235,148
k1_2010_08_12.dat 2009,268,1400 - 2010,224,1000 137,230,235,148
K1_2011_08_18.dat 2010,224,1100 - 2011,230,1600 137,235,230,148
K1_2012_08_08.dat 2011,230,1700 - 2012,227,1700 141,220,152
K1_2014_07_23.dat 2012,221,1800 - 2014,204,1400 141,220,152
k1_2015_07_28.dat 2014,204,1500 - 2015,209,1400 141,220,152

Missing Data:
2000 206 1300 to 2001 95 2100
2001 364 1900 to 2002 24 1600
2002 29 1800 to 2002 35 1700

Probably Legit Gaps:
1999 125 1800
1999 202 1900 to 1999 202 2100

KBurn.dat 2001,95,1300 - 2001,232,2100 112,220

Incomplete Array ID Identification
134 = Met, 10 elements
141 = Met, 10 elements
142 = Snow, 8 elements
143 = Snow, 5 elements
152 = Snow, 4 elements
220 - Soil, 23 elements

K1 Burn Data Downloads

2016-12-06T21:32:33Z

69.161.4.229:

See archived Key Files here (mapping array IDs to data point):
http://ngeedata.iarc.uaf.edu/data/sp_datapro/k-burn/config/

K1 Met Data Data Start - Data Stop Array IDs Present & Array Elements
050599K1.dat 1999,97,1700 - 1999,125,1700 128-19,111-14
051099K1.dat 1999,125,1900 - 1999,130,1300 112-15
052399K1.dat 1999,125,1900 - 1999,143,1400 112-15
K17219.dat 1999,125,1900 - 1999,202,1800 112-15
K1OCT99.dat 1999,202,2200 - 1999,278,1800 112-15,220-22
K1Jann00.dat 1999,278,1900 - 2000,28,1600 112-15,220-22
K1MAY00.dat 2000,28,1700 - 2000,126,2200 112-15,220-22
K1JUN00.dat 2000,126,2300 - 2000,155,1300 112-15,220-22
K1JULY2000.dat 2000,126,2300 - 2000,206,1200 112-15,220-22
BURN.DAT 2002,24,1600 - 2002,29,1700 112-15,112-16,113-15,220-22
2001,95,2200 - 2001,260,800
2002,24,1600 - 2002,29,1700
2001,95,2200 - 2001,364,1900
K1_final_storage_1.dat 2002,35,1800 - 2006,229,1300 113,123,124,220 <--------Best record from logger (2005/09 file lost in HD crash, )
K1_2005-06-04a1.dat 2002,35,1800 - 2005,155,2000 113,123,124,220
k1_2005-07-27a1.dat 2005,155,2100 - 2005,208,1300 124,220
K1_206-10-07.dat 2006,229,1400 - 2006,281,1900 124,220
K1_2007-06-11.dat 2006,281,2000 - 2007,162,1600 124,220
K1_2007-07-22.dat 2007,162,1700 - 2007,203,1800 124,220
K1_2007-09-16.dat 2007,203,1900 - 2007,259,1300 124,220
K1_2008-06-14.dat 2007,259,1400 - 2008,166,1600 131,220
K1_2008-08-27.dat 2008,166,1645 - 2008,240,1720 142,133,220
k1-burn_2009_07_31.dat 2008,166,1800 - 2009,212,1800 142,133,220, 143,134,137,230,235 (thermocouples added?)
k1-burn_2009_09_25.dat 2009,212,1900 - 2009,268,1300 137,235,230,148
K1_2010-06-23.dat 2009,212,1700 - 2010,174,1800 137,230,235,148
k1_2010_08_12.dat 2009,268,1400 - 2010,224,1000 137,230,235,148
K1_2011_08_18.dat 2010,224,1100 - 2011,230,1600 137,235,230,148
K1_2012_08_08.dat 2011,230,1700 - 2012,227,1700 141,220,152
K1_2014_07_23.dat 2012,221,1800 - 2014,204,1400 141,220,152
k1_2015_07_28.dat 2014,204,1500 - 2015,209,1400 141,220,152

Missing Data:
2000 206 1300 to 2001 95 2100
2001 364 1900 to 2002 24 1600
2002 29 1800 to 2002 35 1700

Probably Legit Gaps:
1999 125 1800
1999 202 1900 to 1999 202 2100

KBurn.dat 2001,95,1300 - 2001,232,2100 112,220

Incomplete Array ID Identification
134 = Met, 10 elements
141 = Met, 10 elements
142 = Snow, 8 elements
143 = Snow, 5 elements
152 = Snow, 4 elements
220 - Soil, 23 elements

Teller Bottom Data Downloads

2016-12-06T06:27:34Z

69.161.4.229: /* MicroMet & Radiation Data Tables */

==Data Record for Teller Bottom Site==
===MicroMet & Radiation Data Tables===
<pre>
###############Teller Bottom####################
##############Met & radiation:##################
/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_Snowmelt.dat 2016-04-10 02:00:00 - 2016-04-10 22:15:00
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg"

/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_Snowmelt-2016_04_21.dat 2016-04-10 22:30:00 - 2016-05-02 03:00:00 14 elements
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg"

/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_MicroMet.dat 2016-05-02 03:30:00 - 2016-07-13 19:00:00 15 elements
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg","summer_precip_Tot"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2","mmH2O"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Tot"

/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_MircoMet_2016_08_12.dat 2016-07-13 19:15:00 - 2016-08-12 19:45:00 15 elements
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg","summer_precip_Tot"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2","mmH2O"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Tot"

########################################
## There is a gap here to track down between: 2016-08-12 19:45:00 and 2016-08-17 23:00:00
/var/site/uaf_sp/teller_bottom/raw//teller_bottom_MicroMet_2016_08_17.dat 2016-08-13 05:00:00 - 2016-08-17 22:00:00 21 Elements
"TIMESTAMP","RECORD","at150cm_Avg","dp150cm_Avg","rh150cm","WS_300cm_WVc(1)","WS_300cm_WVc(2)","WS_300cm_WVc(3)","BP","short_up_Avg","short_dn_Avg","long_up_corr_Avg","long_dn_corr_Avg","Net_rad_Avg","albedo_Avg","TargTempC_Avg","TargmV_Avg","SBTempC_Avg","summer_precip_Tot","batt_volt_Min","Ptemp_Min"
"TS","RN","Degrees Celsius","Degrees Celsius","Percent","meters/second","meters/second","meters/second","hPa","","","","","","","Celsius","","","millimeters","Volts DC","Degrees Celsius"
"","","Avg","Avg","Smp","WVc","WVc","WVc","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Tot","Min","Min"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_MicroMet_2016_09_08.dat.backup 2016-08-17 23:00:00 - 2016-09-14 20:00:00 21 elements
"TIMESTAMP","RECORD","at150cm_Avg","dp150cm_Avg","rh150cm","WS_300cm_WVc(1)","WS_300cm_WVc(2)","WS_300cm_WVc(3)","BP","short_up_Avg","short_dn_Avg","long_up_corr_Avg","long_dn_corr_Avg","Net_rad_Avg","albedo_Avg","TargTempC_Avg","TargmV_Avg","SBTempC_Avg","summer_precip_Tot","batt_volt_Min","Ptemp_Min"
"TS","RN","Degrees Celsius","Degrees Celsius","Percent","meters/second","meters/second","meters/second","hPa","","","","","","","Celsius","","","millimeters","Volts DC","Degrees Celsius"
"","","Avg","Avg","Smp","WVc","WVc","WVc","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Tot","Min","Min"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_MicroMet_2016_09_08.dat 2016-09-14 21:00:00 - 2016-09-14 23:00:00 21 elements
"TIMESTAMP","RECORD","at150cm_Avg","dp150cm_Avg","rh150cm","WS_300cm_WVc(1)","WS_300cm_WVc(2)","WS_300cm_WVc(3)","BP","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","Net_rad_Avg","albedo_Avg","TargTempC_Avg","TargmV_Avg","SBTempC_Avg","summer_precip_Tot","batt_volt_Min","Ptemp_Min"
"TS","RN","Degrees Celsius","Degrees Celsius","Percent","meters/second","meters/second","meters/second","hPa","","","","","","","Celsius","","","millimeters","Volts DC","Degrees Celsius"
"","","Avg","Avg","Smp","WVc","WVc","WVc","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Tot","Min","Min"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Spectral.dat 2016-08-13 04:20:00 - 2016-08-17 22:10:00 20 elements
"TIMESTAMP","RECORD","PRI_Up_WB1","PRI_Up_WB2","PRI_Up_orient","PRI_1_WB1","PRI_1_WB2","PRI_1_orient","PRI_2_WB1","PRI_2_WB2","PRI_2_orient","NDVI_Up_WB1","NDVI_Up_WB2","NDVI_Up_orient","NDVI_1_WB1","NDVI_1_WB2","NDVI_1_orient","NDVI_2_WB1","NDVI_2_WB2","NDVI_2_orient"
"TS","RN","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Spectral_2016_09_08.dat.backup 2016-08-17 22:15:00 - 2016-09-09 01:05:00
"TIMESTAMP","RECORD","PRI_Up_WB1","PRI_Up_WB2","PRI_Up_orient","PRI_1_WB1","PRI_1_WB2","PRI_1_orient","PRI_2_WB1","PRI_2_WB2","PRI_2_orient","NDVI_Up_WB1","NDVI_Up_WB2","NDVI_Up_orient","NDVI_1_WB1","NDVI_1_WB2","NDVI_1_orient","NDVI_2_WB1","NDVI_2_WB2","NDVI_2_orient"
"TS","RN","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Spectral_2016_09_08.dat 2016-09-09 01:30:00 - 2016-09-13 23:55:00
"TIMESTAMP","RECORD","PRI_Up_WB1","PRI_Up_WB2","PRI_1_WB1","PRI_1_WB2","PRI_2_WB1","PRI_2_WB2","NDVI_Up_WB1","NDVI_Up_WB2","NDVI_1_WB1","NDVI_1_WB2","NDVI_2_WB1","NDVI_2_WB2"
"TS","RN","W m^-2 nm^-1","W m^-2 nm^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1","W m^-2 nm^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1"
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"
</pre>

===Subsurface===
<pre>
###############Teller Bottom #######################
##############Soils & Subsurface: ##################
/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Subsurface.dat 2016-08-13 05:00:00 - 2016-08-17 22:00:00
"TIMESTAMP","RECORD","SnowDepth_1","SnowDepth_2","at150cm_Avg","VATP_Temps_Avg(1)","VATP_Temps_Avg(2)","VATP_Temps_Avg(3)","VATP_Temps_Avg(4)","VATP_Temps_Avg(5)","VATP_Temps_Avg(6)","VATP_Temps_Avg(7)","VATP_Temps_Avg(8)","VATP_Temps_Avg(9)","VATP_Temps_Avg(10)","VATP_Temps_Avg(11)","VATP_Temps_Avg(12)","VATP_Temps_Avg(13)","VATP_Temps_Avg(14)","VATP_Temps_Avg(15)","VATP_Temps_Avg(16)","Pit_1_Upper_HeatFlux_Avg","Pit_1_Lower_HeatFlux_Avg","pit_temps_Avg(1)","pit_temps_Avg(2)","pit_temps_Avg(3)","pit_temps_Avg(4)","Hydra_moisture(1)","Hydra_moisture(2)","Hydra_moisture(3)","Hydra_moisture(4)","Hydra_moisture(5)","Hydra_temp(1)","Hydra_temp(2)","Hydra_temp(3)","Hydra_temp(4)","Hydra_temp(5)","Hydra_cond(1)","Hydra_cond(2)","Hydra_cond(3)","Hydra_cond(4)","Hydra_cond(5)","Hydra_real_perm(1)","Hydra_real_perm(2)","Hydra_real_perm(3)","Hydra_real_perm(4)","Hydra_real_perm(5)","Hydra_img_perm(1)","Hydra_img_perm(2)","Hydra_img_perm(3)","Hydra_img_perm(4)","Hydra_img_perm(5)","HydraA(1)","HydraA(2)","HydraA(3)","HydraA(4)","HydraA(5)","HydraB(1)","HydraB(2)","HydraB(3)","HydraB(4)","HydraB(5)","HydraC(1)","HydraC(2)","HydraC(3)","HydraC(4)","HydraC(5)","HydraD(1)","HydraD(2)","HydraD(3)","HydraD(4)","HydraD(5)","HydraE(1)","HydraE(2)","HydraE(3)","HydraE(4)","HydraE(5)"
"TS","RN","centimeters","centimeters","Degrees Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","","","Celsius","Celsius","Celsius","Celsius","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Subsurface_2016_09_08.dat 2016-08-17 23:00:0 - 2016-09-14 23:00:00
"TIMESTAMP","RECORD","SnowDepth_1","SnowDepth_2","at150cm_Avg","VATP_Temps_Avg(1)","VATP_Temps_Avg(2)","VATP_Temps_Avg(3)","VATP_Temps_Avg(4)","VATP_Temps_Avg(5)","VATP_Temps_Avg(6)","VATP_Temps_Avg(7)","VATP_Temps_Avg(8)","VATP_Temps_Avg(9)","VATP_Temps_Avg(10)","VATP_Temps_Avg(11)","VATP_Temps_Avg(12)","VATP_Temps_Avg(13)","VATP_Temps_Avg(14)","VATP_Temps_Avg(15)","VATP_Temps_Avg(16)","Pit_1_Upper_HeatFlux_Avg","Pit_1_Lower_HeatFlux_Avg","pit_temps_Avg(1)","pit_temps_Avg(2)","pit_temps_Avg(3)","pit_temps_Avg(4)","Hydra_moisture(1)","Hydra_moisture(2)","Hydra_moisture(3)","Hydra_moisture(4)","Hydra_moisture(5)","Hydra_temp(1)","Hydra_temp(2)","Hydra_temp(3)","Hydra_temp(4)","Hydra_temp(5)","Hydra_cond(1)","Hydra_cond(2)","Hydra_cond(3)","Hydra_cond(4)","Hydra_cond(5)","Hydra_real_perm(1)","Hydra_real_perm(2)","Hydra_real_perm(3)","Hydra_real_perm(4)","Hydra_real_perm(5)","Hydra_img_perm(1)","Hydra_img_perm(2)","Hydra_img_perm(3)","Hydra_img_perm(4)","Hydra_img_perm(5)","HydraA(1)","HydraA(2)","HydraA(3)","HydraA(4)","HydraA(5)","HydraB(1)","HydraB(2)","HydraB(3)","HydraB(4)","HydraB(5)","HydraC(1)","HydraC(2)","HydraC(3)","HydraC(4)","HydraC(5)","HydraD(1)","HydraD(2)","HydraD(3)","HydraD(4)","HydraD(5)","HydraE(1)","HydraE(2)","HydraE(3)","HydraE(4)","HydraE(5)"
"TS","RN","centimeters","centimeters","Degrees Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","","","Celsius","Celsius","Celsius","Celsius","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_TableTC.dat 2016-08-14 00:01:00 - 2016-08-17 00:01:00
"TIMESTAMP","RECORD","time(1)","time(2)","time(3)","time(4)","Trise(1)","Trise(2)","Trise(3)","Trise(4)","init_therm(1)","init_therm(2)","init_therm(3)","init_therm(4)","heat_therm(1)","heat_therm(2)","heat_therm(3)","heat_therm(4)"
"TS","RN","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_TableTC_2016_09_08.dat 2016-08-18 00:01:00 - 2016-09-15 00:08:00
"TIMESTAMP","RECORD","time(1)","time(2)","time(3)","time(4)","Trise(1)","Trise(2)","Trise(3)","Trise(4)","init_therm(1)","init_therm(2)","init_therm(3)","init_therm(4)","heat_therm(1)","heat_therm(2)","heat_therm(3)","heat_therm(4)"
"TS","RN","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"
</pre>
===Snow===
<pre>

############# Teller Bottom ########
############# Snow #################
(4/2016 snow depth is in /var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_Snowmelt.dat

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Snow.dat 2016-08-13 04:20:00 - 2016-09-14 23:15:00
"TIMESTAMP","RECORD","SnowDepth_1","Distance_1","quality_1","SnowDepth_2","Distance_2","quality_2"
"TS","RN","centimeters","","","centimeters","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp"

</pre>

Teller Top Data Downloads

2016-12-05T22:04:21Z

69.161.4.229:

[[Teller Upper Station]]

<pre>
############# Teller Top ########
/var/site/uaf_sp/teller_top/raw/snowmelt_top_Snowmelt-2016_07_13.dat 2016-05-01 00:30:00 - 2016-07-14 00:00:00 11 ELements
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","airtemp_Avg","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg"
"TS","RN","Volts DC","Degrees Celsius","Degrees Celsius","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2"
"","","Min","Min","Avg","Avg","Avg","Avg","Avg","Avg","Avg"

/var/site/uaf_sp/teller_top/raw/snowmelt_top_Snowmelt-2016_08_13.dat 2016-07-14 00:30:00 - 2016-08-13 22:30:00 11 elements
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","airtemp_Avg","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg"
"TS","RN","Volts DC","Degrees Celsius","Degrees Celsius","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2"
"","","Min","Min","Avg","Avg","Avg","Avg","Avg","Avg","Avg"

Top Met & Radiation
/var/site/uaf_sp/teller_top/raw/teller_top_MicroMet-2016_09_10.dat 2016-08-17 02:00:00 - 2016-09-15 02:00:00 20 elements
"TIMESTAMP","RECORD","at150cm_Avg","dp150cm_Avg","rh150cm","WS_300cm_WVc(1)","WS_300cm_WVc(2)","WS_300cm_WVc(3)","short_up_Avg","short_dn_Avg","long_up_corr_Avg","long_dn_corr_Avg","Net_rad_Avg","albedo_Avg","TargTempC_Avg","TargmV_Avg","SBTempC_Avg","summer_precip_Tot","batt_volt_Min","Ptemp_Min"
"TS","RN","Degrees Celsius","Degrees Celsius","Percent","meters/second","meters/second","meters/second","","","","","","","Celsius","","","millimeters","Volts DC","Degrees Celsius"
"","","Avg","Avg","Smp","WVc","WVc","WVc","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Tot","Min","Min"

Subsurface:
/var/site/uaf_sp/teller_top/raw/teller_top_Subsurface-2016_09_10.dat 2016-08-17 02:00:00 - 2016-09-15 02:00:00
/var/site/uaf_sp/teller_top/raw/teller_top_TableTC-2016_09_10.dat 2016-09-15 03:11:30 - 2016-09-15 03:11:30
/var/site/uaf_sp/teller_top/raw/teller_top_TableTC-2016_09_10.dat.backup 2016-08-18 00:01:00 - 2016-09-15 00:01:00

Snow:
/var/site/uaf_sp/teller_top/raw/teller_top_Snow-2016_09_10.dat 2016-08-17 01:55:00 - 2016-09-15 03:30:00
/var/site/uaf_sp/teller_top/raw/teller_top_SnowProfile-2016_09_10.dat 2016-09-15 00:00:00 - 2016-09-15 03:30:00

</pre>

Teller Bottom Data Downloads

2016-12-05T20:25:18Z

69.161.4.229: /* MicroMet & Radiation Data Tables */

==Data Record for Teller Bottom Site==
===MicroMet & Radiation Data Tables===
<pre>
###############Teller Bottom####################
##############Met & radiation:##################
/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_Snowmelt.dat 2016-04-10 02:00:00 - 2016-04-10 22:15:00
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg"

/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_Snowmelt-2016_04_21.dat 2016-04-10 22:30:00 - 2016-05-02 03:00:00
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg"

/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_MicroMet.dat 2016-05-02 03:30:00 - 2016-07-13 19:00:00
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg","summer_precip_Tot"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2","mmH2O"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Tot"

/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_MircoMet_2016_08_12.dat 2016-07-13 19:15:00 - 2016-08-12 19:45:00
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg","summer_precip_Tot"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2","mmH2O"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Tot"

########################################
## There is a gap here to track down between: 2016-08-12 19:45:00 and 2016-08-17 23:00:00

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_MicroMet_2016_09_08.dat.backup 2016-08-17 23:00:00 - 2016-09-14 20:00:00
"TIMESTAMP","RECORD","at150cm_Avg","dp150cm_Avg","rh150cm","WS_300cm_WVc(1)","WS_300cm_WVc(2)","WS_300cm_WVc(3)","BP","short_up_Avg","short_dn_Avg","long_up_corr_Avg","long_dn_corr_Avg","Net_rad_Avg","albedo_Avg","TargTempC_Avg","TargmV_Avg","SBTempC_Avg","summer_precip_Tot","batt_volt_Min","Ptemp_Min"
"TS","RN","Degrees Celsius","Degrees Celsius","Percent","meters/second","meters/second","meters/second","hPa","","","","","","","Celsius","","","millimeters","Volts DC","Degrees Celsius"
"","","Avg","Avg","Smp","WVc","WVc","WVc","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Tot","Min","Min"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_MicroMet_2016_09_08.dat 2016-09-14 21:00:00 - 2016-09-14 23:00:00
"TIMESTAMP","RECORD","at150cm_Avg","dp150cm_Avg","rh150cm","WS_300cm_WVc(1)","WS_300cm_WVc(2)","WS_300cm_WVc(3)","BP","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","Net_rad_Avg","albedo_Avg","TargTempC_Avg","TargmV_Avg","SBTempC_Avg","summer_precip_Tot","batt_volt_Min","Ptemp_Min"
"TS","RN","Degrees Celsius","Degrees Celsius","Percent","meters/second","meters/second","meters/second","hPa","","","","","","","Celsius","","","millimeters","Volts DC","Degrees Celsius"
"","","Avg","Avg","Smp","WVc","WVc","WVc","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Tot","Min","Min"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Spectral.dat 2016-08-13 04:20:00 - 2016-08-17 22:10:00
"TIMESTAMP","RECORD","PRI_Up_WB1","PRI_Up_WB2","PRI_Up_orient","PRI_1_WB1","PRI_1_WB2","PRI_1_orient","PRI_2_WB1","PRI_2_WB2","PRI_2_orient","NDVI_Up_WB1","NDVI_Up_WB2","NDVI_Up_orient","NDVI_1_WB1","NDVI_1_WB2","NDVI_1_orient","NDVI_2_WB1","NDVI_2_WB2","NDVI_2_orient"
"TS","RN","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Spectral_2016_09_08.dat.backup 2016-08-17 22:15:00 - 2016-09-09 01:05:00
"TIMESTAMP","RECORD","PRI_Up_WB1","PRI_Up_WB2","PRI_Up_orient","PRI_1_WB1","PRI_1_WB2","PRI_1_orient","PRI_2_WB1","PRI_2_WB2","PRI_2_orient","NDVI_Up_WB1","NDVI_Up_WB2","NDVI_Up_orient","NDVI_1_WB1","NDVI_1_WB2","NDVI_1_orient","NDVI_2_WB1","NDVI_2_WB2","NDVI_2_orient"
"TS","RN","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Spectral_2016_09_08.dat 2016-09-09 01:30:00 - 2016-09-13 23:55:00
"TIMESTAMP","RECORD","PRI_Up_WB1","PRI_Up_WB2","PRI_1_WB1","PRI_1_WB2","PRI_2_WB1","PRI_2_WB2","NDVI_Up_WB1","NDVI_Up_WB2","NDVI_1_WB1","NDVI_1_WB2","NDVI_2_WB1","NDVI_2_WB2"
"TS","RN","W m^-2 nm^-1","W m^-2 nm^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1","W m^-2 nm^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1"
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"
</pre>

===Subsurface===
<pre>
###############Teller Bottom #######################
##############Soils & Subsurface: ##################
/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Subsurface.dat 2016-08-13 05:00:00 - 2016-08-17 22:00:00
"TIMESTAMP","RECORD","SnowDepth_1","SnowDepth_2","at150cm_Avg","VATP_Temps_Avg(1)","VATP_Temps_Avg(2)","VATP_Temps_Avg(3)","VATP_Temps_Avg(4)","VATP_Temps_Avg(5)","VATP_Temps_Avg(6)","VATP_Temps_Avg(7)","VATP_Temps_Avg(8)","VATP_Temps_Avg(9)","VATP_Temps_Avg(10)","VATP_Temps_Avg(11)","VATP_Temps_Avg(12)","VATP_Temps_Avg(13)","VATP_Temps_Avg(14)","VATP_Temps_Avg(15)","VATP_Temps_Avg(16)","Pit_1_Upper_HeatFlux_Avg","Pit_1_Lower_HeatFlux_Avg","pit_temps_Avg(1)","pit_temps_Avg(2)","pit_temps_Avg(3)","pit_temps_Avg(4)","Hydra_moisture(1)","Hydra_moisture(2)","Hydra_moisture(3)","Hydra_moisture(4)","Hydra_moisture(5)","Hydra_temp(1)","Hydra_temp(2)","Hydra_temp(3)","Hydra_temp(4)","Hydra_temp(5)","Hydra_cond(1)","Hydra_cond(2)","Hydra_cond(3)","Hydra_cond(4)","Hydra_cond(5)","Hydra_real_perm(1)","Hydra_real_perm(2)","Hydra_real_perm(3)","Hydra_real_perm(4)","Hydra_real_perm(5)","Hydra_img_perm(1)","Hydra_img_perm(2)","Hydra_img_perm(3)","Hydra_img_perm(4)","Hydra_img_perm(5)","HydraA(1)","HydraA(2)","HydraA(3)","HydraA(4)","HydraA(5)","HydraB(1)","HydraB(2)","HydraB(3)","HydraB(4)","HydraB(5)","HydraC(1)","HydraC(2)","HydraC(3)","HydraC(4)","HydraC(5)","HydraD(1)","HydraD(2)","HydraD(3)","HydraD(4)","HydraD(5)","HydraE(1)","HydraE(2)","HydraE(3)","HydraE(4)","HydraE(5)"
"TS","RN","centimeters","centimeters","Degrees Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","","","Celsius","Celsius","Celsius","Celsius","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Subsurface_2016_09_08.dat 2016-08-17 23:00:0 - 2016-09-14 23:00:00
"TIMESTAMP","RECORD","SnowDepth_1","SnowDepth_2","at150cm_Avg","VATP_Temps_Avg(1)","VATP_Temps_Avg(2)","VATP_Temps_Avg(3)","VATP_Temps_Avg(4)","VATP_Temps_Avg(5)","VATP_Temps_Avg(6)","VATP_Temps_Avg(7)","VATP_Temps_Avg(8)","VATP_Temps_Avg(9)","VATP_Temps_Avg(10)","VATP_Temps_Avg(11)","VATP_Temps_Avg(12)","VATP_Temps_Avg(13)","VATP_Temps_Avg(14)","VATP_Temps_Avg(15)","VATP_Temps_Avg(16)","Pit_1_Upper_HeatFlux_Avg","Pit_1_Lower_HeatFlux_Avg","pit_temps_Avg(1)","pit_temps_Avg(2)","pit_temps_Avg(3)","pit_temps_Avg(4)","Hydra_moisture(1)","Hydra_moisture(2)","Hydra_moisture(3)","Hydra_moisture(4)","Hydra_moisture(5)","Hydra_temp(1)","Hydra_temp(2)","Hydra_temp(3)","Hydra_temp(4)","Hydra_temp(5)","Hydra_cond(1)","Hydra_cond(2)","Hydra_cond(3)","Hydra_cond(4)","Hydra_cond(5)","Hydra_real_perm(1)","Hydra_real_perm(2)","Hydra_real_perm(3)","Hydra_real_perm(4)","Hydra_real_perm(5)","Hydra_img_perm(1)","Hydra_img_perm(2)","Hydra_img_perm(3)","Hydra_img_perm(4)","Hydra_img_perm(5)","HydraA(1)","HydraA(2)","HydraA(3)","HydraA(4)","HydraA(5)","HydraB(1)","HydraB(2)","HydraB(3)","HydraB(4)","HydraB(5)","HydraC(1)","HydraC(2)","HydraC(3)","HydraC(4)","HydraC(5)","HydraD(1)","HydraD(2)","HydraD(3)","HydraD(4)","HydraD(5)","HydraE(1)","HydraE(2)","HydraE(3)","HydraE(4)","HydraE(5)"
"TS","RN","centimeters","centimeters","Degrees Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","","","Celsius","Celsius","Celsius","Celsius","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_TableTC.dat 2016-08-14 00:01:00 - 2016-08-17 00:01:00
"TIMESTAMP","RECORD","time(1)","time(2)","time(3)","time(4)","Trise(1)","Trise(2)","Trise(3)","Trise(4)","init_therm(1)","init_therm(2)","init_therm(3)","init_therm(4)","heat_therm(1)","heat_therm(2)","heat_therm(3)","heat_therm(4)"
"TS","RN","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_TableTC_2016_09_08.dat 2016-08-18 00:01:00 - 2016-09-15 00:08:00
"TIMESTAMP","RECORD","time(1)","time(2)","time(3)","time(4)","Trise(1)","Trise(2)","Trise(3)","Trise(4)","init_therm(1)","init_therm(2)","init_therm(3)","init_therm(4)","heat_therm(1)","heat_therm(2)","heat_therm(3)","heat_therm(4)"
"TS","RN","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"
</pre>
===Snow===
<pre>

############# Teller Bottom ########
############# Snow #################
(4/2016 snow depth is in /var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_Snowmelt.dat

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Snow.dat 2016-08-13 04:20:00 - 2016-09-14 23:15:00
"TIMESTAMP","RECORD","SnowDepth_1","Distance_1","quality_1","SnowDepth_2","Distance_2","quality_2"
"TS","RN","centimeters","","","centimeters","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp"

</pre>

Teller Surface Water Level Data Downloads

2016-12-05T19:03:54Z

69.161.4.229:

Bottom SW station

===Brief Station History:===
Sensors put in place 4/10/2016. In August 2016 the measurement of the sensors was moved from a CR1000 to a CR10X. The actual sensors were not moved.

===Root Directory for processing===
/var/site/uaf_sp/teller_sw/
Find datapro configuration files:
/var/site/uaf_sp/teller_sw/config/

===Data Downloads===
<pre>
/var/site/uaf_sp/teller_sw/raw/snowmelt_blue_waterlevel-2016_04_21.dat 2016-04-10 22:20:00 - 2016-05-02 05:10:00
"TIMESTAMP","RECORD","ptrans1(1)","ptrans1(2)","ptrans1(3)","ptrans2(1)","ptrans2(2)","ptrans2(3)"
"TS","RN","","","","","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_sw/raw/snowmelt_blue_waterlevel-2016_07_13.dat 2016-05-02 05:20:00 - 2016-07-13 19:00:00
"TIMESTAMP","RECORD","ptrans1(1)","ptrans1(2)","ptrans1(3)","ptrans2(1)","ptrans2(2)","ptrans2(3)"
"TS","RN","","","","","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_sw/raw/snowmelt_blue_waterlevel-2016_08_12.dat 2016-07-13 19:10:00 - 2016-08-12 19:40:00
"TIMESTAMP","RECORD","ptrans1(1)","ptrans1(2)","ptrans1(3)","ptrans2(1)","ptrans2(2)","ptrans2(3)"
"TS","RN","","","","","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_sw/raw/teller_lower_waterlevel-2016_08_12.dat 2016-08-12 19:50:00 - 2016-08-13 03:00:00
"TIMESTAMP","RECORD","ptrans1(1)","ptrans1(2)","ptrans1(3)","ptrans2(1)","ptrans2(2)","ptrans2(3)"
"TS","RN","","","","","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_sw/raw/teller_sw-2016_09_14.dat 2016,225,2210 - 2016,258,2050

</pre>

Teller Bottom Data Downloads

2016-12-05T18:52:22Z

69.161.4.229:

==Data Record for Teller Bottom Site==
===MicroMet & Radiation Data Tables===
<pre>
###############Teller Bottom####################
##############Met & radiation:##################
/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_Snowmelt.dat 2016-04-10 02:00:00 - 2016-04-10 22:15:00
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg"

/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_Snowmelt-2016_04_21.dat 2016-04-10 22:30:00 - 2016-05-02 03:00:00
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg"

/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_MicroMet.dat 2016-05-02 03:30:00 - 2016-07-13 19:00:00
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg","summer_precip_Tot"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2","mmH2O"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Tot"

/var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_MircoMet_2016_08_12.dat 2016-07-13 19:15:00 - 2016-08-12 19:45:00
"TIMESTAMP","RECORD","batt_volt_Min","Ptemp_Min","SnowDepth_1","at1m_Avg","dp1m_Avg","rh1m","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","albedo_Avg","Net_rad_Avg","summer_precip_Tot"
"TS","RN","Volts DC","Degrees Celsius","Centimeters","Degrees Celsius","Degrees Celsius","Percent","Watts / meter^2","Watts / meter^2","Watts / meter^2","Watts / meter^2","Percent","Watts / meter^2","mmH2O"
"","","Min","Min","Smp","Avg","Avg","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Tot"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_MicroMet_2016_09_08.dat.backup 2016-08-17 23:00:00 - 2016-09-14 20:00:00
"TIMESTAMP","RECORD","at150cm_Avg","dp150cm_Avg","rh150cm","WS_300cm_WVc(1)","WS_300cm_WVc(2)","WS_300cm_WVc(3)","BP","short_up_Avg","short_dn_Avg","long_up_corr_Avg","long_dn_corr_Avg","Net_rad_Avg","albedo_Avg","TargTempC_Avg","TargmV_Avg","SBTempC_Avg","summer_precip_Tot","batt_volt_Min","Ptemp_Min"
"TS","RN","Degrees Celsius","Degrees Celsius","Percent","meters/second","meters/second","meters/second","hPa","","","","","","","Celsius","","","millimeters","Volts DC","Degrees Celsius"
"","","Avg","Avg","Smp","WVc","WVc","WVc","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Tot","Min","Min"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_MicroMet_2016_09_08.dat 2016-09-14 21:00:00 - 2016-09-14 23:00:00
"TIMESTAMP","RECORD","at150cm_Avg","dp150cm_Avg","rh150cm","WS_300cm_WVc(1)","WS_300cm_WVc(2)","WS_300cm_WVc(3)","BP","short_up_Avg","short_dn_Avg","long_up_Avg","long_dn_Avg","Net_rad_Avg","albedo_Avg","TargTempC_Avg","TargmV_Avg","SBTempC_Avg","summer_precip_Tot","batt_volt_Min","Ptemp_Min"
"TS","RN","Degrees Celsius","Degrees Celsius","Percent","meters/second","meters/second","meters/second","hPa","","","","","","","Celsius","","","millimeters","Volts DC","Degrees Celsius"
"","","Avg","Avg","Smp","WVc","WVc","WVc","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Tot","Min","Min"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Spectral.dat 2016-08-13 04:20:00 - 2016-08-17 22:10:00
"TIMESTAMP","RECORD","PRI_Up_WB1","PRI_Up_WB2","PRI_Up_orient","PRI_1_WB1","PRI_1_WB2","PRI_1_orient","PRI_2_WB1","PRI_2_WB2","PRI_2_orient","NDVI_Up_WB1","NDVI_Up_WB2","NDVI_Up_orient","NDVI_1_WB1","NDVI_1_WB2","NDVI_1_orient","NDVI_2_WB1","NDVI_2_WB2","NDVI_2_orient"
"TS","RN","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Spectral_2016_09_08.dat.backup 2016-08-17 22:15:00 - 2016-09-09 01:05:00
"TIMESTAMP","RECORD","PRI_Up_WB1","PRI_Up_WB2","PRI_Up_orient","PRI_1_WB1","PRI_1_WB2","PRI_1_orient","PRI_2_WB1","PRI_2_WB2","PRI_2_orient","NDVI_Up_WB1","NDVI_Up_WB2","NDVI_Up_orient","NDVI_1_WB1","NDVI_1_WB2","NDVI_1_orient","NDVI_2_WB1","NDVI_2_WB2","NDVI_2_orient"
"TS","RN","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1","W m^-2 nm^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Spectral_2016_09_08.dat 2016-09-09 01:30:00 - 2016-09-13 23:55:00
"TIMESTAMP","RECORD","PRI_Up_WB1","PRI_Up_WB2","PRI_1_WB1","PRI_1_WB2","PRI_2_WB1","PRI_2_WB2","NDVI_Up_WB1","NDVI_Up_WB2","NDVI_1_WB1","NDVI_1_WB2","NDVI_2_WB1","NDVI_2_WB2"
"TS","RN","W m^-2 nm^-1","W m^-2 nm^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1","W m^-2 nm^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1","W m^-2 nm^-1 sr^-1"
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"
</pre>
===Subsurface===
<pre>
###############Teller Bottom #######################
##############Soils & Subsurface: ##################
/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Subsurface.dat 2016-08-13 05:00:00 - 2016-08-17 22:00:00
"TIMESTAMP","RECORD","SnowDepth_1","SnowDepth_2","at150cm_Avg","VATP_Temps_Avg(1)","VATP_Temps_Avg(2)","VATP_Temps_Avg(3)","VATP_Temps_Avg(4)","VATP_Temps_Avg(5)","VATP_Temps_Avg(6)","VATP_Temps_Avg(7)","VATP_Temps_Avg(8)","VATP_Temps_Avg(9)","VATP_Temps_Avg(10)","VATP_Temps_Avg(11)","VATP_Temps_Avg(12)","VATP_Temps_Avg(13)","VATP_Temps_Avg(14)","VATP_Temps_Avg(15)","VATP_Temps_Avg(16)","Pit_1_Upper_HeatFlux_Avg","Pit_1_Lower_HeatFlux_Avg","pit_temps_Avg(1)","pit_temps_Avg(2)","pit_temps_Avg(3)","pit_temps_Avg(4)","Hydra_moisture(1)","Hydra_moisture(2)","Hydra_moisture(3)","Hydra_moisture(4)","Hydra_moisture(5)","Hydra_temp(1)","Hydra_temp(2)","Hydra_temp(3)","Hydra_temp(4)","Hydra_temp(5)","Hydra_cond(1)","Hydra_cond(2)","Hydra_cond(3)","Hydra_cond(4)","Hydra_cond(5)","Hydra_real_perm(1)","Hydra_real_perm(2)","Hydra_real_perm(3)","Hydra_real_perm(4)","Hydra_real_perm(5)","Hydra_img_perm(1)","Hydra_img_perm(2)","Hydra_img_perm(3)","Hydra_img_perm(4)","Hydra_img_perm(5)","HydraA(1)","HydraA(2)","HydraA(3)","HydraA(4)","HydraA(5)","HydraB(1)","HydraB(2)","HydraB(3)","HydraB(4)","HydraB(5)","HydraC(1)","HydraC(2)","HydraC(3)","HydraC(4)","HydraC(5)","HydraD(1)","HydraD(2)","HydraD(3)","HydraD(4)","HydraD(5)","HydraE(1)","HydraE(2)","HydraE(3)","HydraE(4)","HydraE(5)"
"TS","RN","centimeters","centimeters","Degrees Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","","","Celsius","Celsius","Celsius","Celsius","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Subsurface_2016_09_08.dat 2016-08-17 23:00:0 - 2016-09-14 23:00:00
"TIMESTAMP","RECORD","SnowDepth_1","SnowDepth_2","at150cm_Avg","VATP_Temps_Avg(1)","VATP_Temps_Avg(2)","VATP_Temps_Avg(3)","VATP_Temps_Avg(4)","VATP_Temps_Avg(5)","VATP_Temps_Avg(6)","VATP_Temps_Avg(7)","VATP_Temps_Avg(8)","VATP_Temps_Avg(9)","VATP_Temps_Avg(10)","VATP_Temps_Avg(11)","VATP_Temps_Avg(12)","VATP_Temps_Avg(13)","VATP_Temps_Avg(14)","VATP_Temps_Avg(15)","VATP_Temps_Avg(16)","Pit_1_Upper_HeatFlux_Avg","Pit_1_Lower_HeatFlux_Avg","pit_temps_Avg(1)","pit_temps_Avg(2)","pit_temps_Avg(3)","pit_temps_Avg(4)","Hydra_moisture(1)","Hydra_moisture(2)","Hydra_moisture(3)","Hydra_moisture(4)","Hydra_moisture(5)","Hydra_temp(1)","Hydra_temp(2)","Hydra_temp(3)","Hydra_temp(4)","Hydra_temp(5)","Hydra_cond(1)","Hydra_cond(2)","Hydra_cond(3)","Hydra_cond(4)","Hydra_cond(5)","Hydra_real_perm(1)","Hydra_real_perm(2)","Hydra_real_perm(3)","Hydra_real_perm(4)","Hydra_real_perm(5)","Hydra_img_perm(1)","Hydra_img_perm(2)","Hydra_img_perm(3)","Hydra_img_perm(4)","Hydra_img_perm(5)","HydraA(1)","HydraA(2)","HydraA(3)","HydraA(4)","HydraA(5)","HydraB(1)","HydraB(2)","HydraB(3)","HydraB(4)","HydraB(5)","HydraC(1)","HydraC(2)","HydraC(3)","HydraC(4)","HydraC(5)","HydraD(1)","HydraD(2)","HydraD(3)","HydraD(4)","HydraD(5)","HydraE(1)","HydraE(2)","HydraE(3)","HydraE(4)","HydraE(5)"
"TS","RN","centimeters","centimeters","Degrees Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","Celsius","","","Celsius","Celsius","Celsius","Celsius","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Avg","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_TableTC.dat 2016-08-14 00:01:00 - 2016-08-17 00:01:00
"TIMESTAMP","RECORD","time(1)","time(2)","time(3)","time(4)","Trise(1)","Trise(2)","Trise(3)","Trise(4)","init_therm(1)","init_therm(2)","init_therm(3)","init_therm(4)","heat_therm(1)","heat_therm(2)","heat_therm(3)","heat_therm(4)"
"TS","RN","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_TableTC_2016_09_08.dat 2016-08-18 00:01:00 - 2016-09-15 00:08:00
"TIMESTAMP","RECORD","time(1)","time(2)","time(3)","time(4)","Trise(1)","Trise(2)","Trise(3)","Trise(4)","init_therm(1)","init_therm(2)","init_therm(3)","init_therm(4)","heat_therm(1)","heat_therm(2)","heat_therm(3)","heat_therm(4)"
"TS","RN","","","","","","","","","","","","","","","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp","Smp"
</pre>
===Snow===
<pre>

############# Teller Bottom ########
############# Snow #################
(4/2016 snow depth is in /var/site/uaf_sp/teller_bottom/raw/snowmelt_blue_Snowmelt.dat

/var/site/uaf_sp/teller_bottom/raw/teller_bottom_Snow.dat 2016-08-13 04:20:00 - 2016-09-14 23:15:00
"TIMESTAMP","RECORD","SnowDepth_1","Distance_1","quality_1","SnowDepth_2","Distance_2","quality_2"
"TS","RN","centimeters","","","centimeters","",""
"","","Smp","Smp","Smp","Smp","Smp","Smp"

</pre>

Teller Lower Station

2016-12-05T18:24:40Z

69.161.4.229:

Station also known as Teller Bottom
===Left to fix===
* need to send program one last time to fix out of range error for 5cmbgs soil moisture sensor.
* Emily saw the Barrow Sampling frequency was 30 seconds. You have 60 seconds for Teller

===Sensor Heights===
====Meteorological====
* Air Temperature / Relative Humidity / Dew Point Temperature
** Vaisala HMP155
** 1.85 meters above ground surface
* Windspeed / Wind Direction
** RM Young 05103 Alpine
** 3.74 meters above ground surface
* Summer Precipitation
** Texas Electronics TE-525M
** No altar shield
** Mounted at 0.88 meters
* Barometric Pressure
** Vaisala ____
====Snow====
* Snow Depth at station
** Mounted over blueberry patch
** 2.02 meters
* Snow Depth away from station
** Mounted over 1.5 meter soil temperature profile
* Snow Temperature Profile
** Thermistors every 4cm 0 to 88 cm above ground service (I need to confirm this)
** located in an area that will be drifted in due to presence of willows.
** Need to confirm the model number

====Radiation====
* Interval camera
** Looking generally to the north, 2-4 photos per day
* Skin Surface Temperature
** Apogee SI-111
** 1.50 meters above ground
* 4 component Net Radiation
** Kipp & Zonen CNR4, No fan or heater active
** Shortwave Up/Down
** Longwave Up/Down
** 1.76 meters above ground surface
* NDVI & PRI Up
** Decagon SRS NDVI, SRS PRI
** 1.60 meters above ground surface
* NDVI & PRI #1
** Decagon SRS NDVI, SRS PRI
** 1.60 meters above ground surface
** pointed at area below CNR4
* NDVI & PRI #2
** Decagon SRS NDVI, SRS PRI
** 1.60 meters above ground surface
** pointed at willows
====Subsurface====
* Soil Temperature Profile
** Sensor positions are same as for the Barrow sites, 0 to 1.50 meters below ground surface
** Need to confirm the model number
* Soil Pit
** Thermal Conductivity
*** East30 sensors
*** 10, 20, 30, 40 cm below ground surface
** Soil Temperature
*** East30 sensors
*** 10, 20, 30, 40 cm below ground surface
** Soil Moisture
*** Stevens HydraProbe II
*** 5, 10, 20, 30, 40 cm below ground surface
** Soil Heat Flux
** Hukseflux HFP01
** 5 & 50 cm below ground surface
====Surface Water====
* Station 8/2016 to present @ Northing: 7178304.038, Easting: 454885.819, Elevation: 70.702
* Station 4/2016 to 82016 @ Northing: 7178297.340, Easting: 454864.331, Elevation: 75.558
** PTs in same location 4/2016 to present.
*** First Reading: 4/10/2016 22:20
** 9/2016 one PT removed for winter.
* Stage
** Two INWUSA PT12 (5psi gauge pressure pressure transducers)
* Rebar 1 @ Northing: 7178307.9 Easting: 454898.0 Elevation: 69.931
* Rebar 2 @ Northing: 7178307.933 Easting: 454897.936 Elevation: 70.003
* PT @ (I believe point xsec-pt1: Northing 7178307.626 Easting: 454897.894 Elevation: 69.804

====Coordinates submitted to Lauren @ LANL====
Preliminary Coordinates
<pre>
TL-Bottom-bridge# is the outline of the bridge.
TL-Bottom-netrad = Permanent Net Radiation sensor location 8/2016 forward
TL-Bottom-precip = Permanent Precipitation sensor location 8/2016 forward
TL-Bottom-snopro = Permanent snow temperature profile location 8/2016 forward
TL-Bottom-MM-Stn = Permanent Micromet Tripod location 8/2016 forward
TL-Bottom-SW-Stn = Permanent Surface Water Data logger location 8/2016 forward
TL-Bottom-SnowDepth1 = Snow depth at MM Tripod (estimated with ArcGIS) location 8/2016 forward
TL-Bottom-SnowDepth2 = Snow depth at VATP location 8/2016 forward
TL-Bottom-VATP = 1.5m soil temperature profile location 8/2016 forward
TL-SnowMelt-atrh = Snow melt period Micromet air temperature / relative humidity sensor 4/2016 to 8/2016 (now moved)
TL-SnowMelt-netrad = Snow melt period Micromet net radiation sensor 4/2016 to 8/2016 (now moved)
TL-SnowMelt-precip = Snow melt period Micromet Precipitation sensor 4/2016 to 8/2016 (now moved)
TL-SnowMelt-snopro = Snow melt period snow temperature profile sensor 4/2016 to 8/2016 (now moved)
TL-SW-PT_Rebar-1-3 = the rebars we used to reference for stage reading
TL-SW-pt-xsec-# = Roughly, the cross section at the pressure transducers (1 = the pressure transducer location)
TL-SW-disch-xsec-# = Roughly, the measurement cross section I have been using for discharge measurements

At the top, I didn't have on hand the coordinates from John Peterson so we surveyed in some of the existing infrastructure around TL_IS5
TL_IS5_NE = intensive area corner
TL_IS5_NW = intensive area corner
TL_IS5_SE = intensive area corner
TL_IS5_SW = intensive area corner
TL_IS5_seisFlag1 = LBNL seismic marker flag
TL_IS5_seisFlag2 = LBNL seismic marker flag
TL_IS5_seisFlag3 = LBNL seismic marker flag
TL_IS5_TEONbox = Romanovsky Soil moisture/temperature profile instrument box

TL-Top-netrad = Permanent Net Radiation sensor location 8/2016 forward
TL-Top-precip = Permanent Precipitation sensor location 8/2016 forward
TL-Top-snowpro = Permanent snow temperature profile location 8/2016 forward
TL-Top-SnowDepth1 = snow depth at MM Tripod 8/2016 forward
TL-Top-SnowDepth2 = snow depth away from tripod on top of a hummock deal 8/2016 forward
TL-Top-VATP = soil temperature 1 meter profile location 8/2016 forward
TL-Top-Tripd-WestLeg = Used a bigger tripod than for the bottom site so we surveyed in the legs
TL-Top-Tripd-EastLeg = Used a bigger tripod than for the bottom site so we surveyed in the legs
TL-Top-Tripd-NorthLeg = Used a bigger tripod than for the bottom site so we surveyed in the legs
</pre>

As a CSV file these can be viewed directly in ArcGIS... It looks like I have a bit more work to do here...
<pre>
Point ID,Northing,Easting,Elevation,Geoid Height,Latitude (Local),Longitude (Local),Attributes,X (ECEF),Y (ECEF),Z (ECEF)
new-netrad,7178296.860,454835.642,81.960,,64.72636,-165.94809,,-2648901.993,-662995.035,5744824.810
old-netrad,7178294.609,454862.839,80.916,,64.72635,-165.94752,,-2648896.565,-663021.755,5744823.078
new-precip,7178301.724,454843.494,81.919,,64.72641,-165.94793,,-2648895.716,-663001.485,5744826.900
old-precip,7178300.739,454855.192,81.383,,64.72640,-165.94768,,-2648893.360,-663012.973,5744826.069
new-snopro,7178296.643,454830.391,82.148,,64.72636,-165.94820,,-2648903.605,-662990.027,5744824.853
old-snopro,7178295.372,454827.643,82.058,,64.72635,-165.94826,,-2648905.381,-662987.657,5744824.212
new-station,7178298.590,454835.192,82.140,,64.72638,-165.94810,,-2648900.671,-662994.213,5744825.709
new-sw-stn,7178304.038,454885.819,75.960,,64.72644,-165.94704,,-2648880.391,-663041.256,5744822.770
old-atrh,7178297.340,454864.331,80.816,,64.72637,-165.94749,,-2648893.755,-663022.547,5744824.164
sr50a2,7178307.196,454840.181,82.300,,64.72646,-165.94800,,-2648891.941,-662997.040,5744829.560
teller2 0,7178948.902,454819.322,127.711,,64.73221,-165.94864,,-2648355.330,-662831.320,5745144.509
telsnw2,7178258.459,455055.399,83.085,,64.72605,-165.94347,,-2648879.756,-663216.660,5744810.828
xsec-pt1,7178307.626,454897.894,75.062,,64.72647,-165.94679,,-2648873.792,-663052.000,5744823.567
xsec-pt2,7178307.730,454898.200,75.067,,64.72647,-165.94678,,-2648873.625,-663052.272,5744823.618
xsec-pt3,7178307.635,454898.423,75.062,,64.72647,-165.94678,,-2648873.649,-663052.509,5744823.574
xsec-pt4,7178307.587,454898.545,75.063,,64.72647,-165.94677,,-2648873.660,-663052.639,5744823.555
xsec-pt5,7178307.582,454898.762,75.092,,64.72647,-165.94677,,-2648873.621,-663052.852,5744823.581
xsec-pt6,7178307.486,454898.875,75.192,,64.72647,-165.94677,,-2648873.717,-663052.995,5744823.631
xsec-pt7,7178307.441,454899.028,75.251,,64.72647,-165.94676,,-2648873.741,-663053.160,5744823.666
xsec-pt8,7178307.462,454899.287,75.421,,64.72647,-165.94676,,-2648873.727,-663053.422,5744823.831
xsec-pt9,7178307.770,454897.717,75.578,,64.72647,-165.94679,,-2648873.925,-663051.848,5744824.094
bridge1,7178305.827,454897.430,75.958,,64.72645,-165.94680,,-2648875.854,-663052.065,5744823.606
bridge2,7178305.408,454897.409,76.006,,64.72645,-165.94680,,-2648876.245,-663052.148,5744823.471
bridge3,7178306.011,454900.453,75.879,,64.72645,-165.94673,,-2648874.887,-663054.937,5744823.633
bridge4,7178305.505,454900.524,75.855,,64.72645,-165.94673,,-2648875.301,-663055.122,5744823.395
bxsec-disch1,7178305.994,454899.426,75.351,,64.72645,-165.94676,,-2648874.946,-663053.894,5744823.142
bxsec-disch2,7178305.952,454899.280,75.267,,64.72645,-165.94676,,-2648874.985,-663053.753,5744823.047
bxsec-disch3,7178305.956,454899.144,75.234,,64.72645,-165.94676,,-2648875.003,-663053.617,5744823.017
bxsec-disch4,7178306.012,454899.012,75.200,,64.72645,-165.94676,,-2648874.973,-663053.473,5744823.010
bxsec-disch5,7178305.980,454898.655,75.202,,64.72645,-165.94677,,-2648875.094,-663053.136,5744822.996
bxsec-disch6,7178305.929,454898.410,75.195,,64.72645,-165.94678,,-2648875.198,-663052.910,5744822.966
bxsec-disch7,7178305.957,454898.211,75.147,,64.72645,-165.94678,,-2648875.204,-663052.706,5744822.934
bxsec-disch8,7178305.929,454898.062,75.124,,64.72645,-165.94678,,-2648875.258,-663052.566,5744822.900
bxsec-disch9,7178305.899,454897.882,75.125,,64.72645,-165.94679,,-2648875.330,-663052.399,5744822.886
ptrb-1,7178307.895,454898.036,75.189,,64.72647,-165.94679,,-2648873.574,-663052.088,5744823.798
ptrb-2,7178307.933,454897.936,75.262,,64.72647,-165.94679,,-2648873.596,-663051.989,5744823.879
ptrb-3,7178308.449,454897.478,75.907,,64.72648,-165.94680,,-2648873.530,-663051.493,5744824.681

</pre>

Teller Surface Water Level Data Downloads

2016-12-05T07:28:37Z

69.161.4.229: Created page with "Bottom SW station <pre> /var/site/uaf_sp/teller_sw/raw/snowmelt_blue_waterlevel-2016_04_21.dat 2016-04-10 22:20:00 - 2016-05-02 05:10:00 /var/site/uaf_sp/teller_sw/raw/snow..."

Bottom SW station
<pre>
/var/site/uaf_sp/teller_sw/raw/snowmelt_blue_waterlevel-2016_04_21.dat 2016-04-10 22:20:00 - 2016-05-02 05:10:00
/var/site/uaf_sp/teller_sw/raw/snowmelt_blue_waterlevel-2016_07_13.dat 2016-05-02 05:20:00 - 2016-07-13 19:00:00
/var/site/uaf_sp/teller_sw/raw/snowmelt_blue_waterlevel-2016_08_12.dat 2016-07-13 19:10:00 - 2016-08-12 19:40:00
/var/site/uaf_sp/teller_sw/raw/teller_lower_waterlevel-2016_08_12.dat 2016-08-12 19:50:00 - 2016-08-13 03:00:00
/var/site/uaf_sp/teller_sw/raw/teller_sw-2016_09_14.dat 2016,225,2210 - 2016,258,2050
</pre>

Seward Peninsula Sites

2016-12-05T07:27:35Z

69.161.4.229: /* Data */

==== Trip Itineraries etc.====
* [[2007 September Nome Trip]]
* [[2008 June Nome Trip]]
* [[2008 August Nome Trip]]
* [[2009 August Nome Trip]]
* [[2009 September Nome Trip]]
* [[2010 June Nome Trip]]
* [[2010 August Nome Trip]]
* [[2010 November Nome Trip]]
* [[2011 August Nome Trip]]
* [[2012 August Nome Trip]]
* [[2014 July Nome Trip]]
* [[2015 July Nome Trip]]
* [[2015 September Nome Trip]]
* [[2016 June Nome Trip]]
* [[2016 April Nome Trip]]
* [[2016 Summer Nome Trip]]
* [[2016 NGEE August Nome Trip]]
* [[2016 NGEE September Nome Trip]]
* [[2017 NGEE June Nome Trip]]
* [[ General Site Checklist]]
* [[Storage Inventory]]
* Kougarok Storage: Combo changed in 2015. Ask Bob Busey for access if needed.
* [[Base Callbook]]

==== Data ====
Unofficial data location on ngeedata: 
/home/bbusey/working_files/sp_datapro/raw/ 
Internal wiki data: 
https://wiki.alaska.edu/display/iarcsp/

* [[C1-Grid Data Downloads]]
* [[C2-Blueberry Data Downloads]]
* [[C3-Guy Rowe Data Downloads]]
* [[K1 Burn Data Downloads]]
* [[K2 Met Data Downloads]]
* [[K3 Mauze Data Downloads]]
* [[K1 Snow Thermocouples Data Downloads]]
* [[K2 Snow Thermocouples Data Downloads]]
* [[Anvil Mountain Data Downloads]]
* [[Skookum Pass Data Downloads]]
* [[Kigluaiks Repeater Data Downloads]]
* [[Kigluaiks Repeater 2 Data Downloads]]
* [[Teller Top Data Downloads]]
* [[Teller Bottom Data Downloads]]
* [[Teller Surface Water Level Data Downloads]]
* [[Online Archive Fixes]]

==== Instrumentation Serial Numbers ====
* [[C1-Grid Instrumentation History]]
* [[C2-Blueberry Instrumentation History]]
* [[C3-Guy Rowe Instrumentation History]]
* [[K1 Burn Instrumentation History]]
* [[K2 Met Instrumentation History]]
* [[K3 Mauze Instrumentation History]]
* [[K1 Snow Thermocouples Instrumentation History]]
* [[K2 Snow Thermocouples Instrumentation History]]
* [[Anvil Mountain Instrumentation History]]
* [[Skookum Pass Instrumentation History]]
* [[Kigluaiks Repeater Instrumentation History]]
* [[Kigluaiks Repeater 2 Instrumentation History]]
* [[Teller Upper Station]]
* [[Teller Lower Station]]

====Data QA Notes====
* [[C1-Grid Notes]]
* [[C2-Blueberry Notes]]
* [[C3-Guy Rowe Notes]]
* [[K1 Burn Notes]]
* [[K2 Met Notes]]
* [[K3 Mauze Notes]]
* [[Anvil Mountain NOtes]]
* [[Skookum Pass Notes]]
* [[Kigluaiks Repeater Notes]]
* [[Kigluaiks Repeater 2 Notes]]
* [[Teller Top Notes]]
* [[Teller Bottom Notes]]
* [[Teller Surface Water Level Notes]]

====Sites Background====
As carry over from the ATLAS and HARC projects WERC maintains a network of telemetered data stations on the Seward Peninsula. Archived data is currently found on the [http://www.uaf.edu/water/project/atlas/atlas.html UAF WERC site]

Principal Data site, Council Area:
[[C1 Grid]] (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U27]), active
[[C2 Blueberry]], active
C3 Guy Rowe, inactive

Principal Data site, Kougarok Area:
[[K1 Burn]] site, active
[[K2 Met]], active
[[K3 Mauze]] Gulch (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U28]), active

We use radio telemetry to communicate with our stations out on the peninsula. So, there are several radio repeaters in that list, too. They are:
[[Anvil]] Mountain (near Nome)
[[Skookum]] Pass (near Council)
[[Kigluaiks]] (in Kigluaik Mountains across the valley to the south of Kougarok)

In addition there are several other sites:
[[Snake]] River (outside Nome on Nome-Teller Highway)
[[Kuzitrin]] River (60+ miles up the Kougarok Road from Nome)
[[Niagara]] Creek (~1 mile from K2 Met)

See most of them on a map here:
[http://data.ine.uaf.edu/seward/ Site Map]

Radios turned off for C1-grid, C2-blueberry, Skookum Pass, Anvil Mountain on 11/4/2009

====Project Background====
Originally there were three Council sites, C1, C2, and C3. Each was selected to be in a different type of landscape but C3 kept getting attacked by bears so we eventually removed it. C1 is called C1-Grid because it is located in the valley bottom near Council along the perimeter of a CALM 1 km x 1 km active layer grid. So, the meteorological station was set up to complement the field data collected on the CALM grid. In terms of importance to the original project it was a secondary station so the data elements collected there are basically: Air Temperature / Relative Humidity (1m & 3m), Wind Speed (1m & 3m) & Wind Direction (3m), Net Radiation (in summer), precipitation (in summer), snow depth, soil moisture (vertically profiled), and soil temperature (vertically profiled). The station is also a shorter 3 meter tower.

===Council Sites===
The blueberry site is located on Blueberry Hill to the SW of Council. It is the primary site for the Council area so has more sensors and the tower is a 10 meter high setup. In addition to the sensors mentioned for C1 there is also up facing shortwave & longwave radiation, and down facing shortwave & longwave radiation sensors. Wind speed & AT/RH (as a single sensor) are also measured at more heights than at C1 (1m, 3m & 10m). In 2007 I made some changes to the station. In the past longwave radiation was measured using a battery. However, it was a lead-mercury battery, which is considered hazardous and has been discontinued so I re-did the wiring. This will be reflected in the data as a couple extra data columns that appear last year. Blueberry is across the Niukluk river (I think that's the river, not 100%) and is sometimes unreachable in fall. So, we weren't able to get across the river to remove the net radiometer in Fall 2007. Data for it shouldn't be trusted much past October 1st though because once the snow falls or frost accumulates on the domes that protect the radiation sensor.

On a similar note, the precipitation gages are out year-round but we only record summer rainfall. So, you should mark with '7777' the winter time of year. For measured precipitation I don't generally count it until the air temperature is consistently above freezing. It's kind of subjective on the shoulder seasons figuring out what is snow vs what is rainfall and if you don't feel comfortable with that, I'm happy to look over the data and define select which points to include and which to exclude. It may sound like we're doing something funky but we aren't. The tipping bucket rain gage is only designed to measure liquid precipitation. So, when snow accumulates in the bucket no measurements are recorded until the bucket temperature gets above freezing to melt the snow in the bucket. Since those gages are just used in the summer then we exclude all solid precip that may melt in and be counted at the data logger.

===Kougarok Sites===
K1-burn is another secondary site. It is located on an area burned by an old tundra fire up on a low sloping bench above the Kougarok river. The vegetation in this area is dry tussock and grasses. Much of the organic soil layer burnt off in the fire so recovery has been slow. This site gets visited fairly frequently by animals so you'll see most of the soil moisture sensors have had their cables eaten, as has also happened to the soil temperature sensors. This site is quite similar to C1 that I talked about earlier in terms of sensor setup. In the fall of 2007 we added a snow depth sensor to the site so that will be a new thing. I'm not sure about its reporting though.

K2-met is the 10 meter tower for the Kougarok area. It is located on very wet tundra 300 meters or so up slope from the Kougarok river, so down in the valley. It is a full site similar to C2 although it also has a snow sensor installed. A broken, older sensor was replaced with a newer one in fall 2007.

K3 mauze is located at the perimeter of the Kougarok area CALM grid. It is another 3 meter tower. This site has been hammered by the bears in recent years so some sensors are in and out in terms of their quality.

===Nome Base===
The base radio and moxa server are located at the UAF Northwest campus in Nome.

[[File:NomeAntenna.jpg|400px]]
[[File:NomeRadio.jpg|400px]]

===Discussion of the Station Data===
So, that's a short bit of background on the stations. The data on the CD I gave you is predominantly archived data or data that has been through some level of processing. You can find the up to the day data here: 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/ 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-blueberry 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-burn 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-met 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-kmauze 

Those are all quite large at this point text files. Let me know if you find any gaps in the data. Sometimes there is trouble with the back end connection between the UAF webserver and the computer that retrieves this data over the radio network from Nome.

So, when you look at the data there it will probably look like a bunch of not real helpful numbers. Plus you'll see an Array ID at the start of the line which seems to alternate each hour. That is because the meteorological data arrays are mixed together withe subsurface data arrays. It's kind of a by product of the data logger we use. The design for them was done in the early 90s or late 80s so the electronic data storage architecture is robust but perhaps a bit primitive compared to the computers of today. So, to split out all the data I should have included on that cd a directory called 'batch' or 'bin'. If you look in there you should see some .bat files which are dos batch files. I don't know if you're using linux or windows, if you're using linux then pop open .bat file and you'll see 'grep .........' you should be able to change the directory paths there (I can help if you have questions about this) but these little batch files basically read through the data above (e.g. atlas-blueberry) and pull out all the met data arrays and then all the soil data array lines and dump them into different files. So, when you mention seeing different data files in the CD I gave you, the data is in various steps of processing. Oh, also, if you are using a windows computer (as I am for the data processing) then you should have cygwin installed. It's a program that gives you many of the free utilities that come with linux/unix/debian (most helpfully for us today, grep). You can download it here if you need:
http://www.cygwin.com

So, if you're using windows, just modify the batch files. I like to use this program:
[http://www.context.cx Context]
for viewing text files. I find it much more useful than notepad, but if you have a favorite text editing program, feel free to use it. Anyway, if you're using windows you'll need to modify the batch file to grep out the 2007 data I think. The general format for the grep command is:
grep [text string to search for] [input data file] > or >> [output data file]
c:\cygwin\bin\grep 228,3018,2007 "c:\data\atlas\c1-grid\atlas-c1-grid.txt" > "c:\data\atlas\c1-grid\c1-grid_228-2007-soil.csv"
You'll see both '>' and '>>' in the batch files. '>' means put all the data line matches into the output file and create a new file each time. '>>' means put all the data line matches into the output file and append them to the end of the file if it already exists.

So, you'll need to change the paths (e.g. 'c:\data\'), to whatever you're using. I like
c:\data\project\station_name from windows or /home/bob/data/project/station_name from linux but totally your call on how you organize. From linux the line above would like more like this:
grep 228,3018,2007 "/home/bob/data/atlas/c1-grid/atlas-c1-grid.txt" > "/home/bob/data/atlas/c1-grid/c1-grid_228-2007-soil.csv"

So, when you see stuff like this:
CD/atlas/C1-grid/c1-grid_123-2006-met.csv
CD/atlas/C1-grid/c1-grid_123-2007-met.csv
CD/atlas/C1-grid/c1-grid_228-2004-soil.csv
etc. etc.
That is output from the grep program. the 123/228 references the array ID. So, that file contains met data from 2007 from this file:
[http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid C1 Data File]
Here's a sample line:
123,3018,2005,1,100,-0.47,-0.176,97.7,96.1,3.639,162.7,4.414,92.7,0,-2.046,12.35

I see for some flies the '123' has been replaced by the regular date & time. But that's probably the only change in the file. I would have done that if I was just checking things out.

So, moving forwards a bit, my rough method for processing the data is this:
# download data off website
# use GREP to separate out by data line type and year using the batch file for dos.. or if you use linux then you can actually edit the .bat files to be the correct path (like I mentioned above) and then from the command line type this: chmod u+x c1-grid.bat this will allow you to just type './c1-grid.bat' just as I have been doing but you'll do it from linux rather than windows.
# open in excel or use a program I wrote to process the data (more on this below)
# view the data with a data visualization program I used Origin but Excel is semi okay (origin is nice because you can quickly change the time scales) or if you have another program you really like like matlab that's fine, too.
# delete bad data that is apparent through using the visualization program (replace with '6999')
# have someone else do a final check (like me probably)
# post data to the internet

'process the data' is pretty nebulous. I have a program I wrote which converts thermistor resistances (soil temperature data) from resistance to temperature and then formats the data as it should look on the web. That's the main reason you see so many similarly named files. Some are from step 1, some are from step 2 etc. So, that is probably a bit confusing. But for you, probably using a spreadsheet program like excel or whatever you're most comfortable with is better. You can find header information for what is in each column of the data file. The other main bit of 'processing' that happens is to correct the net radiation data for wind. I can dig that equation up for you if you can't find it in those spreadsheets.
So, also along the process the data lines, some things you'll want to look out for:
# missing lines of data... make a note and check with me if you find any
# repeated lines of data... more likely to find due to the rsyncing process (the way the uaf website talks to the server downloading data from the radio network).
There are a couple ways of converting julian dates into regular. You could use these programs I've written for visual basic:
http://werc.engr.uaf.edu/mediawiki/index.php/Excel_Macros
or, another way is this:
a) type into a cell '12/31/06'
b) change the format of the cell to number.
c) if your data is in the format column 1 is the date you'd like, column 2 is the year, column 3 is the day, column 4 is the hour then in column 1 you can print (using data for 2007) '= 39082 + b1 + c1/2400' then format this column as a date & time and you'll be in business. Do some simple boolean stuff to check for gaps of data or, just create another column that has as the first cell '1/1/07 1:00' then the second row in the column '=a1 + 1/24' then look to see if the calculated date (mentioned first) ever varies from the second method. If it doesn't you're golden. If they do vary then you can track down where the problem is. In general, if there is multiple lines of the same data (or so it looks like) I will delete the first instance and keep the last. The reason for this is related to the radio telemetry. Some of the connections are quite weak so we don't always get the data on an hourly basis. Sometimes we'll get half the line so we'll need to get things resent later (with regard to the radio network and data retrieval from the data loggers).

I guess another note on the array IDs. The met data and the radiation data are contained in the same array ID line. The soil temperature and soil moisture data (most of which has broken sensors) is in a separate array ID. That is the 123, 127 etc. that you wonder about below.

Well, I've written a book, maybe I will post this to our wiki and we can go back and forth on there, too. Totally let me know if you have any questions. I'll be in town through Friday (just learned) so I should be able to help more in the short term than the medium term. thanks

===== Part 2 =====
Here's a bit about the radiation. I will get you more calibration values once we're back from the June trip the week of 6/16.

So, in the data files you've looked at off the internet is the output voltage in millivolts as recorded by the data logger. To get from milliVolt output from the data logger to Net Radiation in Watts / m^2 you'll need to do a couple things:
1) correct for wind speed.
2) apply the sensor specific calibration factor (positive or negative depending on sensor output)

Via this manual (pdf page 5 document page 3):
Net Radiometers -- REBS Q-7.1 (Radiation Energy Balance Systems)
http://www.campbellsci.com/documents/manuals/q-7-1.pdf

Are these wind speed correction equations (where wind speed in in meters per second as we have at our stations):

Positive Wind Speed Correction Factor:
Wind_CF_Pos = 1 + (0.066 * 0.2 * 1m_Wind_Speed) / (0.066 + (0.2 * 1m_Wind_Speed) )

Negative Wind Speed Correction Factor:
Wind_CF_Neg = (0.00174 * 1m_Wind_Speed) + 0.99755

So, if Net Radiation (in mVolts as recorded by the data logger) is positive use this equation:
Corrected_Net_Rad = Wind_CF_Pos * Uncor_Net_Rad * Pos_Calib

If Net Radiation (in mVolts as recorded by the data logger) is negative use this equation:
Corrected_Net_Rad = Wind_CF_Neg * Uncor_Net_Rad * Neg_Calib

Here's the equation for C2 as it looks for me in Excel:
=IF(ABS([NetRadiometer Cell])<> 6999,IF(L4087>0,9.15*[Net Radiometer Cell]*(1+(0.066*0.2*[1 meter Wind Speed])/(0.066+0.2*[1 meter Wind Speed])),11.13*L4087*(0.00174*[1 meter Wind Speed]+0.99755)),7777)

So, given all that you need one more thing, the calibration values for positive and negative calibrations. Here's a first batch and I'll get you some more after we come back from out trip in a couple weeks.
C1 Grid:
Positive Calibration: 9.18
Negative Calibration: 11.31

C2 Blueberry:
Positive Calibration: 9.15
Negative Calibration: 11.13

K1 Burn:
[Need to track down yet, after 2008 June visit]
K2 Met:
[Need to track down yet, after 2008 June visit]
K3 Mauze Gulch: Q99012
Positive Calibration: 8.97
Negative Calibration: 10.90

The raw data input column information you should find in the excel files or here, too:

http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/C1Grid/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/Blueberry/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-burn/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-met/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/kmauze/current-raw.html

The output column information (how everything should be laid out) is here:

http://www.uaf.edu/water/projects/atlas/council/c1/data/met/c1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/soil/c1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/rad/c1_rad_2005.csv

http://www.uaf.edu/water/projects/atlas/council/c2/data/met/c2_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/soil/c2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/rad/c2_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/met/k1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/soil/k1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/rad/k1_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/met/k2_met_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/soil/k2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/rad/k2_rad_2004.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/met/Mauze_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/soil/Mauze_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/rad/Mauze_rad_2006.csv

===Datapro processing notes===
On 5/23/16 we wanted to update the datapro processed output files to use the directly downloaded logger files instead of the ones from the radio network. Here is what we did, using C1 as an example:

*A record of all directly downloaded data files is found here http://ngeedata.iarc.uaf.edu/wiki/C1-Grid_Data_Downloads
*Config files such as Key files to run data pro are here: ngeedata /home/bbusey/working_files/sp_datapro/c1-grid/config/
*Copied raw directly downloaded data files copied from owncloud folder(owncloud/Modern Seward Peninsula/ Seward_Data/raw_data_archive) to /home/bbusey/working_files/sp_datapro/raw/c1-grid/raw
*Deleted some data in each output file in /home/bbusey/working_files/sp_datapro/c1-grid/outputs so that data ended on 12/31/12 or before
**note 1: in this folder are some shortcut links called symlinks for the website).
**note 2: Datapro looks to see what the last line of data is so it doesn’t append if the data is already in the file
*Copied the bash script from /home/bbusey/bin/process_sp.sh too /home/bbusey/process_direct_downloads_bin/process_c1grid.sh so we could edit it and run it on the directly downloaded data instead of the radio downloads
**about bash: when you use ssh to bring up a terminal window, that particular instance is bash terminal or bash shell. so, you can use a bash script to string together a bunch of bash commands. in this case here we will use a bash script to process each .dat direct download using datapro
**this line specifies that we are running datapro version 3, the key file for array 27, and the direct data file we downloaded from C1 in 2014
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_array_127_key.txt --alt_data_files=$ROOT_DIR_SP/raw/c1-grid/RAW/c1_grid_2014_07_22.dat
**then we run that line again with the next data file, or next key file, etc
**we run the bash script for every new line and put the old line in a comments
*Now the files in the /home/bbusey/working_files/sp_datapro/c1-grid/outputs/ folder should then be updated to the end of the last data file
**you can check this by outputting the last two lines of each .csv file in the folder by typing
tail -n2 /home/bbusey/working_files/sp_datapro/c1-grid/outputs/*.csv
* the script we paused that runs hourly on the radio downloaded files looks like
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_key.txt

Seward Peninsula Sites

2016-12-05T07:27:20Z

69.161.4.229: /* Data QA Notes */

==== Trip Itineraries etc.====
* [[2007 September Nome Trip]]
* [[2008 June Nome Trip]]
* [[2008 August Nome Trip]]
* [[2009 August Nome Trip]]
* [[2009 September Nome Trip]]
* [[2010 June Nome Trip]]
* [[2010 August Nome Trip]]
* [[2010 November Nome Trip]]
* [[2011 August Nome Trip]]
* [[2012 August Nome Trip]]
* [[2014 July Nome Trip]]
* [[2015 July Nome Trip]]
* [[2015 September Nome Trip]]
* [[2016 June Nome Trip]]
* [[2016 April Nome Trip]]
* [[2016 Summer Nome Trip]]
* [[2016 NGEE August Nome Trip]]
* [[2016 NGEE September Nome Trip]]
* [[2017 NGEE June Nome Trip]]
* [[ General Site Checklist]]
* [[Storage Inventory]]
* Kougarok Storage: Combo changed in 2015. Ask Bob Busey for access if needed.
* [[Base Callbook]]

==== Data ====
Unofficial data location on ngeedata: 
/home/bbusey/working_files/sp_datapro/raw/ 
Internal wiki data: 
https://wiki.alaska.edu/display/iarcsp/

* [[C1-Grid Data Downloads]]
* [[C2-Blueberry Data Downloads]]
* [[C3-Guy Rowe Data Downloads]]
* [[K1 Burn Data Downloads]]
* [[K2 Met Data Downloads]]
* [[K3 Mauze Data Downloads]]
* [[K1 Snow Thermocouples Data Downloads]]
* [[K2 Snow Thermocouples Data Downloads]]
* [[Anvil Mountain Data Downloads]]
* [[Skookum Pass Data Downloads]]
* [[Kigluaiks Repeater Data Downloads]]
* [[Kigluaiks Repeater 2 Data Downloads]]
* [[Teller Top Data Downloads]]
* [[Teller Bottom Data Downloads]]
* [[Teller Water Level Data Downloads]]
* [[Online Archive Fixes]]

==== Instrumentation Serial Numbers ====
* [[C1-Grid Instrumentation History]]
* [[C2-Blueberry Instrumentation History]]
* [[C3-Guy Rowe Instrumentation History]]
* [[K1 Burn Instrumentation History]]
* [[K2 Met Instrumentation History]]
* [[K3 Mauze Instrumentation History]]
* [[K1 Snow Thermocouples Instrumentation History]]
* [[K2 Snow Thermocouples Instrumentation History]]
* [[Anvil Mountain Instrumentation History]]
* [[Skookum Pass Instrumentation History]]
* [[Kigluaiks Repeater Instrumentation History]]
* [[Kigluaiks Repeater 2 Instrumentation History]]
* [[Teller Upper Station]]
* [[Teller Lower Station]]

====Data QA Notes====
* [[C1-Grid Notes]]
* [[C2-Blueberry Notes]]
* [[C3-Guy Rowe Notes]]
* [[K1 Burn Notes]]
* [[K2 Met Notes]]
* [[K3 Mauze Notes]]
* [[Anvil Mountain NOtes]]
* [[Skookum Pass Notes]]
* [[Kigluaiks Repeater Notes]]
* [[Kigluaiks Repeater 2 Notes]]
* [[Teller Top Notes]]
* [[Teller Bottom Notes]]
* [[Teller Surface Water Level Notes]]

====Sites Background====
As carry over from the ATLAS and HARC projects WERC maintains a network of telemetered data stations on the Seward Peninsula. Archived data is currently found on the [http://www.uaf.edu/water/project/atlas/atlas.html UAF WERC site]

Principal Data site, Council Area:
[[C1 Grid]] (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U27]), active
[[C2 Blueberry]], active
C3 Guy Rowe, inactive

Principal Data site, Kougarok Area:
[[K1 Burn]] site, active
[[K2 Met]], active
[[K3 Mauze]] Gulch (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U28]), active

We use radio telemetry to communicate with our stations out on the peninsula. So, there are several radio repeaters in that list, too. They are:
[[Anvil]] Mountain (near Nome)
[[Skookum]] Pass (near Council)
[[Kigluaiks]] (in Kigluaik Mountains across the valley to the south of Kougarok)

In addition there are several other sites:
[[Snake]] River (outside Nome on Nome-Teller Highway)
[[Kuzitrin]] River (60+ miles up the Kougarok Road from Nome)
[[Niagara]] Creek (~1 mile from K2 Met)

See most of them on a map here:
[http://data.ine.uaf.edu/seward/ Site Map]

Radios turned off for C1-grid, C2-blueberry, Skookum Pass, Anvil Mountain on 11/4/2009

====Project Background====
Originally there were three Council sites, C1, C2, and C3. Each was selected to be in a different type of landscape but C3 kept getting attacked by bears so we eventually removed it. C1 is called C1-Grid because it is located in the valley bottom near Council along the perimeter of a CALM 1 km x 1 km active layer grid. So, the meteorological station was set up to complement the field data collected on the CALM grid. In terms of importance to the original project it was a secondary station so the data elements collected there are basically: Air Temperature / Relative Humidity (1m & 3m), Wind Speed (1m & 3m) & Wind Direction (3m), Net Radiation (in summer), precipitation (in summer), snow depth, soil moisture (vertically profiled), and soil temperature (vertically profiled). The station is also a shorter 3 meter tower.

===Council Sites===
The blueberry site is located on Blueberry Hill to the SW of Council. It is the primary site for the Council area so has more sensors and the tower is a 10 meter high setup. In addition to the sensors mentioned for C1 there is also up facing shortwave & longwave radiation, and down facing shortwave & longwave radiation sensors. Wind speed & AT/RH (as a single sensor) are also measured at more heights than at C1 (1m, 3m & 10m). In 2007 I made some changes to the station. In the past longwave radiation was measured using a battery. However, it was a lead-mercury battery, which is considered hazardous and has been discontinued so I re-did the wiring. This will be reflected in the data as a couple extra data columns that appear last year. Blueberry is across the Niukluk river (I think that's the river, not 100%) and is sometimes unreachable in fall. So, we weren't able to get across the river to remove the net radiometer in Fall 2007. Data for it shouldn't be trusted much past October 1st though because once the snow falls or frost accumulates on the domes that protect the radiation sensor.

On a similar note, the precipitation gages are out year-round but we only record summer rainfall. So, you should mark with '7777' the winter time of year. For measured precipitation I don't generally count it until the air temperature is consistently above freezing. It's kind of subjective on the shoulder seasons figuring out what is snow vs what is rainfall and if you don't feel comfortable with that, I'm happy to look over the data and define select which points to include and which to exclude. It may sound like we're doing something funky but we aren't. The tipping bucket rain gage is only designed to measure liquid precipitation. So, when snow accumulates in the bucket no measurements are recorded until the bucket temperature gets above freezing to melt the snow in the bucket. Since those gages are just used in the summer then we exclude all solid precip that may melt in and be counted at the data logger.

===Kougarok Sites===
K1-burn is another secondary site. It is located on an area burned by an old tundra fire up on a low sloping bench above the Kougarok river. The vegetation in this area is dry tussock and grasses. Much of the organic soil layer burnt off in the fire so recovery has been slow. This site gets visited fairly frequently by animals so you'll see most of the soil moisture sensors have had their cables eaten, as has also happened to the soil temperature sensors. This site is quite similar to C1 that I talked about earlier in terms of sensor setup. In the fall of 2007 we added a snow depth sensor to the site so that will be a new thing. I'm not sure about its reporting though.

K2-met is the 10 meter tower for the Kougarok area. It is located on very wet tundra 300 meters or so up slope from the Kougarok river, so down in the valley. It is a full site similar to C2 although it also has a snow sensor installed. A broken, older sensor was replaced with a newer one in fall 2007.

K3 mauze is located at the perimeter of the Kougarok area CALM grid. It is another 3 meter tower. This site has been hammered by the bears in recent years so some sensors are in and out in terms of their quality.

===Nome Base===
The base radio and moxa server are located at the UAF Northwest campus in Nome.

[[File:NomeAntenna.jpg|400px]]
[[File:NomeRadio.jpg|400px]]

===Discussion of the Station Data===
So, that's a short bit of background on the stations. The data on the CD I gave you is predominantly archived data or data that has been through some level of processing. You can find the up to the day data here: 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/ 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-blueberry 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-burn 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-met 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-kmauze 

Those are all quite large at this point text files. Let me know if you find any gaps in the data. Sometimes there is trouble with the back end connection between the UAF webserver and the computer that retrieves this data over the radio network from Nome.

So, when you look at the data there it will probably look like a bunch of not real helpful numbers. Plus you'll see an Array ID at the start of the line which seems to alternate each hour. That is because the meteorological data arrays are mixed together withe subsurface data arrays. It's kind of a by product of the data logger we use. The design for them was done in the early 90s or late 80s so the electronic data storage architecture is robust but perhaps a bit primitive compared to the computers of today. So, to split out all the data I should have included on that cd a directory called 'batch' or 'bin'. If you look in there you should see some .bat files which are dos batch files. I don't know if you're using linux or windows, if you're using linux then pop open .bat file and you'll see 'grep .........' you should be able to change the directory paths there (I can help if you have questions about this) but these little batch files basically read through the data above (e.g. atlas-blueberry) and pull out all the met data arrays and then all the soil data array lines and dump them into different files. So, when you mention seeing different data files in the CD I gave you, the data is in various steps of processing. Oh, also, if you are using a windows computer (as I am for the data processing) then you should have cygwin installed. It's a program that gives you many of the free utilities that come with linux/unix/debian (most helpfully for us today, grep). You can download it here if you need:
http://www.cygwin.com

So, if you're using windows, just modify the batch files. I like to use this program:
[http://www.context.cx Context]
for viewing text files. I find it much more useful than notepad, but if you have a favorite text editing program, feel free to use it. Anyway, if you're using windows you'll need to modify the batch file to grep out the 2007 data I think. The general format for the grep command is:
grep [text string to search for] [input data file] > or >> [output data file]
c:\cygwin\bin\grep 228,3018,2007 "c:\data\atlas\c1-grid\atlas-c1-grid.txt" > "c:\data\atlas\c1-grid\c1-grid_228-2007-soil.csv"
You'll see both '>' and '>>' in the batch files. '>' means put all the data line matches into the output file and create a new file each time. '>>' means put all the data line matches into the output file and append them to the end of the file if it already exists.

So, you'll need to change the paths (e.g. 'c:\data\'), to whatever you're using. I like
c:\data\project\station_name from windows or /home/bob/data/project/station_name from linux but totally your call on how you organize. From linux the line above would like more like this:
grep 228,3018,2007 "/home/bob/data/atlas/c1-grid/atlas-c1-grid.txt" > "/home/bob/data/atlas/c1-grid/c1-grid_228-2007-soil.csv"

So, when you see stuff like this:
CD/atlas/C1-grid/c1-grid_123-2006-met.csv
CD/atlas/C1-grid/c1-grid_123-2007-met.csv
CD/atlas/C1-grid/c1-grid_228-2004-soil.csv
etc. etc.
That is output from the grep program. the 123/228 references the array ID. So, that file contains met data from 2007 from this file:
[http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid C1 Data File]
Here's a sample line:
123,3018,2005,1,100,-0.47,-0.176,97.7,96.1,3.639,162.7,4.414,92.7,0,-2.046,12.35

I see for some flies the '123' has been replaced by the regular date & time. But that's probably the only change in the file. I would have done that if I was just checking things out.

So, moving forwards a bit, my rough method for processing the data is this:
# download data off website
# use GREP to separate out by data line type and year using the batch file for dos.. or if you use linux then you can actually edit the .bat files to be the correct path (like I mentioned above) and then from the command line type this: chmod u+x c1-grid.bat this will allow you to just type './c1-grid.bat' just as I have been doing but you'll do it from linux rather than windows.
# open in excel or use a program I wrote to process the data (more on this below)
# view the data with a data visualization program I used Origin but Excel is semi okay (origin is nice because you can quickly change the time scales) or if you have another program you really like like matlab that's fine, too.
# delete bad data that is apparent through using the visualization program (replace with '6999')
# have someone else do a final check (like me probably)
# post data to the internet

'process the data' is pretty nebulous. I have a program I wrote which converts thermistor resistances (soil temperature data) from resistance to temperature and then formats the data as it should look on the web. That's the main reason you see so many similarly named files. Some are from step 1, some are from step 2 etc. So, that is probably a bit confusing. But for you, probably using a spreadsheet program like excel or whatever you're most comfortable with is better. You can find header information for what is in each column of the data file. The other main bit of 'processing' that happens is to correct the net radiation data for wind. I can dig that equation up for you if you can't find it in those spreadsheets.
So, also along the process the data lines, some things you'll want to look out for:
# missing lines of data... make a note and check with me if you find any
# repeated lines of data... more likely to find due to the rsyncing process (the way the uaf website talks to the server downloading data from the radio network).
There are a couple ways of converting julian dates into regular. You could use these programs I've written for visual basic:
http://werc.engr.uaf.edu/mediawiki/index.php/Excel_Macros
or, another way is this:
a) type into a cell '12/31/06'
b) change the format of the cell to number.
c) if your data is in the format column 1 is the date you'd like, column 2 is the year, column 3 is the day, column 4 is the hour then in column 1 you can print (using data for 2007) '= 39082 + b1 + c1/2400' then format this column as a date & time and you'll be in business. Do some simple boolean stuff to check for gaps of data or, just create another column that has as the first cell '1/1/07 1:00' then the second row in the column '=a1 + 1/24' then look to see if the calculated date (mentioned first) ever varies from the second method. If it doesn't you're golden. If they do vary then you can track down where the problem is. In general, if there is multiple lines of the same data (or so it looks like) I will delete the first instance and keep the last. The reason for this is related to the radio telemetry. Some of the connections are quite weak so we don't always get the data on an hourly basis. Sometimes we'll get half the line so we'll need to get things resent later (with regard to the radio network and data retrieval from the data loggers).

I guess another note on the array IDs. The met data and the radiation data are contained in the same array ID line. The soil temperature and soil moisture data (most of which has broken sensors) is in a separate array ID. That is the 123, 127 etc. that you wonder about below.

Well, I've written a book, maybe I will post this to our wiki and we can go back and forth on there, too. Totally let me know if you have any questions. I'll be in town through Friday (just learned) so I should be able to help more in the short term than the medium term. thanks

===== Part 2 =====
Here's a bit about the radiation. I will get you more calibration values once we're back from the June trip the week of 6/16.

So, in the data files you've looked at off the internet is the output voltage in millivolts as recorded by the data logger. To get from milliVolt output from the data logger to Net Radiation in Watts / m^2 you'll need to do a couple things:
1) correct for wind speed.
2) apply the sensor specific calibration factor (positive or negative depending on sensor output)

Via this manual (pdf page 5 document page 3):
Net Radiometers -- REBS Q-7.1 (Radiation Energy Balance Systems)
http://www.campbellsci.com/documents/manuals/q-7-1.pdf

Are these wind speed correction equations (where wind speed in in meters per second as we have at our stations):

Positive Wind Speed Correction Factor:
Wind_CF_Pos = 1 + (0.066 * 0.2 * 1m_Wind_Speed) / (0.066 + (0.2 * 1m_Wind_Speed) )

Negative Wind Speed Correction Factor:
Wind_CF_Neg = (0.00174 * 1m_Wind_Speed) + 0.99755

So, if Net Radiation (in mVolts as recorded by the data logger) is positive use this equation:
Corrected_Net_Rad = Wind_CF_Pos * Uncor_Net_Rad * Pos_Calib

If Net Radiation (in mVolts as recorded by the data logger) is negative use this equation:
Corrected_Net_Rad = Wind_CF_Neg * Uncor_Net_Rad * Neg_Calib

Here's the equation for C2 as it looks for me in Excel:
=IF(ABS([NetRadiometer Cell])<> 6999,IF(L4087>0,9.15*[Net Radiometer Cell]*(1+(0.066*0.2*[1 meter Wind Speed])/(0.066+0.2*[1 meter Wind Speed])),11.13*L4087*(0.00174*[1 meter Wind Speed]+0.99755)),7777)

So, given all that you need one more thing, the calibration values for positive and negative calibrations. Here's a first batch and I'll get you some more after we come back from out trip in a couple weeks.
C1 Grid:
Positive Calibration: 9.18
Negative Calibration: 11.31

C2 Blueberry:
Positive Calibration: 9.15
Negative Calibration: 11.13

K1 Burn:
[Need to track down yet, after 2008 June visit]
K2 Met:
[Need to track down yet, after 2008 June visit]
K3 Mauze Gulch: Q99012
Positive Calibration: 8.97
Negative Calibration: 10.90

The raw data input column information you should find in the excel files or here, too:

http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/C1Grid/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/Blueberry/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-burn/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-met/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/kmauze/current-raw.html

The output column information (how everything should be laid out) is here:

http://www.uaf.edu/water/projects/atlas/council/c1/data/met/c1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/soil/c1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/rad/c1_rad_2005.csv

http://www.uaf.edu/water/projects/atlas/council/c2/data/met/c2_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/soil/c2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/rad/c2_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/met/k1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/soil/k1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/rad/k1_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/met/k2_met_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/soil/k2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/rad/k2_rad_2004.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/met/Mauze_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/soil/Mauze_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/rad/Mauze_rad_2006.csv

===Datapro processing notes===
On 5/23/16 we wanted to update the datapro processed output files to use the directly downloaded logger files instead of the ones from the radio network. Here is what we did, using C1 as an example:

*A record of all directly downloaded data files is found here http://ngeedata.iarc.uaf.edu/wiki/C1-Grid_Data_Downloads
*Config files such as Key files to run data pro are here: ngeedata /home/bbusey/working_files/sp_datapro/c1-grid/config/
*Copied raw directly downloaded data files copied from owncloud folder(owncloud/Modern Seward Peninsula/ Seward_Data/raw_data_archive) to /home/bbusey/working_files/sp_datapro/raw/c1-grid/raw
*Deleted some data in each output file in /home/bbusey/working_files/sp_datapro/c1-grid/outputs so that data ended on 12/31/12 or before
**note 1: in this folder are some shortcut links called symlinks for the website).
**note 2: Datapro looks to see what the last line of data is so it doesn’t append if the data is already in the file
*Copied the bash script from /home/bbusey/bin/process_sp.sh too /home/bbusey/process_direct_downloads_bin/process_c1grid.sh so we could edit it and run it on the directly downloaded data instead of the radio downloads
**about bash: when you use ssh to bring up a terminal window, that particular instance is bash terminal or bash shell. so, you can use a bash script to string together a bunch of bash commands. in this case here we will use a bash script to process each .dat direct download using datapro
**this line specifies that we are running datapro version 3, the key file for array 27, and the direct data file we downloaded from C1 in 2014
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_array_127_key.txt --alt_data_files=$ROOT_DIR_SP/raw/c1-grid/RAW/c1_grid_2014_07_22.dat
**then we run that line again with the next data file, or next key file, etc
**we run the bash script for every new line and put the old line in a comments
*Now the files in the /home/bbusey/working_files/sp_datapro/c1-grid/outputs/ folder should then be updated to the end of the last data file
**you can check this by outputting the last two lines of each .csv file in the folder by typing
tail -n2 /home/bbusey/working_files/sp_datapro/c1-grid/outputs/*.csv
* the script we paused that runs hourly on the radio downloaded files looks like
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_key.txt

Teller Bottom Data Downloads

2016-12-05T07:27:00Z

69.161.4.229: Created page with "<pre> snowmelt_blue_MicroMet.dat 2016-05-02 03:30:00 - 2016-07-13 19:00:00 snowmelt_blue_Snowmelt-2016_04_21.dat 2016-04-10 22:30:00 - 2016-05-02 ..."

<pre>
snowmelt_blue_MicroMet.dat 2016-05-02 03:30:00 - 2016-07-13 19:00:00
snowmelt_blue_Snowmelt-2016_04_21.dat 2016-04-10 22:30:00 - 2016-05-02 03:00:00
snowmelt_blue_waterlevel-2016_04_21.dat 2016-04-10 22:20:00 - 2016-05-02 05:10:00
snowmelt_blue_waterlevel-2016_07_13.dat 2016-05-02 05:20:00 - 2016-07-13 19:00:00

Teller Top Data Downloads

2016-12-04T22:03:18Z

69.161.4.229:

[[Teller Upper Station]]

<pre>
teller_top_MicroMet-2016_09_10.dat
teller_top_Snow-2016_09_10.dat
teller_top_SnowProfile-2016_09_10.dat
teller_top_Subsurface-2016_09_10.dat
teller_top_TableTC-2016_09_10.dat
teller_top_TableTC-2016_09_10.dat.backup
</pre>

Teller Bottom Notes

2016-12-04T21:59:44Z

69.161.4.229: Created page with "Teller Lower Station"

[[Teller Lower Station]]

Seward Peninsula Sites

2016-12-04T21:59:07Z

69.161.4.229: /* Instrumentation Serial Numbers */

==== Trip Itineraries etc.====
* [[2007 September Nome Trip]]
* [[2008 June Nome Trip]]
* [[2008 August Nome Trip]]
* [[2009 August Nome Trip]]
* [[2009 September Nome Trip]]
* [[2010 June Nome Trip]]
* [[2010 August Nome Trip]]
* [[2010 November Nome Trip]]
* [[2011 August Nome Trip]]
* [[2012 August Nome Trip]]
* [[2014 July Nome Trip]]
* [[2015 July Nome Trip]]
* [[2015 September Nome Trip]]
* [[2016 June Nome Trip]]
* [[2016 April Nome Trip]]
* [[2016 Summer Nome Trip]]
* [[2016 NGEE August Nome Trip]]
* [[2016 NGEE September Nome Trip]]
* [[2017 NGEE June Nome Trip]]
* [[ General Site Checklist]]
* [[Storage Inventory]]
* Kougarok Storage: Combo changed in 2015. Ask Bob Busey for access if needed.
* [[Base Callbook]]

==== Data ====
Unofficial data location on ngeedata: 
/home/bbusey/working_files/sp_datapro/raw/ 
Internal wiki data: 
https://wiki.alaska.edu/display/iarcsp/

* [[C1-Grid Data Downloads]]
* [[C2-Blueberry Data Downloads]]
* [[C3-Guy Rowe Data Downloads]]
* [[K1 Burn Data Downloads]]
* [[K2 Met Data Downloads]]
* [[K3 Mauze Data Downloads]]
* [[K1 Snow Thermocouples Data Downloads]]
* [[K2 Snow Thermocouples Data Downloads]]
* [[Anvil Mountain Data Downloads]]
* [[Skookum Pass Data Downloads]]
* [[Kigluaiks Repeater Data Downloads]]
* [[Kigluaiks Repeater 2 Data Downloads]]
* [[Teller Top Data Downloads]]
* [[Teller Bottom Data Downloads]]
* [[Teller Water Level Data Downloads]]
* [[Online Archive Fixes]]

==== Instrumentation Serial Numbers ====
* [[C1-Grid Instrumentation History]]
* [[C2-Blueberry Instrumentation History]]
* [[C3-Guy Rowe Instrumentation History]]
* [[K1 Burn Instrumentation History]]
* [[K2 Met Instrumentation History]]
* [[K3 Mauze Instrumentation History]]
* [[K1 Snow Thermocouples Instrumentation History]]
* [[K2 Snow Thermocouples Instrumentation History]]
* [[Anvil Mountain Instrumentation History]]
* [[Skookum Pass Instrumentation History]]
* [[Kigluaiks Repeater Instrumentation History]]
* [[Kigluaiks Repeater 2 Instrumentation History]]
* [[Teller Upper Station]]
* [[Teller Lower Station]]

====Data QA Notes====
* [[C1-Grid Notes]]
* [[C2-Blueberry Notes]]
* [[C3-Guy Rowe Notes]]
* [[K1 Burn Notes]]
* [[K2 Met Notes]]
* [[K3 Mauze Notes]]
* [[Anvil Mountain NOtes]]
* [[Skookum Pass Notes]]
* [[Kigluaiks Repeater Notes]]
* [[Kigluaiks Repeater 2 Notes]]
* [[Teller Top Notes]]
* [[Teller Bottom Notes]]
* [[Teller Water Level Notes]]

====Sites Background====
As carry over from the ATLAS and HARC projects WERC maintains a network of telemetered data stations on the Seward Peninsula. Archived data is currently found on the [http://www.uaf.edu/water/project/atlas/atlas.html UAF WERC site]

Principal Data site, Council Area:
[[C1 Grid]] (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U27]), active
[[C2 Blueberry]], active
C3 Guy Rowe, inactive

Principal Data site, Kougarok Area:
[[K1 Burn]] site, active
[[K2 Met]], active
[[K3 Mauze]] Gulch (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U28]), active

We use radio telemetry to communicate with our stations out on the peninsula. So, there are several radio repeaters in that list, too. They are:
[[Anvil]] Mountain (near Nome)
[[Skookum]] Pass (near Council)
[[Kigluaiks]] (in Kigluaik Mountains across the valley to the south of Kougarok)

In addition there are several other sites:
[[Snake]] River (outside Nome on Nome-Teller Highway)
[[Kuzitrin]] River (60+ miles up the Kougarok Road from Nome)
[[Niagara]] Creek (~1 mile from K2 Met)

See most of them on a map here:
[http://data.ine.uaf.edu/seward/ Site Map]

Radios turned off for C1-grid, C2-blueberry, Skookum Pass, Anvil Mountain on 11/4/2009

====Project Background====
Originally there were three Council sites, C1, C2, and C3. Each was selected to be in a different type of landscape but C3 kept getting attacked by bears so we eventually removed it. C1 is called C1-Grid because it is located in the valley bottom near Council along the perimeter of a CALM 1 km x 1 km active layer grid. So, the meteorological station was set up to complement the field data collected on the CALM grid. In terms of importance to the original project it was a secondary station so the data elements collected there are basically: Air Temperature / Relative Humidity (1m & 3m), Wind Speed (1m & 3m) & Wind Direction (3m), Net Radiation (in summer), precipitation (in summer), snow depth, soil moisture (vertically profiled), and soil temperature (vertically profiled). The station is also a shorter 3 meter tower.

===Council Sites===
The blueberry site is located on Blueberry Hill to the SW of Council. It is the primary site for the Council area so has more sensors and the tower is a 10 meter high setup. In addition to the sensors mentioned for C1 there is also up facing shortwave & longwave radiation, and down facing shortwave & longwave radiation sensors. Wind speed & AT/RH (as a single sensor) are also measured at more heights than at C1 (1m, 3m & 10m). In 2007 I made some changes to the station. In the past longwave radiation was measured using a battery. However, it was a lead-mercury battery, which is considered hazardous and has been discontinued so I re-did the wiring. This will be reflected in the data as a couple extra data columns that appear last year. Blueberry is across the Niukluk river (I think that's the river, not 100%) and is sometimes unreachable in fall. So, we weren't able to get across the river to remove the net radiometer in Fall 2007. Data for it shouldn't be trusted much past October 1st though because once the snow falls or frost accumulates on the domes that protect the radiation sensor.

On a similar note, the precipitation gages are out year-round but we only record summer rainfall. So, you should mark with '7777' the winter time of year. For measured precipitation I don't generally count it until the air temperature is consistently above freezing. It's kind of subjective on the shoulder seasons figuring out what is snow vs what is rainfall and if you don't feel comfortable with that, I'm happy to look over the data and define select which points to include and which to exclude. It may sound like we're doing something funky but we aren't. The tipping bucket rain gage is only designed to measure liquid precipitation. So, when snow accumulates in the bucket no measurements are recorded until the bucket temperature gets above freezing to melt the snow in the bucket. Since those gages are just used in the summer then we exclude all solid precip that may melt in and be counted at the data logger.

===Kougarok Sites===
K1-burn is another secondary site. It is located on an area burned by an old tundra fire up on a low sloping bench above the Kougarok river. The vegetation in this area is dry tussock and grasses. Much of the organic soil layer burnt off in the fire so recovery has been slow. This site gets visited fairly frequently by animals so you'll see most of the soil moisture sensors have had their cables eaten, as has also happened to the soil temperature sensors. This site is quite similar to C1 that I talked about earlier in terms of sensor setup. In the fall of 2007 we added a snow depth sensor to the site so that will be a new thing. I'm not sure about its reporting though.

K2-met is the 10 meter tower for the Kougarok area. It is located on very wet tundra 300 meters or so up slope from the Kougarok river, so down in the valley. It is a full site similar to C2 although it also has a snow sensor installed. A broken, older sensor was replaced with a newer one in fall 2007.

K3 mauze is located at the perimeter of the Kougarok area CALM grid. It is another 3 meter tower. This site has been hammered by the bears in recent years so some sensors are in and out in terms of their quality.

===Nome Base===
The base radio and moxa server are located at the UAF Northwest campus in Nome.

[[File:NomeAntenna.jpg|400px]]
[[File:NomeRadio.jpg|400px]]

===Discussion of the Station Data===
So, that's a short bit of background on the stations. The data on the CD I gave you is predominantly archived data or data that has been through some level of processing. You can find the up to the day data here: 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/ 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-blueberry 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-burn 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-met 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-kmauze 

Those are all quite large at this point text files. Let me know if you find any gaps in the data. Sometimes there is trouble with the back end connection between the UAF webserver and the computer that retrieves this data over the radio network from Nome.

So, when you look at the data there it will probably look like a bunch of not real helpful numbers. Plus you'll see an Array ID at the start of the line which seems to alternate each hour. That is because the meteorological data arrays are mixed together withe subsurface data arrays. It's kind of a by product of the data logger we use. The design for them was done in the early 90s or late 80s so the electronic data storage architecture is robust but perhaps a bit primitive compared to the computers of today. So, to split out all the data I should have included on that cd a directory called 'batch' or 'bin'. If you look in there you should see some .bat files which are dos batch files. I don't know if you're using linux or windows, if you're using linux then pop open .bat file and you'll see 'grep .........' you should be able to change the directory paths there (I can help if you have questions about this) but these little batch files basically read through the data above (e.g. atlas-blueberry) and pull out all the met data arrays and then all the soil data array lines and dump them into different files. So, when you mention seeing different data files in the CD I gave you, the data is in various steps of processing. Oh, also, if you are using a windows computer (as I am for the data processing) then you should have cygwin installed. It's a program that gives you many of the free utilities that come with linux/unix/debian (most helpfully for us today, grep). You can download it here if you need:
http://www.cygwin.com

So, if you're using windows, just modify the batch files. I like to use this program:
[http://www.context.cx Context]
for viewing text files. I find it much more useful than notepad, but if you have a favorite text editing program, feel free to use it. Anyway, if you're using windows you'll need to modify the batch file to grep out the 2007 data I think. The general format for the grep command is:
grep [text string to search for] [input data file] > or >> [output data file]
c:\cygwin\bin\grep 228,3018,2007 "c:\data\atlas\c1-grid\atlas-c1-grid.txt" > "c:\data\atlas\c1-grid\c1-grid_228-2007-soil.csv"
You'll see both '>' and '>>' in the batch files. '>' means put all the data line matches into the output file and create a new file each time. '>>' means put all the data line matches into the output file and append them to the end of the file if it already exists.

So, you'll need to change the paths (e.g. 'c:\data\'), to whatever you're using. I like
c:\data\project\station_name from windows or /home/bob/data/project/station_name from linux but totally your call on how you organize. From linux the line above would like more like this:
grep 228,3018,2007 "/home/bob/data/atlas/c1-grid/atlas-c1-grid.txt" > "/home/bob/data/atlas/c1-grid/c1-grid_228-2007-soil.csv"

So, when you see stuff like this:
CD/atlas/C1-grid/c1-grid_123-2006-met.csv
CD/atlas/C1-grid/c1-grid_123-2007-met.csv
CD/atlas/C1-grid/c1-grid_228-2004-soil.csv
etc. etc.
That is output from the grep program. the 123/228 references the array ID. So, that file contains met data from 2007 from this file:
[http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid C1 Data File]
Here's a sample line:
123,3018,2005,1,100,-0.47,-0.176,97.7,96.1,3.639,162.7,4.414,92.7,0,-2.046,12.35

I see for some flies the '123' has been replaced by the regular date & time. But that's probably the only change in the file. I would have done that if I was just checking things out.

So, moving forwards a bit, my rough method for processing the data is this:
# download data off website
# use GREP to separate out by data line type and year using the batch file for dos.. or if you use linux then you can actually edit the .bat files to be the correct path (like I mentioned above) and then from the command line type this: chmod u+x c1-grid.bat this will allow you to just type './c1-grid.bat' just as I have been doing but you'll do it from linux rather than windows.
# open in excel or use a program I wrote to process the data (more on this below)
# view the data with a data visualization program I used Origin but Excel is semi okay (origin is nice because you can quickly change the time scales) or if you have another program you really like like matlab that's fine, too.
# delete bad data that is apparent through using the visualization program (replace with '6999')
# have someone else do a final check (like me probably)
# post data to the internet

'process the data' is pretty nebulous. I have a program I wrote which converts thermistor resistances (soil temperature data) from resistance to temperature and then formats the data as it should look on the web. That's the main reason you see so many similarly named files. Some are from step 1, some are from step 2 etc. So, that is probably a bit confusing. But for you, probably using a spreadsheet program like excel or whatever you're most comfortable with is better. You can find header information for what is in each column of the data file. The other main bit of 'processing' that happens is to correct the net radiation data for wind. I can dig that equation up for you if you can't find it in those spreadsheets.
So, also along the process the data lines, some things you'll want to look out for:
# missing lines of data... make a note and check with me if you find any
# repeated lines of data... more likely to find due to the rsyncing process (the way the uaf website talks to the server downloading data from the radio network).
There are a couple ways of converting julian dates into regular. You could use these programs I've written for visual basic:
http://werc.engr.uaf.edu/mediawiki/index.php/Excel_Macros
or, another way is this:
a) type into a cell '12/31/06'
b) change the format of the cell to number.
c) if your data is in the format column 1 is the date you'd like, column 2 is the year, column 3 is the day, column 4 is the hour then in column 1 you can print (using data for 2007) '= 39082 + b1 + c1/2400' then format this column as a date & time and you'll be in business. Do some simple boolean stuff to check for gaps of data or, just create another column that has as the first cell '1/1/07 1:00' then the second row in the column '=a1 + 1/24' then look to see if the calculated date (mentioned first) ever varies from the second method. If it doesn't you're golden. If they do vary then you can track down where the problem is. In general, if there is multiple lines of the same data (or so it looks like) I will delete the first instance and keep the last. The reason for this is related to the radio telemetry. Some of the connections are quite weak so we don't always get the data on an hourly basis. Sometimes we'll get half the line so we'll need to get things resent later (with regard to the radio network and data retrieval from the data loggers).

I guess another note on the array IDs. The met data and the radiation data are contained in the same array ID line. The soil temperature and soil moisture data (most of which has broken sensors) is in a separate array ID. That is the 123, 127 etc. that you wonder about below.

Well, I've written a book, maybe I will post this to our wiki and we can go back and forth on there, too. Totally let me know if you have any questions. I'll be in town through Friday (just learned) so I should be able to help more in the short term than the medium term. thanks

===== Part 2 =====
Here's a bit about the radiation. I will get you more calibration values once we're back from the June trip the week of 6/16.

So, in the data files you've looked at off the internet is the output voltage in millivolts as recorded by the data logger. To get from milliVolt output from the data logger to Net Radiation in Watts / m^2 you'll need to do a couple things:
1) correct for wind speed.
2) apply the sensor specific calibration factor (positive or negative depending on sensor output)

Via this manual (pdf page 5 document page 3):
Net Radiometers -- REBS Q-7.1 (Radiation Energy Balance Systems)
http://www.campbellsci.com/documents/manuals/q-7-1.pdf

Are these wind speed correction equations (where wind speed in in meters per second as we have at our stations):

Positive Wind Speed Correction Factor:
Wind_CF_Pos = 1 + (0.066 * 0.2 * 1m_Wind_Speed) / (0.066 + (0.2 * 1m_Wind_Speed) )

Negative Wind Speed Correction Factor:
Wind_CF_Neg = (0.00174 * 1m_Wind_Speed) + 0.99755

So, if Net Radiation (in mVolts as recorded by the data logger) is positive use this equation:
Corrected_Net_Rad = Wind_CF_Pos * Uncor_Net_Rad * Pos_Calib

If Net Radiation (in mVolts as recorded by the data logger) is negative use this equation:
Corrected_Net_Rad = Wind_CF_Neg * Uncor_Net_Rad * Neg_Calib

Here's the equation for C2 as it looks for me in Excel:
=IF(ABS([NetRadiometer Cell])<> 6999,IF(L4087>0,9.15*[Net Radiometer Cell]*(1+(0.066*0.2*[1 meter Wind Speed])/(0.066+0.2*[1 meter Wind Speed])),11.13*L4087*(0.00174*[1 meter Wind Speed]+0.99755)),7777)

So, given all that you need one more thing, the calibration values for positive and negative calibrations. Here's a first batch and I'll get you some more after we come back from out trip in a couple weeks.
C1 Grid:
Positive Calibration: 9.18
Negative Calibration: 11.31

C2 Blueberry:
Positive Calibration: 9.15
Negative Calibration: 11.13

K1 Burn:
[Need to track down yet, after 2008 June visit]
K2 Met:
[Need to track down yet, after 2008 June visit]
K3 Mauze Gulch: Q99012
Positive Calibration: 8.97
Negative Calibration: 10.90

The raw data input column information you should find in the excel files or here, too:

http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/C1Grid/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/Blueberry/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-burn/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-met/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/kmauze/current-raw.html

The output column information (how everything should be laid out) is here:

http://www.uaf.edu/water/projects/atlas/council/c1/data/met/c1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/soil/c1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/rad/c1_rad_2005.csv

http://www.uaf.edu/water/projects/atlas/council/c2/data/met/c2_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/soil/c2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/rad/c2_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/met/k1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/soil/k1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/rad/k1_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/met/k2_met_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/soil/k2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/rad/k2_rad_2004.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/met/Mauze_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/soil/Mauze_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/rad/Mauze_rad_2006.csv

===Datapro processing notes===
On 5/23/16 we wanted to update the datapro processed output files to use the directly downloaded logger files instead of the ones from the radio network. Here is what we did, using C1 as an example:

*A record of all directly downloaded data files is found here http://ngeedata.iarc.uaf.edu/wiki/C1-Grid_Data_Downloads
*Config files such as Key files to run data pro are here: ngeedata /home/bbusey/working_files/sp_datapro/c1-grid/config/
*Copied raw directly downloaded data files copied from owncloud folder(owncloud/Modern Seward Peninsula/ Seward_Data/raw_data_archive) to /home/bbusey/working_files/sp_datapro/raw/c1-grid/raw
*Deleted some data in each output file in /home/bbusey/working_files/sp_datapro/c1-grid/outputs so that data ended on 12/31/12 or before
**note 1: in this folder are some shortcut links called symlinks for the website).
**note 2: Datapro looks to see what the last line of data is so it doesn’t append if the data is already in the file
*Copied the bash script from /home/bbusey/bin/process_sp.sh too /home/bbusey/process_direct_downloads_bin/process_c1grid.sh so we could edit it and run it on the directly downloaded data instead of the radio downloads
**about bash: when you use ssh to bring up a terminal window, that particular instance is bash terminal or bash shell. so, you can use a bash script to string together a bunch of bash commands. in this case here we will use a bash script to process each .dat direct download using datapro
**this line specifies that we are running datapro version 3, the key file for array 27, and the direct data file we downloaded from C1 in 2014
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_array_127_key.txt --alt_data_files=$ROOT_DIR_SP/raw/c1-grid/RAW/c1_grid_2014_07_22.dat
**then we run that line again with the next data file, or next key file, etc
**we run the bash script for every new line and put the old line in a comments
*Now the files in the /home/bbusey/working_files/sp_datapro/c1-grid/outputs/ folder should then be updated to the end of the last data file
**you can check this by outputting the last two lines of each .csv file in the folder by typing
tail -n2 /home/bbusey/working_files/sp_datapro/c1-grid/outputs/*.csv
* the script we paused that runs hourly on the radio downloaded files looks like
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_key.txt

Teller Top Data Downloads

2016-12-04T21:58:38Z

69.161.4.229:

[[Teller Upper Station]]

Teller Top Data Downloads

2016-12-04T21:58:00Z

69.161.4.229: Created page with "Teller Lower Station"

[[Teller Lower Station]]

Seward Peninsula Sites

2016-12-04T21:57:36Z

69.161.4.229:

==== Trip Itineraries etc.====
* [[2007 September Nome Trip]]
* [[2008 June Nome Trip]]
* [[2008 August Nome Trip]]
* [[2009 August Nome Trip]]
* [[2009 September Nome Trip]]
* [[2010 June Nome Trip]]
* [[2010 August Nome Trip]]
* [[2010 November Nome Trip]]
* [[2011 August Nome Trip]]
* [[2012 August Nome Trip]]
* [[2014 July Nome Trip]]
* [[2015 July Nome Trip]]
* [[2015 September Nome Trip]]
* [[2016 June Nome Trip]]
* [[2016 April Nome Trip]]
* [[2016 Summer Nome Trip]]
* [[2016 NGEE August Nome Trip]]
* [[2016 NGEE September Nome Trip]]
* [[2017 NGEE June Nome Trip]]
* [[ General Site Checklist]]
* [[Storage Inventory]]
* Kougarok Storage: Combo changed in 2015. Ask Bob Busey for access if needed.
* [[Base Callbook]]

==== Data ====
Unofficial data location on ngeedata: 
/home/bbusey/working_files/sp_datapro/raw/ 
Internal wiki data: 
https://wiki.alaska.edu/display/iarcsp/

* [[C1-Grid Data Downloads]]
* [[C2-Blueberry Data Downloads]]
* [[C3-Guy Rowe Data Downloads]]
* [[K1 Burn Data Downloads]]
* [[K2 Met Data Downloads]]
* [[K3 Mauze Data Downloads]]
* [[K1 Snow Thermocouples Data Downloads]]
* [[K2 Snow Thermocouples Data Downloads]]
* [[Anvil Mountain Data Downloads]]
* [[Skookum Pass Data Downloads]]
* [[Kigluaiks Repeater Data Downloads]]
* [[Kigluaiks Repeater 2 Data Downloads]]
* [[Teller Top Data Downloads]]
* [[Teller Bottom Data Downloads]]
* [[Teller Water Level Data Downloads]]
* [[Online Archive Fixes]]

==== Instrumentation Serial Numbers ====
* [[C1-Grid Instrumentation History]]
* [[C2-Blueberry Instrumentation History]]
* [[C3-Guy Rowe Instrumentation History]]
* [[K1 Burn Instrumentation History]]
* [[K2 Met Instrumentation History]]
* [[K3 Mauze Instrumentation History]]
* [[K1 Snow Thermocouples Instrumentation History]]
* [[K2 Snow Thermocouples Instrumentation History]]
* [[Anvil Mountain Instrumentation History]]
* [[Skookum Pass Instrumentation History]]
* [[Kigluaiks Repeater Instrumentation History]]
* [[Kigluaiks Repeater 2 Instrumentation History]]

====Data QA Notes====
* [[C1-Grid Notes]]
* [[C2-Blueberry Notes]]
* [[C3-Guy Rowe Notes]]
* [[K1 Burn Notes]]
* [[K2 Met Notes]]
* [[K3 Mauze Notes]]
* [[Anvil Mountain NOtes]]
* [[Skookum Pass Notes]]
* [[Kigluaiks Repeater Notes]]
* [[Kigluaiks Repeater 2 Notes]]
* [[Teller Top Notes]]
* [[Teller Bottom Notes]]
* [[Teller Water Level Notes]]

====Sites Background====
As carry over from the ATLAS and HARC projects WERC maintains a network of telemetered data stations on the Seward Peninsula. Archived data is currently found on the [http://www.uaf.edu/water/project/atlas/atlas.html UAF WERC site]

Principal Data site, Council Area:
[[C1 Grid]] (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U27]), active
[[C2 Blueberry]], active
C3 Guy Rowe, inactive

Principal Data site, Kougarok Area:
[[K1 Burn]] site, active
[[K2 Met]], active
[[K3 Mauze]] Gulch (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U28]), active

We use radio telemetry to communicate with our stations out on the peninsula. So, there are several radio repeaters in that list, too. They are:
[[Anvil]] Mountain (near Nome)
[[Skookum]] Pass (near Council)
[[Kigluaiks]] (in Kigluaik Mountains across the valley to the south of Kougarok)

In addition there are several other sites:
[[Snake]] River (outside Nome on Nome-Teller Highway)
[[Kuzitrin]] River (60+ miles up the Kougarok Road from Nome)
[[Niagara]] Creek (~1 mile from K2 Met)

See most of them on a map here:
[http://data.ine.uaf.edu/seward/ Site Map]

Radios turned off for C1-grid, C2-blueberry, Skookum Pass, Anvil Mountain on 11/4/2009

====Project Background====
Originally there were three Council sites, C1, C2, and C3. Each was selected to be in a different type of landscape but C3 kept getting attacked by bears so we eventually removed it. C1 is called C1-Grid because it is located in the valley bottom near Council along the perimeter of a CALM 1 km x 1 km active layer grid. So, the meteorological station was set up to complement the field data collected on the CALM grid. In terms of importance to the original project it was a secondary station so the data elements collected there are basically: Air Temperature / Relative Humidity (1m & 3m), Wind Speed (1m & 3m) & Wind Direction (3m), Net Radiation (in summer), precipitation (in summer), snow depth, soil moisture (vertically profiled), and soil temperature (vertically profiled). The station is also a shorter 3 meter tower.

===Council Sites===
The blueberry site is located on Blueberry Hill to the SW of Council. It is the primary site for the Council area so has more sensors and the tower is a 10 meter high setup. In addition to the sensors mentioned for C1 there is also up facing shortwave & longwave radiation, and down facing shortwave & longwave radiation sensors. Wind speed & AT/RH (as a single sensor) are also measured at more heights than at C1 (1m, 3m & 10m). In 2007 I made some changes to the station. In the past longwave radiation was measured using a battery. However, it was a lead-mercury battery, which is considered hazardous and has been discontinued so I re-did the wiring. This will be reflected in the data as a couple extra data columns that appear last year. Blueberry is across the Niukluk river (I think that's the river, not 100%) and is sometimes unreachable in fall. So, we weren't able to get across the river to remove the net radiometer in Fall 2007. Data for it shouldn't be trusted much past October 1st though because once the snow falls or frost accumulates on the domes that protect the radiation sensor.

On a similar note, the precipitation gages are out year-round but we only record summer rainfall. So, you should mark with '7777' the winter time of year. For measured precipitation I don't generally count it until the air temperature is consistently above freezing. It's kind of subjective on the shoulder seasons figuring out what is snow vs what is rainfall and if you don't feel comfortable with that, I'm happy to look over the data and define select which points to include and which to exclude. It may sound like we're doing something funky but we aren't. The tipping bucket rain gage is only designed to measure liquid precipitation. So, when snow accumulates in the bucket no measurements are recorded until the bucket temperature gets above freezing to melt the snow in the bucket. Since those gages are just used in the summer then we exclude all solid precip that may melt in and be counted at the data logger.

===Kougarok Sites===
K1-burn is another secondary site. It is located on an area burned by an old tundra fire up on a low sloping bench above the Kougarok river. The vegetation in this area is dry tussock and grasses. Much of the organic soil layer burnt off in the fire so recovery has been slow. This site gets visited fairly frequently by animals so you'll see most of the soil moisture sensors have had their cables eaten, as has also happened to the soil temperature sensors. This site is quite similar to C1 that I talked about earlier in terms of sensor setup. In the fall of 2007 we added a snow depth sensor to the site so that will be a new thing. I'm not sure about its reporting though.

K2-met is the 10 meter tower for the Kougarok area. It is located on very wet tundra 300 meters or so up slope from the Kougarok river, so down in the valley. It is a full site similar to C2 although it also has a snow sensor installed. A broken, older sensor was replaced with a newer one in fall 2007.

K3 mauze is located at the perimeter of the Kougarok area CALM grid. It is another 3 meter tower. This site has been hammered by the bears in recent years so some sensors are in and out in terms of their quality.

===Nome Base===
The base radio and moxa server are located at the UAF Northwest campus in Nome.

[[File:NomeAntenna.jpg|400px]]
[[File:NomeRadio.jpg|400px]]

===Discussion of the Station Data===
So, that's a short bit of background on the stations. The data on the CD I gave you is predominantly archived data or data that has been through some level of processing. You can find the up to the day data here: 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/ 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-blueberry 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-burn 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-met 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-kmauze 

Those are all quite large at this point text files. Let me know if you find any gaps in the data. Sometimes there is trouble with the back end connection between the UAF webserver and the computer that retrieves this data over the radio network from Nome.

So, when you look at the data there it will probably look like a bunch of not real helpful numbers. Plus you'll see an Array ID at the start of the line which seems to alternate each hour. That is because the meteorological data arrays are mixed together withe subsurface data arrays. It's kind of a by product of the data logger we use. The design for them was done in the early 90s or late 80s so the electronic data storage architecture is robust but perhaps a bit primitive compared to the computers of today. So, to split out all the data I should have included on that cd a directory called 'batch' or 'bin'. If you look in there you should see some .bat files which are dos batch files. I don't know if you're using linux or windows, if you're using linux then pop open .bat file and you'll see 'grep .........' you should be able to change the directory paths there (I can help if you have questions about this) but these little batch files basically read through the data above (e.g. atlas-blueberry) and pull out all the met data arrays and then all the soil data array lines and dump them into different files. So, when you mention seeing different data files in the CD I gave you, the data is in various steps of processing. Oh, also, if you are using a windows computer (as I am for the data processing) then you should have cygwin installed. It's a program that gives you many of the free utilities that come with linux/unix/debian (most helpfully for us today, grep). You can download it here if you need:
http://www.cygwin.com

So, if you're using windows, just modify the batch files. I like to use this program:
[http://www.context.cx Context]
for viewing text files. I find it much more useful than notepad, but if you have a favorite text editing program, feel free to use it. Anyway, if you're using windows you'll need to modify the batch file to grep out the 2007 data I think. The general format for the grep command is:
grep [text string to search for] [input data file] > or >> [output data file]
c:\cygwin\bin\grep 228,3018,2007 "c:\data\atlas\c1-grid\atlas-c1-grid.txt" > "c:\data\atlas\c1-grid\c1-grid_228-2007-soil.csv"
You'll see both '>' and '>>' in the batch files. '>' means put all the data line matches into the output file and create a new file each time. '>>' means put all the data line matches into the output file and append them to the end of the file if it already exists.

So, you'll need to change the paths (e.g. 'c:\data\'), to whatever you're using. I like
c:\data\project\station_name from windows or /home/bob/data/project/station_name from linux but totally your call on how you organize. From linux the line above would like more like this:
grep 228,3018,2007 "/home/bob/data/atlas/c1-grid/atlas-c1-grid.txt" > "/home/bob/data/atlas/c1-grid/c1-grid_228-2007-soil.csv"

So, when you see stuff like this:
CD/atlas/C1-grid/c1-grid_123-2006-met.csv
CD/atlas/C1-grid/c1-grid_123-2007-met.csv
CD/atlas/C1-grid/c1-grid_228-2004-soil.csv
etc. etc.
That is output from the grep program. the 123/228 references the array ID. So, that file contains met data from 2007 from this file:
[http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid C1 Data File]
Here's a sample line:
123,3018,2005,1,100,-0.47,-0.176,97.7,96.1,3.639,162.7,4.414,92.7,0,-2.046,12.35

I see for some flies the '123' has been replaced by the regular date & time. But that's probably the only change in the file. I would have done that if I was just checking things out.

So, moving forwards a bit, my rough method for processing the data is this:
# download data off website
# use GREP to separate out by data line type and year using the batch file for dos.. or if you use linux then you can actually edit the .bat files to be the correct path (like I mentioned above) and then from the command line type this: chmod u+x c1-grid.bat this will allow you to just type './c1-grid.bat' just as I have been doing but you'll do it from linux rather than windows.
# open in excel or use a program I wrote to process the data (more on this below)
# view the data with a data visualization program I used Origin but Excel is semi okay (origin is nice because you can quickly change the time scales) or if you have another program you really like like matlab that's fine, too.
# delete bad data that is apparent through using the visualization program (replace with '6999')
# have someone else do a final check (like me probably)
# post data to the internet

'process the data' is pretty nebulous. I have a program I wrote which converts thermistor resistances (soil temperature data) from resistance to temperature and then formats the data as it should look on the web. That's the main reason you see so many similarly named files. Some are from step 1, some are from step 2 etc. So, that is probably a bit confusing. But for you, probably using a spreadsheet program like excel or whatever you're most comfortable with is better. You can find header information for what is in each column of the data file. The other main bit of 'processing' that happens is to correct the net radiation data for wind. I can dig that equation up for you if you can't find it in those spreadsheets.
So, also along the process the data lines, some things you'll want to look out for:
# missing lines of data... make a note and check with me if you find any
# repeated lines of data... more likely to find due to the rsyncing process (the way the uaf website talks to the server downloading data from the radio network).
There are a couple ways of converting julian dates into regular. You could use these programs I've written for visual basic:
http://werc.engr.uaf.edu/mediawiki/index.php/Excel_Macros
or, another way is this:
a) type into a cell '12/31/06'
b) change the format of the cell to number.
c) if your data is in the format column 1 is the date you'd like, column 2 is the year, column 3 is the day, column 4 is the hour then in column 1 you can print (using data for 2007) '= 39082 + b1 + c1/2400' then format this column as a date & time and you'll be in business. Do some simple boolean stuff to check for gaps of data or, just create another column that has as the first cell '1/1/07 1:00' then the second row in the column '=a1 + 1/24' then look to see if the calculated date (mentioned first) ever varies from the second method. If it doesn't you're golden. If they do vary then you can track down where the problem is. In general, if there is multiple lines of the same data (or so it looks like) I will delete the first instance and keep the last. The reason for this is related to the radio telemetry. Some of the connections are quite weak so we don't always get the data on an hourly basis. Sometimes we'll get half the line so we'll need to get things resent later (with regard to the radio network and data retrieval from the data loggers).

I guess another note on the array IDs. The met data and the radiation data are contained in the same array ID line. The soil temperature and soil moisture data (most of which has broken sensors) is in a separate array ID. That is the 123, 127 etc. that you wonder about below.

Well, I've written a book, maybe I will post this to our wiki and we can go back and forth on there, too. Totally let me know if you have any questions. I'll be in town through Friday (just learned) so I should be able to help more in the short term than the medium term. thanks

===== Part 2 =====
Here's a bit about the radiation. I will get you more calibration values once we're back from the June trip the week of 6/16.

So, in the data files you've looked at off the internet is the output voltage in millivolts as recorded by the data logger. To get from milliVolt output from the data logger to Net Radiation in Watts / m^2 you'll need to do a couple things:
1) correct for wind speed.
2) apply the sensor specific calibration factor (positive or negative depending on sensor output)

Via this manual (pdf page 5 document page 3):
Net Radiometers -- REBS Q-7.1 (Radiation Energy Balance Systems)
http://www.campbellsci.com/documents/manuals/q-7-1.pdf

Are these wind speed correction equations (where wind speed in in meters per second as we have at our stations):

Positive Wind Speed Correction Factor:
Wind_CF_Pos = 1 + (0.066 * 0.2 * 1m_Wind_Speed) / (0.066 + (0.2 * 1m_Wind_Speed) )

Negative Wind Speed Correction Factor:
Wind_CF_Neg = (0.00174 * 1m_Wind_Speed) + 0.99755

So, if Net Radiation (in mVolts as recorded by the data logger) is positive use this equation:
Corrected_Net_Rad = Wind_CF_Pos * Uncor_Net_Rad * Pos_Calib

If Net Radiation (in mVolts as recorded by the data logger) is negative use this equation:
Corrected_Net_Rad = Wind_CF_Neg * Uncor_Net_Rad * Neg_Calib

Here's the equation for C2 as it looks for me in Excel:
=IF(ABS([NetRadiometer Cell])<> 6999,IF(L4087>0,9.15*[Net Radiometer Cell]*(1+(0.066*0.2*[1 meter Wind Speed])/(0.066+0.2*[1 meter Wind Speed])),11.13*L4087*(0.00174*[1 meter Wind Speed]+0.99755)),7777)

So, given all that you need one more thing, the calibration values for positive and negative calibrations. Here's a first batch and I'll get you some more after we come back from out trip in a couple weeks.
C1 Grid:
Positive Calibration: 9.18
Negative Calibration: 11.31

C2 Blueberry:
Positive Calibration: 9.15
Negative Calibration: 11.13

K1 Burn:
[Need to track down yet, after 2008 June visit]
K2 Met:
[Need to track down yet, after 2008 June visit]
K3 Mauze Gulch: Q99012
Positive Calibration: 8.97
Negative Calibration: 10.90

The raw data input column information you should find in the excel files or here, too:

http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/C1Grid/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/Blueberry/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-burn/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-met/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/kmauze/current-raw.html

The output column information (how everything should be laid out) is here:

http://www.uaf.edu/water/projects/atlas/council/c1/data/met/c1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/soil/c1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/rad/c1_rad_2005.csv

http://www.uaf.edu/water/projects/atlas/council/c2/data/met/c2_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/soil/c2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/rad/c2_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/met/k1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/soil/k1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/rad/k1_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/met/k2_met_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/soil/k2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/rad/k2_rad_2004.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/met/Mauze_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/soil/Mauze_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/rad/Mauze_rad_2006.csv

===Datapro processing notes===
On 5/23/16 we wanted to update the datapro processed output files to use the directly downloaded logger files instead of the ones from the radio network. Here is what we did, using C1 as an example:

*A record of all directly downloaded data files is found here http://ngeedata.iarc.uaf.edu/wiki/C1-Grid_Data_Downloads
*Config files such as Key files to run data pro are here: ngeedata /home/bbusey/working_files/sp_datapro/c1-grid/config/
*Copied raw directly downloaded data files copied from owncloud folder(owncloud/Modern Seward Peninsula/ Seward_Data/raw_data_archive) to /home/bbusey/working_files/sp_datapro/raw/c1-grid/raw
*Deleted some data in each output file in /home/bbusey/working_files/sp_datapro/c1-grid/outputs so that data ended on 12/31/12 or before
**note 1: in this folder are some shortcut links called symlinks for the website).
**note 2: Datapro looks to see what the last line of data is so it doesn’t append if the data is already in the file
*Copied the bash script from /home/bbusey/bin/process_sp.sh too /home/bbusey/process_direct_downloads_bin/process_c1grid.sh so we could edit it and run it on the directly downloaded data instead of the radio downloads
**about bash: when you use ssh to bring up a terminal window, that particular instance is bash terminal or bash shell. so, you can use a bash script to string together a bunch of bash commands. in this case here we will use a bash script to process each .dat direct download using datapro
**this line specifies that we are running datapro version 3, the key file for array 27, and the direct data file we downloaded from C1 in 2014
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_array_127_key.txt --alt_data_files=$ROOT_DIR_SP/raw/c1-grid/RAW/c1_grid_2014_07_22.dat
**then we run that line again with the next data file, or next key file, etc
**we run the bash script for every new line and put the old line in a comments
*Now the files in the /home/bbusey/working_files/sp_datapro/c1-grid/outputs/ folder should then be updated to the end of the last data file
**you can check this by outputting the last two lines of each .csv file in the folder by typing
tail -n2 /home/bbusey/working_files/sp_datapro/c1-grid/outputs/*.csv
* the script we paused that runs hourly on the radio downloaded files looks like
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_key.txt

NGEE Data Log Record

2016-12-04T21:42:50Z

69.161.4.229: /* Manual Reviews */

== Radiation Dataset Barrow ==

* Intense Long Dn Avg, SW Up Avg: 5/22/14 sites B, C, D duplicate timestep EKY
* Intense Long Dn Avg: 12/8/15-1/15/16 missing data site A EKY
==Manual Reviews==
* Checked out and updated all data elements through 12/1/2016
* created several excel qc fix files that are currently on computer elephant but should be moved to ngeedata.iarc
* later in december should be set to move working data directory to /var/site.
* http://ngeedata.iarc.uaf.edu/data/barrow/viz/
** I don't have that page being created automatically but will later this month (12/2016)

NGEE Data Log Record

2016-12-04T21:42:01Z

69.161.4.229:

C2-Blueberry Notes

2016-11-29T22:02:11Z

69.161.4.229: Created page with "<pre> AT10m -- Is gap/bad data 3/23/2003 to 4/11/2003 legit? 6999 out 5/23/2004 12:00 to 6/10/2004 19:00 Is gap/bad data 1/26/2008 to 6/13/2008 legit? Is gap/bad data 12/18/2..."

<pre>
AT10m --
Is gap/bad data 3/23/2003 to 4/11/2003 legit?
6999 out 5/23/2004 12:00 to 6/10/2004 19:00
Is gap/bad data 1/26/2008 to 6/13/2008 legit?
Is gap/bad data 12/18/2008 to 8/5/2009 legit?
Is bad data 12/23/2011 to present legit?

AT1m --
Is gap/bad data 3/23/2003 to 4/11/2003 legit?
6999 out 5/23/2004 12:00 to 6/10/2004 19:00
Is bad data 12/23/2011 to 8/6/2012 musk ox?

AT3m --
Is gap/bad data 3/23/2003 to 4/11/2003 legit?
6999 out 5/23/2004 15:00 to 6/10/2004 17:00

Is bad data 12/23/2011 to 8/6/2012 musk ox?

Battery --
Data gap 3/23/2003 to 4/11/2003 legit?
Super Funk 5/23/2004 12:00 to 6/10/2004 19:00 --> could it be a bear?
Dead Battery: 1/26/2008 to 6/13/2008

SM10 --
</pre>

C2-Blueberry Data Downloads

2016-11-29T06:57:57Z

69.161.4.229:

C2 Blueberry Data Data Start - Data Stop Array IDs Present
C2JULY2000.CSV 1999,209,1400 - 2000,208,1600
C2OCT99.DAT 1999,209,1400 - 1999,275,1900
BLUE.DAT 2001,97,1400 - 2001,137,600
C20401.DAT 2001,9,500 - 2001,97,1300
C2102701.DAT 2000,229,1200 - 2000,301,1200
COUN2001.DAT 2000,256,1500 - 2000,301,1200
COUN2002.DAT JUST: 2000,301,1300
COUNT2003.DAT 2000,301,1400 - 2001,27,1400
BLUEBERR.DAT 2001,97,140 - 2001,281,1900
BLUEBR01.DAT 2001,192,1200 - 2001,229,1200
BBDATA1999.XLS 1999,209,1500 - 1999,365,2400
BBDATA2000.XLS 2000,1,300 - 2000,366,2400
BBDATA2001.XLS 2001,1,300 - 2001,281,900
bluedat.txt 2001,277,600 - 2001,297,1900
Blueberry_2005-06-05-a1.dat 2002,70,1700 - 2005,156,2100
Blueberry_2005-07-28-a1.dat 2005,156,2200 - 2005,209,2200 131,230
C2_final_storage_1.dat 2002,231,2400 - 2006,227,1100 130,131,230
C2_2006-10-06.dat 2006,227,1200 - 2006,279,1700 131,230
C2_2007-06-09.dat 2006,279,1800 - 2007,160,1800 131,230
C2_2007-07-26.dat 2007,160,1900 - 2007,207,1100 167,168,230
C2_2008-06-13.dat 2007,207,1200 - 2008,25,2400 168,230
C2_2008-08-26.dat 2008,165,1500 - 2008,239,1900 179,191,237 (still alpha thermistors for thermistor string)
c2_blueberry_2009_08_05.dat 2008,239,1900 - 2009,217,1200 179,191,237 (ysi thermistors for thermistor string)
c2_blueberry_2009_09_27.dat 2009,217,1300 - 2009,270,1000 179,196,237
C2_2010_06_26.dat 2009,217,1200 - 2010,177,2100 179,196,237
C2_2010_08_10.dat 2009,270,1100 - 2010,222,1900 179,196.237
c2_blueberry_2011_08_16.dat 2010,222,2100 - 2011,228,1800 179,182,196,238
c2_blueberry_2012_08_06.dat 2010,282,1500 - 2012,219,1700 179,182,238,196,250
c2_blueberry_2012_radio-only_gap.dat 2012,219,1800 - 2012,288,600 182,238,250
c2_blueberry_2014_07_24.dat 2012,288,600 - 2014,205,1200 182,238,250
c2_blueberry_2014_radio-only_gap.dat 2014,205,100 - 2014,269,2400 182,238,250
c2_blueberry_2016_08_18.dat 2014,249,400 - 2016,231,1000 182,238,250

SHAND RECORD 2001,365,2400 - PRESENT
Ralph now carries out the duties of Shand.

Legit Gaps: 2003 82 2100 to 2003 101 1500
2002 168 2100 to 2002 217 1800
2002 97 900 to 2002 97 1500
2008 26 0000 to 2008 165 1400 (dead battery)
2010 222 2000 (station maintenance)
2014 0124 0200 to 2014 0206 1200 (low battery voltage. There are a few data points in that period, but spotty)
2015 8/24 15:00 -- After shand.engr shut down there is some major funkyness going down with the clock (see note below)
I think this will only be resolved with a site visit.

Incomplete Array ID Identification:
130 = 23 elements, Met
131 = 24 elements, Met
168 = 30 elements, Met
179 = 29 elements, Met
182 = 29 elements, Met
191 = 5 elements, Snow
191 = 7 elements, Snow
196 = 4 elements, Snow
250 = 4 elements, Snow
230 - 22, Soil
230 - 23, Soil
237 - 24, Soil
238 = 25, Soil

==========================================================================
Just generally, in the fall of 2009 the Incoming shortwave radiometer wires at the radiation stand become loose. There are some garbage values interspersed that fall that I got tired of cleaning up. Primarily mid October to Mid December. Maybe all of that data should just be zeroed out.

===Fall 2015 to Spring 2016===
In August 2015 shand.engr.uaf.edu was shut down. Shand had been the remote data retrieval computer from 2002 to 2015. Over the fall of 2015 we had a number of issues transitioning away from shand. It quickly became obvious that LoggerNet could not consistently retrieve data from the tenuous radio links on the SP. I moved the most recent version of Loggerdata.pl to host lognet so that we could take advantage of the older, more stable data retrieval software on the new computer. However, it turns out that there is a major bug specific to CentOS, the operating system running on LogNet. The time zone of the logger specified in the configuration files used by Loggerdata.pl was ignored on centOS so August 2015 to early January 2016 there was some fighting back and forth with the two competing data retrieval programs. I inadvertently fixed this by disabling loggerdata downloads for this site in late December / early January. However, due to poor performance I re-enabled again in March to try to get the sites caught up before our April trip. Around then I figured out the timezone bug present when running things on LogNet / CentOS. So, data retrievals were moved to host Ralph which is a debian OS based computer. Downloading became more consistent at that point and I'm not sure why we haven't heard from the Council sites since early May but could be a bear again. But, in light of this there is quite a bit of data in the Blueberry hill data stream with a mix of timezone (UTC vs Ak Standard at UTC-9hrs). It is only apparent which time zone the data is in if you look at the incoming shortwave radiation data and so I thought I would create this section to delineate as best I can tell. Definitely a bit of an own goal...

Station origination to
"2015-08-24 15:00:00" AKSTD
"2015-08-24 01:00:00" UTC
"2015-08-25 01:00:00" AKSTD
"2015-08-25 00:00:00" From here till the next several months, a low value @ 0:00 or 1:00 = AKSTD, a high value = UTC
If that makes any sense.
Like this sampler:
"2015-09-02 00:00:00",0.00 (AKSTD)
"2015-09-02 00:00:00",0.00 (AKSTD)
"2015-09-02 00:00:00",0.00 (AKSTD)
"2015-09-02 01:00:00",0.00 (AKSTD)
"2015-09-02 01:00:00",11.24 (AKSTD)
"2015-09-02 00:00:00",126.00 (UTC)
"2015-09-02 01:00:00",147.70 (UTC)
"2015-09-02 01:00:00",128.10 (UTC)
"2015-09-02 01:00:00",54.21 (UTC)
"2015-09-02 00:00:00",9.77 (AKSTD)
"2015-09-02 01:00:00",0.07 (AKSTD)
"2015-12-20 00:00:00" UTC
"2015-12-29 00:00:00" AKSTD
"2016-03-28 18:00:00" UTC
"2016-03-29 17:00:00" AKSTD
"2016-03-30 00:00:00" UTC
"2016-03-30 16:00:00" AKSTD

C1-Grid Notes

2016-11-16T07:38:13Z

69.161.4.229: /* Soils Data */

===Met Data===
* I'm not going to re-review 1999-2008
====2009====
* Air Temperature: All Good
* Rain:
** Start: 5/20/2009 9:00
** End: 10/13/2009 6:00
* RH: All Good
* WS/WD: All Good
====2010====
* Air Temperature: All Good
* Rain:
** Start: 5/12/2010 7:00
** End: 8/8/2010 13:00 (start of legit data gap)
* RH: All Good
* WS/WD: All Good
====2011====
* Air Temperature: All Good
** Dead Battery: 1/1/2011 to 2/10/2011 13:00
* Rain:
** Start: 5/12/2011 9:00
** End: 10/15/2011 21:00
* RH: All Good
** Dead Battery: 1/1/2011 to 2/10/2011 13:00
* WS/WD: All Good
====2012====
* Air Temperature: All Good
* Rain:
** Start: 5/15/2012 11:00
** End: 10/7/2012 9:00
* RH: All Good
* WS/WD: All Good
====2013====
* Air Temperature: All Good
* Rain:
** Start: 5/24/2013 8:00
** End: 10/29/2013 7:00
* RH: All Good
* WS/WD: All Good
====2014====
* Air Temperature: All Good
* Rain:
** Start: 5/7/2014 12:00
** End: 10/4/2014 1:00
* RH: All Good
* WS/WD: All Good
====2015====
* Air Temperature: All Good
* Rain:
** Start: 5/6/2015 9:00
** End: 9/25/2015 11:00
* RH: All Good
* WS/WD: All Good

===Soils Data===
I think this is the thermistor model here (found an old spreadsheet from Andy):
D:\projects_usb_drive\UAF\From Crane\archived files from past workers\ANDYS\STUFF\Thermistortemplate.xls
=IF(A3>60,6999,IF(A3<3,6999,IF(A3=0,6999,IF(A3=-6999,-6999,IF(A3=6999,6999,1/(0.00128+0.000237*(LN(A3*1000))+0.0000000906*((LN(A3*1000))^3))-273.15)))))

Here is the info that matches that from my 'thermistor_information.xls'
Alpha 13A5001-DOC 5/29/1998 25694 275 ±0.1 C 1.28E-03 2.37E-04 9.06E-08
August 2008:
YSI 44004 Jessie 06D2432 -- ±0.2 C 0.001465769 0.000238619 0.000000100 -13.4 From Jessie installed on SP in August 2008

* I'll just note here, I wanted to extract old soil moisture data and using datapro on ''raw/c1-grid/RAW/CR10X_final_storage_1.dat'' the date index was totally screwed up. Well, the starting data was correct but the y values were out of order in a big way. I ended up grepping the 228 array into another text file and used that as the source for datapro. Messed me up for a good day or so.
 
* Ended up reprocessing the soil moisture data, the data archive was a mix of garbage and who knows what.

'''Need to confirm depths for all thermistors... otherwise complete'''
====1999====
* Soil temps: Good
* Soil Moisture: OK - Use data from archive

====2000====
* Soil temps: Good
* Soil Moisture: OK - Use data from archive
====2001====
* Soil temps: Good
* Soil Moisture: OK -- Use data from archive
====2002====
* Soil temps: Good
* Soil Moisture: OK -- Use reprocessed data
====2003====
* Soil temps: Couple blips in April, added to QA sheets.
* Soil Moisture: OK -- Use reprocessed data
====2004====
* Soil temps: Good
* Soil Moisture: OK -- Use reprocessed data
====2005====
* Soil temps: '''Accidentally Dropped from Record?'''
* Soil Moisture: OK -- Use reprocessed data
====2006====
* Soil temps: '''Accidentally Dropped from Record?'''
* Soil Moisture: OK -- Use reprocessed data
====2007====
* Soil temps: '''Accidentally Dropped from Record?'''
* Soil Moisture: OK -- Use reprocessed data
====2008====
* Soil temps: '''Accidentally Dropped from Record?;''' Need to add third string on bit of 2008 here.
* Soil Moisture: OK -- Use reprocessed data
====2009====
* Soil temps:OK
* Soil Moisture: OK -- Use reprocessed data
====2010====
* Soil temps: OK
* Soil Moisture: OK -- Use reprocessed data

====2011====
* Soil temps: Cable Chewed Fall 2011.
* Soil Moisture: OK -- Use reprocessed data

====2012====
* Soil temps:
* Soil Moisture: OK -- Use reprocessed data

====2013====
* Soil temps:
* Soil Moisture: OK -- Use reprocessed data

====2014====
* Soil temps:
* Soil Moisture: OK -- Use reprocessed data

====2015====
* Soil temps:
* Soil Moisture: OK -- Use reprocessed data

====MRC Probe====
* 2012 Summer jacked about 8 cm (from photo, Lumix TSC4)
* 2013: Did not visit
* 2014 7/22/14: Jacked out 16 cm (from photo)
* 2016 August: 37 cm (Cohen fieldbook)

@ Installation, 7/24/2011:
T1 7.6 cmbgs
T2 12.7 cm bgs
T3 22.9 cm bgs
T4 37.7 cm bgs
T5 52.5 cm bgs
T6 67.3 cm bgs
T7 82.7 cm bgs
T8 108 cm bgs

C1-Grid Notes

2016-11-16T07:37:13Z

69.161.4.229: /* Soils Data */

===Met Data===
* I'm not going to re-review 1999-2008
====2009====
* Air Temperature: All Good
* Rain:
** Start: 5/20/2009 9:00
** End: 10/13/2009 6:00
* RH: All Good
* WS/WD: All Good
====2010====
* Air Temperature: All Good
* Rain:
** Start: 5/12/2010 7:00
** End: 8/8/2010 13:00 (start of legit data gap)
* RH: All Good
* WS/WD: All Good
====2011====
* Air Temperature: All Good
** Dead Battery: 1/1/2011 to 2/10/2011 13:00
* Rain:
** Start: 5/12/2011 9:00
** End: 10/15/2011 21:00
* RH: All Good
** Dead Battery: 1/1/2011 to 2/10/2011 13:00
* WS/WD: All Good
====2012====
* Air Temperature: All Good
* Rain:
** Start: 5/15/2012 11:00
** End: 10/7/2012 9:00
* RH: All Good
* WS/WD: All Good
====2013====
* Air Temperature: All Good
* Rain:
** Start: 5/24/2013 8:00
** End: 10/29/2013 7:00
* RH: All Good
* WS/WD: All Good
====2014====
* Air Temperature: All Good
* Rain:
** Start: 5/7/2014 12:00
** End: 10/4/2014 1:00
* RH: All Good
* WS/WD: All Good
====2015====
* Air Temperature: All Good
* Rain:
** Start: 5/6/2015 9:00
** End: 9/25/2015 11:00
* RH: All Good
* WS/WD: All Good

===Soils Data===
I think this is the thermistor model here (found an old spreadsheet from Andy):
D:\projects_usb_drive\UAF\From Crane\archived files from past workers\ANDYS\STUFF\Thermistortemplate.xls
=IF(A3>60,6999,IF(A3<3,6999,IF(A3=0,6999,IF(A3=-6999,-6999,IF(A3=6999,6999,1/(0.00128+0.000237*(LN(A3*1000))+0.0000000906*((LN(A3*1000))^3))-273.15)))))

Here is the info that matches that from my 'thermistor_information.xls'
Alpha 13A5001-DOC 5/29/1998 25694 275 ±0.1 C 1.28E-03 2.37E-04 9.06E-08
August 2008:
YSI 44004 Jessie 06D2432 -- ±0.2 C 0.001465769 0.000238619 0.000000100 -13.4 From Jessie installed on SP in August 2008

* I'll just note here, I wanted to extract old soil moisture data and using datapro on ''raw/c1-grid/RAW/CR10X_final_storage_1.dat'' the date index was totally screwed up. Well, the starting data was correct but the y values were out of order in a big way. I ended up grepping the 228 array into another text file and used that as the source for datapro. Messed me up for a good day or so.
 
* Ended up reprocessing the soil moisture data, the data archive was a mix of garbage and who knows what.
====1999====
* Soil temps: Good
* Soil Moisture: OK - Use data from archive

====2000====
* Soil temps: Good
* Soil Moisture: OK - Use data from archive
====2001====
* Soil temps: Good
* Soil Moisture: OK -- Use data from archive
====2002====
* Soil temps: Good
* Soil Moisture: OK -- Use reprocessed data
====2003====
* Soil temps: Couple blips in April, added to QA sheets.
* Soil Moisture: OK -- Use reprocessed data
====2004====
* Soil temps: Good
* Soil Moisture: OK -- Use reprocessed data
====2005====
* Soil temps: '''Accidentally Dropped from Record?'''
* Soil Moisture: OK -- Use reprocessed data
====2006====
* Soil temps: '''Accidentally Dropped from Record?'''
* Soil Moisture: OK -- Use reprocessed data
====2007====
* Soil temps: '''Accidentally Dropped from Record?'''
* Soil Moisture: OK -- Use reprocessed data
====2008====
* Soil temps: '''Accidentally Dropped from Record?;''' Need to add third string on bit of 2008 here.
* Soil Moisture: OK -- Use reprocessed data
====2009====
* Soil temps:OK
* Soil Moisture: OK -- Use reprocessed data
====2010====
* Soil temps: OK
* Soil Moisture: OK -- Use reprocessed data

====2011====
* Soil temps: Cable Chewed Fall 2011.
* Soil Moisture: OK -- Use reprocessed data

====2012====
* Soil temps:
* Soil Moisture: OK -- Use reprocessed data

====2013====
* Soil temps:
* Soil Moisture: OK -- Use reprocessed data

====2014====
* Soil temps:
* Soil Moisture: OK -- Use reprocessed data

====2015====
* Soil temps:
* Soil Moisture: OK -- Use reprocessed data

====MRC Probe====
* 2012 Summer jacked about 8 cm (from photo, Lumix TSC4)
* 2013: Did not visit
* 2014 7/22/14: Jacked out 16 cm (from photo)
* 2016 August: 37 cm (Cohen fieldbook)

@ Installation, 7/24/2011:
T1 7.6 cmbgs
T2 12.7 cm bgs
T3 22.9 cm bgs
T4 37.7 cm bgs
T5 52.5 cm bgs
T6 67.3 cm bgs
T7 82.7 cm bgs
T8 108 cm bgs

C1-Grid Notes

2016-11-16T05:59:41Z

69.161.4.229: /* 1999 */

===Met Data===
* I'm not going to re-review 1999-2008
====2009====
* Air Temperature: All Good
* Rain:
** Start: 5/20/2009 9:00
** End: 10/13/2009 6:00
* RH: All Good
* WS/WD: All Good
====2010====
* Air Temperature: All Good
* Rain:
** Start: 5/12/2010 7:00
** End: 8/8/2010 13:00 (start of legit data gap)
* RH: All Good
* WS/WD: All Good
====2011====
* Air Temperature: All Good
** Dead Battery: 1/1/2011 to 2/10/2011 13:00
* Rain:
** Start: 5/12/2011 9:00
** End: 10/15/2011 21:00
* RH: All Good
** Dead Battery: 1/1/2011 to 2/10/2011 13:00
* WS/WD: All Good
====2012====
* Air Temperature: All Good
* Rain:
** Start: 5/15/2012 11:00
** End: 10/7/2012 9:00
* RH: All Good
* WS/WD: All Good
====2013====
* Air Temperature: All Good
* Rain:
** Start: 5/24/2013 8:00
** End: 10/29/2013 7:00
* RH: All Good
* WS/WD: All Good
====2014====
* Air Temperature: All Good
* Rain:
** Start: 5/7/2014 12:00
** End: 10/4/2014 1:00
* RH: All Good
* WS/WD: All Good
====2015====
* Air Temperature: All Good
* Rain:
** Start: 5/6/2015 9:00
** End: 9/25/2015 11:00
* RH: All Good
* WS/WD: All Good

===Soils Data===
I think this is the thermistor model here (found an old spreadsheet from Andy):
D:\projects_usb_drive\UAF\From Crane\archived files from past workers\ANDYS\STUFF\Thermistortemplate.xls
=IF(A3>60,6999,IF(A3<3,6999,IF(A3=0,6999,IF(A3=-6999,-6999,IF(A3=6999,6999,1/(0.00128+0.000237*(LN(A3*1000))+0.0000000906*((LN(A3*1000))^3))-273.15)))))

Here is the info that matches that from my 'thermistor_information.xls'
Alpha 13A5001-DOC 5/29/1998 25694 275 ±0.1 C 1.28E-03 2.37E-04 9.06E-08
August 2008:
YSI 44004 Jessie 06D2432 -- ±0.2 C 0.001465769 0.000238619 0.000000100 -13.4 From Jessie installed on SP in August 2008

* I'll just note here, I wanted to extract old soil moisture data and using datapro on ''raw/c1-grid/RAW/CR10X_final_storage_1.dat'' the date index was totally screwed up. Well, the starting data was correct but the y values were out of order in a big way. I ended up grepping the 228 array into another text file and used that as the source for datapro. Messed me up for a good day or so.
 
* Ended up reprocessing the soil moisture data, the data archive was a mix of garbage and who knows what.
====1999====
* Soil temps: Good
* Soil Moisture: OK - Use data from archive

====2000====
* Soil temps: Good
* Soil Moisture: OK
====2001====
* Soil temps: Good
* Soil Moisture: OK
====2002====
* Soil temps: Good
* Soil Moisture: SM5 has turned into a soil temperature value.
====2003====
* Soil temps: Couple blips in April, added to QA sheets.
* Soil Moisture:
====2004====
* Soil temps: Good
* Soil Moisture:
====2005====
* Soil temps: Dropped from Record?
* Soil Moisture:

====MRC Probe====
* 2012 Summer jacked about 8 cm (from photo, Lumix TSC4)
* 2013: Did not visit
* 2014 7/22/14: Jacked out 16 cm (from photo)
* 2016 August: 37 cm (Cohen fieldbook)

@ Installation, 7/24/2011:
T1 7.6 cmbgs
T2 12.7 cm bgs
T3 22.9 cm bgs
T4 37.7 cm bgs
T5 52.5 cm bgs
T6 67.3 cm bgs
T7 82.7 cm bgs
T8 108 cm bgs

C1-Grid Notes

2016-11-16T05:43:32Z

69.161.4.229: /* Soils Data */

===Met Data===
* I'm not going to re-review 1999-2008
====2009====
* Air Temperature: All Good
* Rain:
** Start: 5/20/2009 9:00
** End: 10/13/2009 6:00
* RH: All Good
* WS/WD: All Good
====2010====
* Air Temperature: All Good
* Rain:
** Start: 5/12/2010 7:00
** End: 8/8/2010 13:00 (start of legit data gap)
* RH: All Good
* WS/WD: All Good
====2011====
* Air Temperature: All Good
** Dead Battery: 1/1/2011 to 2/10/2011 13:00
* Rain:
** Start: 5/12/2011 9:00
** End: 10/15/2011 21:00
* RH: All Good
** Dead Battery: 1/1/2011 to 2/10/2011 13:00
* WS/WD: All Good
====2012====
* Air Temperature: All Good
* Rain:
** Start: 5/15/2012 11:00
** End: 10/7/2012 9:00
* RH: All Good
* WS/WD: All Good
====2013====
* Air Temperature: All Good
* Rain:
** Start: 5/24/2013 8:00
** End: 10/29/2013 7:00
* RH: All Good
* WS/WD: All Good
====2014====
* Air Temperature: All Good
* Rain:
** Start: 5/7/2014 12:00
** End: 10/4/2014 1:00
* RH: All Good
* WS/WD: All Good
====2015====
* Air Temperature: All Good
* Rain:
** Start: 5/6/2015 9:00
** End: 9/25/2015 11:00
* RH: All Good
* WS/WD: All Good

===Soils Data===
I think this is the thermistor model here (found an old spreadsheet from Andy):
D:\projects_usb_drive\UAF\From Crane\archived files from past workers\ANDYS\STUFF\Thermistortemplate.xls
=IF(A3>60,6999,IF(A3<3,6999,IF(A3=0,6999,IF(A3=-6999,-6999,IF(A3=6999,6999,1/(0.00128+0.000237*(LN(A3*1000))+0.0000000906*((LN(A3*1000))^3))-273.15)))))

Here is the info that matches that from my 'thermistor_information.xls'
Alpha 13A5001-DOC 5/29/1998 25694 275 ±0.1 C 1.28E-03 2.37E-04 9.06E-08
August 2008:
YSI 44004 Jessie 06D2432 -- ±0.2 C 0.001465769 0.000238619 0.000000100 -13.4 From Jessie installed on SP in August 2008

* I'll just note here, I wanted to extract old soil moisture data and using datapro on ''raw/c1-grid/RAW/CR10X_final_storage_1.dat'' the date index was totally screwed up. Well, the starting data was correct but the y values were out of order in a big way. I ended up grepping the 228 array into another text file and used that as the source for datapro. Messed me up for a good day or so.
 
* Ended up reprocessing the soil moisture data, the data archive was a mix of garbage and who knows what.
====1999====
* Soil temps: Good
* Soil Moisture: OK
====2000====
* Soil temps: Good
* Soil Moisture: OK
====2001====
* Soil temps: Good
* Soil Moisture: OK
====2002====
* Soil temps: Good
* Soil Moisture: SM5 has turned into a soil temperature value.
====2003====
* Soil temps: Couple blips in April, added to QA sheets.
* Soil Moisture:
====2004====
* Soil temps: Good
* Soil Moisture:
====2005====
* Soil temps: Dropped from Record?
* Soil Moisture:

====MRC Probe====
* 2012 Summer jacked about 8 cm (from photo, Lumix TSC4)
* 2013: Did not visit
* 2014 7/22/14: Jacked out 16 cm (from photo)
* 2016 August: 37 cm (Cohen fieldbook)

@ Installation, 7/24/2011:
T1 7.6 cmbgs
T2 12.7 cm bgs
T3 22.9 cm bgs
T4 37.7 cm bgs
T5 52.5 cm bgs
T6 67.3 cm bgs
T7 82.7 cm bgs
T8 108 cm bgs

C1-Grid Notes

2016-11-16T04:36:57Z

69.161.4.229: /* Soils Data */

C1-Grid Notes

2016-11-16T02:48:05Z

69.161.4.229: /* Soils Data */

C1-Grid Notes

2016-11-15T20:37:15Z

69.161.4.229: /* MRC Probe */

C1-Grid Notes

2016-11-15T20:10:32Z

69.161.4.229: /* Soils Data */

C1-Grid Notes

2016-11-15T18:30:34Z

69.161.4.229:

C1-Grid Notes

2016-11-14T18:56:23Z

69.161.4.229: Created page with "===Met Data=== * I'm not going to re-review 1999-2008 ====2009==== * Air Temperature: All Good * Rain: ** Start: 5/20/2009 9:00 ** End: 10/13/2009 6:00 * RH: All Good * WS/WD:..."

Seward Peninsula Sites

2016-11-14T07:44:56Z

69.161.4.229:

==== Trip Itineraries etc.====
* [[2007 September Nome Trip]]
* [[2008 June Nome Trip]]
* [[2008 August Nome Trip]]
* [[2009 August Nome Trip]]
* [[2009 September Nome Trip]]
* [[2010 June Nome Trip]]
* [[2010 August Nome Trip]]
* [[2010 November Nome Trip]]
* [[2011 August Nome Trip]]
* [[2012 August Nome Trip]]
* [[2014 July Nome Trip]]
* [[2015 July Nome Trip]]
* [[2015 September Nome Trip]]
* [[2016 June Nome Trip]]
* [[2016 April Nome Trip]]
* [[2016 Summer Nome Trip]]
* [[2016 NGEE August Nome Trip]]
* [[2016 NGEE September Nome Trip]]
* [[2017 NGEE June Nome Trip]]
* [[ General Site Checklist]]
* [[Storage Inventory]]
* Kougarok Storage: Combo changed in 2015. Ask Bob Busey for access if needed.
* [[Base Callbook]]

==== Data ====
Unofficial data location on ngeedata: 
/home/bbusey/working_files/sp_datapro/raw/ 
Internal wiki data: 
https://wiki.alaska.edu/display/iarcsp/

* [[C1-Grid Data Downloads]]
* [[C2-Blueberry Data Downloads]]
* [[C3-Guy Rowe Data Downloads]]
* [[K1 Burn Data Downloads]]
* [[K2 Met Data Downloads]]
* [[K3 Mauze Data Downloads]]
* [[K1 Snow Thermocouples Data Downloads]]
* [[K2 Snow Thermocouples Data Downloads]]
* [[Anvil Mountain Data Downloads]]
* [[Skookum Pass Data Downloads]]
* [[Kigluaiks Repeater Data Downloads]]
* [[Kigluaiks Repeater 2 Data Downloads]]
* [[Online Archive Fixes]]

==== Instrumentation Serial Numbers ====
* [[C1-Grid Instrumentation History]]
* [[C2-Blueberry Instrumentation History]]
* [[C3-Guy Rowe Instrumentation History]]
* [[K1 Burn Instrumentation History]]
* [[K2 Met Instrumentation History]]
* [[K3 Mauze Instrumentation History]]
* [[K1 Snow Thermocouples Instrumentation History]]
* [[K2 Snow Thermocouples Instrumentation History]]
* [[Anvil Mountain Instrumentation History]]
* [[Skookum Pass Instrumentation History]]
* [[Kigluaiks Repeater Instrumentation History]]
* [[Kigluaiks Repeater 2 Instrumentation History]]

====Data QA Notes====
* [[C1-Grid Notes]]
* [[C2-Blueberry Notes]]
* [[C3-Guy Rowe Notes]]
* [[K1 Burn Notes]]
* [[K2 Met Notes]]
* [[K3 Mauze Notes]]
* [[Anvil Mountain NOtes]]
* [[Skookum Pass Notes]]
* [[Kigluaiks Repeater Notes]]
* [[Kigluaiks Repeater 2 Notes]]

====Sites Background====
As carry over from the ATLAS and HARC projects WERC maintains a network of telemetered data stations on the Seward Peninsula. Archived data is currently found on the [http://www.uaf.edu/water/project/atlas/atlas.html UAF WERC site]

Principal Data site, Council Area:
[[C1 Grid]] (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U27]), active
[[C2 Blueberry]], active
C3 Guy Rowe, inactive

Principal Data site, Kougarok Area:
[[K1 Burn]] site, active
[[K2 Met]], active
[[K3 Mauze]] Gulch (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U28]), active

We use radio telemetry to communicate with our stations out on the peninsula. So, there are several radio repeaters in that list, too. They are:
[[Anvil]] Mountain (near Nome)
[[Skookum]] Pass (near Council)
[[Kigluaiks]] (in Kigluaik Mountains across the valley to the south of Kougarok)

In addition there are several other sites:
[[Snake]] River (outside Nome on Nome-Teller Highway)
[[Kuzitrin]] River (60+ miles up the Kougarok Road from Nome)
[[Niagara]] Creek (~1 mile from K2 Met)

See most of them on a map here:
[http://data.ine.uaf.edu/seward/ Site Map]

Radios turned off for C1-grid, C2-blueberry, Skookum Pass, Anvil Mountain on 11/4/2009

====Project Background====
Originally there were three Council sites, C1, C2, and C3. Each was selected to be in a different type of landscape but C3 kept getting attacked by bears so we eventually removed it. C1 is called C1-Grid because it is located in the valley bottom near Council along the perimeter of a CALM 1 km x 1 km active layer grid. So, the meteorological station was set up to complement the field data collected on the CALM grid. In terms of importance to the original project it was a secondary station so the data elements collected there are basically: Air Temperature / Relative Humidity (1m & 3m), Wind Speed (1m & 3m) & Wind Direction (3m), Net Radiation (in summer), precipitation (in summer), snow depth, soil moisture (vertically profiled), and soil temperature (vertically profiled). The station is also a shorter 3 meter tower.

===Council Sites===
The blueberry site is located on Blueberry Hill to the SW of Council. It is the primary site for the Council area so has more sensors and the tower is a 10 meter high setup. In addition to the sensors mentioned for C1 there is also up facing shortwave & longwave radiation, and down facing shortwave & longwave radiation sensors. Wind speed & AT/RH (as a single sensor) are also measured at more heights than at C1 (1m, 3m & 10m). In 2007 I made some changes to the station. In the past longwave radiation was measured using a battery. However, it was a lead-mercury battery, which is considered hazardous and has been discontinued so I re-did the wiring. This will be reflected in the data as a couple extra data columns that appear last year. Blueberry is across the Niukluk river (I think that's the river, not 100%) and is sometimes unreachable in fall. So, we weren't able to get across the river to remove the net radiometer in Fall 2007. Data for it shouldn't be trusted much past October 1st though because once the snow falls or frost accumulates on the domes that protect the radiation sensor.

On a similar note, the precipitation gages are out year-round but we only record summer rainfall. So, you should mark with '7777' the winter time of year. For measured precipitation I don't generally count it until the air temperature is consistently above freezing. It's kind of subjective on the shoulder seasons figuring out what is snow vs what is rainfall and if you don't feel comfortable with that, I'm happy to look over the data and define select which points to include and which to exclude. It may sound like we're doing something funky but we aren't. The tipping bucket rain gage is only designed to measure liquid precipitation. So, when snow accumulates in the bucket no measurements are recorded until the bucket temperature gets above freezing to melt the snow in the bucket. Since those gages are just used in the summer then we exclude all solid precip that may melt in and be counted at the data logger.

===Kougarok Sites===
K1-burn is another secondary site. It is located on an area burned by an old tundra fire up on a low sloping bench above the Kougarok river. The vegetation in this area is dry tussock and grasses. Much of the organic soil layer burnt off in the fire so recovery has been slow. This site gets visited fairly frequently by animals so you'll see most of the soil moisture sensors have had their cables eaten, as has also happened to the soil temperature sensors. This site is quite similar to C1 that I talked about earlier in terms of sensor setup. In the fall of 2007 we added a snow depth sensor to the site so that will be a new thing. I'm not sure about its reporting though.

K2-met is the 10 meter tower for the Kougarok area. It is located on very wet tundra 300 meters or so up slope from the Kougarok river, so down in the valley. It is a full site similar to C2 although it also has a snow sensor installed. A broken, older sensor was replaced with a newer one in fall 2007.

K3 mauze is located at the perimeter of the Kougarok area CALM grid. It is another 3 meter tower. This site has been hammered by the bears in recent years so some sensors are in and out in terms of their quality.

===Nome Base===
The base radio and moxa server are located at the UAF Northwest campus in Nome.

[[File:NomeAntenna.jpg|400px]]
[[File:NomeRadio.jpg|400px]]

===Discussion of the Station Data===
So, that's a short bit of background on the stations. The data on the CD I gave you is predominantly archived data or data that has been through some level of processing. You can find the up to the day data here: 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/ 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-blueberry 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-burn 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-met 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-kmauze 

Those are all quite large at this point text files. Let me know if you find any gaps in the data. Sometimes there is trouble with the back end connection between the UAF webserver and the computer that retrieves this data over the radio network from Nome.

So, when you look at the data there it will probably look like a bunch of not real helpful numbers. Plus you'll see an Array ID at the start of the line which seems to alternate each hour. That is because the meteorological data arrays are mixed together withe subsurface data arrays. It's kind of a by product of the data logger we use. The design for them was done in the early 90s or late 80s so the electronic data storage architecture is robust but perhaps a bit primitive compared to the computers of today. So, to split out all the data I should have included on that cd a directory called 'batch' or 'bin'. If you look in there you should see some .bat files which are dos batch files. I don't know if you're using linux or windows, if you're using linux then pop open .bat file and you'll see 'grep .........' you should be able to change the directory paths there (I can help if you have questions about this) but these little batch files basically read through the data above (e.g. atlas-blueberry) and pull out all the met data arrays and then all the soil data array lines and dump them into different files. So, when you mention seeing different data files in the CD I gave you, the data is in various steps of processing. Oh, also, if you are using a windows computer (as I am for the data processing) then you should have cygwin installed. It's a program that gives you many of the free utilities that come with linux/unix/debian (most helpfully for us today, grep). You can download it here if you need:
http://www.cygwin.com

So, if you're using windows, just modify the batch files. I like to use this program:
[http://www.context.cx Context]
for viewing text files. I find it much more useful than notepad, but if you have a favorite text editing program, feel free to use it. Anyway, if you're using windows you'll need to modify the batch file to grep out the 2007 data I think. The general format for the grep command is:
grep [text string to search for] [input data file] > or >> [output data file]
c:\cygwin\bin\grep 228,3018,2007 "c:\data\atlas\c1-grid\atlas-c1-grid.txt" > "c:\data\atlas\c1-grid\c1-grid_228-2007-soil.csv"
You'll see both '>' and '>>' in the batch files. '>' means put all the data line matches into the output file and create a new file each time. '>>' means put all the data line matches into the output file and append them to the end of the file if it already exists.

So, you'll need to change the paths (e.g. 'c:\data\'), to whatever you're using. I like
c:\data\project\station_name from windows or /home/bob/data/project/station_name from linux but totally your call on how you organize. From linux the line above would like more like this:
grep 228,3018,2007 "/home/bob/data/atlas/c1-grid/atlas-c1-grid.txt" > "/home/bob/data/atlas/c1-grid/c1-grid_228-2007-soil.csv"

So, when you see stuff like this:
CD/atlas/C1-grid/c1-grid_123-2006-met.csv
CD/atlas/C1-grid/c1-grid_123-2007-met.csv
CD/atlas/C1-grid/c1-grid_228-2004-soil.csv
etc. etc.
That is output from the grep program. the 123/228 references the array ID. So, that file contains met data from 2007 from this file:
[http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid C1 Data File]
Here's a sample line:
123,3018,2005,1,100,-0.47,-0.176,97.7,96.1,3.639,162.7,4.414,92.7,0,-2.046,12.35

I see for some flies the '123' has been replaced by the regular date & time. But that's probably the only change in the file. I would have done that if I was just checking things out.

So, moving forwards a bit, my rough method for processing the data is this:
# download data off website
# use GREP to separate out by data line type and year using the batch file for dos.. or if you use linux then you can actually edit the .bat files to be the correct path (like I mentioned above) and then from the command line type this: chmod u+x c1-grid.bat this will allow you to just type './c1-grid.bat' just as I have been doing but you'll do it from linux rather than windows.
# open in excel or use a program I wrote to process the data (more on this below)
# view the data with a data visualization program I used Origin but Excel is semi okay (origin is nice because you can quickly change the time scales) or if you have another program you really like like matlab that's fine, too.
# delete bad data that is apparent through using the visualization program (replace with '6999')
# have someone else do a final check (like me probably)
# post data to the internet

'process the data' is pretty nebulous. I have a program I wrote which converts thermistor resistances (soil temperature data) from resistance to temperature and then formats the data as it should look on the web. That's the main reason you see so many similarly named files. Some are from step 1, some are from step 2 etc. So, that is probably a bit confusing. But for you, probably using a spreadsheet program like excel or whatever you're most comfortable with is better. You can find header information for what is in each column of the data file. The other main bit of 'processing' that happens is to correct the net radiation data for wind. I can dig that equation up for you if you can't find it in those spreadsheets.
So, also along the process the data lines, some things you'll want to look out for:
# missing lines of data... make a note and check with me if you find any
# repeated lines of data... more likely to find due to the rsyncing process (the way the uaf website talks to the server downloading data from the radio network).
There are a couple ways of converting julian dates into regular. You could use these programs I've written for visual basic:
http://werc.engr.uaf.edu/mediawiki/index.php/Excel_Macros
or, another way is this:
a) type into a cell '12/31/06'
b) change the format of the cell to number.
c) if your data is in the format column 1 is the date you'd like, column 2 is the year, column 3 is the day, column 4 is the hour then in column 1 you can print (using data for 2007) '= 39082 + b1 + c1/2400' then format this column as a date & time and you'll be in business. Do some simple boolean stuff to check for gaps of data or, just create another column that has as the first cell '1/1/07 1:00' then the second row in the column '=a1 + 1/24' then look to see if the calculated date (mentioned first) ever varies from the second method. If it doesn't you're golden. If they do vary then you can track down where the problem is. In general, if there is multiple lines of the same data (or so it looks like) I will delete the first instance and keep the last. The reason for this is related to the radio telemetry. Some of the connections are quite weak so we don't always get the data on an hourly basis. Sometimes we'll get half the line so we'll need to get things resent later (with regard to the radio network and data retrieval from the data loggers).

I guess another note on the array IDs. The met data and the radiation data are contained in the same array ID line. The soil temperature and soil moisture data (most of which has broken sensors) is in a separate array ID. That is the 123, 127 etc. that you wonder about below.

Well, I've written a book, maybe I will post this to our wiki and we can go back and forth on there, too. Totally let me know if you have any questions. I'll be in town through Friday (just learned) so I should be able to help more in the short term than the medium term. thanks

===== Part 2 =====
Here's a bit about the radiation. I will get you more calibration values once we're back from the June trip the week of 6/16.

So, in the data files you've looked at off the internet is the output voltage in millivolts as recorded by the data logger. To get from milliVolt output from the data logger to Net Radiation in Watts / m^2 you'll need to do a couple things:
1) correct for wind speed.
2) apply the sensor specific calibration factor (positive or negative depending on sensor output)

Via this manual (pdf page 5 document page 3):
Net Radiometers -- REBS Q-7.1 (Radiation Energy Balance Systems)
http://www.campbellsci.com/documents/manuals/q-7-1.pdf

Are these wind speed correction equations (where wind speed in in meters per second as we have at our stations):

Positive Wind Speed Correction Factor:
Wind_CF_Pos = 1 + (0.066 * 0.2 * 1m_Wind_Speed) / (0.066 + (0.2 * 1m_Wind_Speed) )

Negative Wind Speed Correction Factor:
Wind_CF_Neg = (0.00174 * 1m_Wind_Speed) + 0.99755

So, if Net Radiation (in mVolts as recorded by the data logger) is positive use this equation:
Corrected_Net_Rad = Wind_CF_Pos * Uncor_Net_Rad * Pos_Calib

If Net Radiation (in mVolts as recorded by the data logger) is negative use this equation:
Corrected_Net_Rad = Wind_CF_Neg * Uncor_Net_Rad * Neg_Calib

Here's the equation for C2 as it looks for me in Excel:
=IF(ABS([NetRadiometer Cell])<> 6999,IF(L4087>0,9.15*[Net Radiometer Cell]*(1+(0.066*0.2*[1 meter Wind Speed])/(0.066+0.2*[1 meter Wind Speed])),11.13*L4087*(0.00174*[1 meter Wind Speed]+0.99755)),7777)

So, given all that you need one more thing, the calibration values for positive and negative calibrations. Here's a first batch and I'll get you some more after we come back from out trip in a couple weeks.
C1 Grid:
Positive Calibration: 9.18
Negative Calibration: 11.31

C2 Blueberry:
Positive Calibration: 9.15
Negative Calibration: 11.13

K1 Burn:
[Need to track down yet, after 2008 June visit]
K2 Met:
[Need to track down yet, after 2008 June visit]
K3 Mauze Gulch: Q99012
Positive Calibration: 8.97
Negative Calibration: 10.90

The raw data input column information you should find in the excel files or here, too:

http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/C1Grid/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/Blueberry/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-burn/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-met/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/kmauze/current-raw.html

The output column information (how everything should be laid out) is here:

http://www.uaf.edu/water/projects/atlas/council/c1/data/met/c1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/soil/c1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/rad/c1_rad_2005.csv

http://www.uaf.edu/water/projects/atlas/council/c2/data/met/c2_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/soil/c2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/rad/c2_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/met/k1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/soil/k1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/rad/k1_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/met/k2_met_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/soil/k2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/rad/k2_rad_2004.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/met/Mauze_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/soil/Mauze_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/rad/Mauze_rad_2006.csv

===Datapro processing notes===
On 5/23/16 we wanted to update the datapro processed output files to use the directly downloaded logger files instead of the ones from the radio network. Here is what we did, using C1 as an example:

*A record of all directly downloaded data files is found here http://ngeedata.iarc.uaf.edu/wiki/C1-Grid_Data_Downloads
*Config files such as Key files to run data pro are here: ngeedata /home/bbusey/working_files/sp_datapro/c1-grid/config/
*Copied raw directly downloaded data files copied from owncloud folder(owncloud/Modern Seward Peninsula/ Seward_Data/raw_data_archive) to /home/bbusey/working_files/sp_datapro/raw/c1-grid/raw
*Deleted some data in each output file in /home/bbusey/working_files/sp_datapro/c1-grid/outputs so that data ended on 12/31/12 or before
**note 1: in this folder are some shortcut links called symlinks for the website).
**note 2: Datapro looks to see what the last line of data is so it doesn’t append if the data is already in the file
*Copied the bash script from /home/bbusey/bin/process_sp.sh too /home/bbusey/process_direct_downloads_bin/process_c1grid.sh so we could edit it and run it on the directly downloaded data instead of the radio downloads
**about bash: when you use ssh to bring up a terminal window, that particular instance is bash terminal or bash shell. so, you can use a bash script to string together a bunch of bash commands. in this case here we will use a bash script to process each .dat direct download using datapro
**this line specifies that we are running datapro version 3, the key file for array 27, and the direct data file we downloaded from C1 in 2014
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_array_127_key.txt --alt_data_files=$ROOT_DIR_SP/raw/c1-grid/RAW/c1_grid_2014_07_22.dat
**then we run that line again with the next data file, or next key file, etc
**we run the bash script for every new line and put the old line in a comments
*Now the files in the /home/bbusey/working_files/sp_datapro/c1-grid/outputs/ folder should then be updated to the end of the last data file
**you can check this by outputting the last two lines of each .csv file in the folder by typing
tail -n2 /home/bbusey/working_files/sp_datapro/c1-grid/outputs/*.csv
* the script we paused that runs hourly on the radio downloaded files looks like
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_key.txt

2017 NGEE June Nome Trip

2016-11-14T05:30:25Z

69.161.4.229:

* @ [[Kougraok M64]]
** if we get permission then site at the top perhaps?
** If no permission then maybe a subset of instruments adjacent to the GIPL Pilgrim site and or hold off until September / the spring.
*** soil pit
**** soil moisture @ 5, 10, 20, 30, 40 cm bgs
**** soil thermal conductivity @ 10, 20, 30, 40 cm bgs
**** soil temperature @ 10, 20, 30, 40 cm bgs
**** soil heat flux near base of active layer & plant / soil boundary
*** 1.5m soil temperature profile
*** snow depth @ tripod
*** snow depth away from tripod
*** interval camera
*** 4 component radiation
*** Spectral response sensors: NDVI & PRI up looking and two vegetation communities
*** 1.5m Air temperature, Relative Humidity, & Dew Point Temperature
*** Wind speed and direction at 3m
*** summer precipitation
* @ Teller Bottom
** send one more program to fix 5cm bgs soil moisture out of range array error
* @ C1 Grid
** We have a CNR4 for this site I believe.
** fix MRC portion of program last change we screwed something up and sent the 2011 program with a but rather than the 2012 version.
* @ C2 Blueberry
** Replace the LW Transmitted cable, will need to bring the whole package from Fairbanks
** 10meter AT/RH has not worked since 2011

Current work flow for setting up datapro to run on a new site with integration into the diagnostic web page

2016-11-08T01:47:03Z

69.161.4.229:

===Initial Stuff===

* download a copy of the data onto your local machine

* either by hand or using an automated tool set up the parameter and key files.
* transfer the configuration files to the production machine someplace like:
** /var/site/$STATION_GROUP/$STATION_NAME/config
* create a bash script to process the data
** change the battery variable to "battery" rather than "Battery_Avg" etc
** change the panel temperature to "paneltemp" rather than whatever it is too.
** both of these require file name 'battery.csv' and 'paneltemp.csv' for their information to be loaded onto the diagnostic web page.
** for these variable name changes, find them in the parameter csv file.
* test the bash script
* once it works add a reference to it to the primary site processing cron script

===Adding the new site to the diagnostic page===
* on the main diagnostic web directory such as /var/www/html/data/ make directory $STATION_GROUP/$STATION_NAME
mkdir /var/www/html/data/$STATION_GROUP
mkdir /var/www/html/data/$STATION_GROUP/$STATION_NAME
* copy /var/www/html/data/viz_template/full.html to /var/www/html/data/ $STATION_GROUP/$STATION_NAME
cp /var/www/html/data/viz_template/full.html /var/www/html/data/$STATION_GROUP/$STATION_NAME/
cp /var/www/html/data/viz_template/recent.html /var/www/html/data/$STATION_GROUP/$STATION_NAME/
* edit /var/www/html/data/diag.json and add the information for this new site.
* create symlinks for the output & web directories from the processed data into the web directory
* cd to the web/ and /outputs directories and dump the folder contents into a file called directory.txt
cd /var/www/html/data/$STATION_GROUP/$STATION_NAME/web
ls *.csv > directory.txt
cd /var/www/html/data/$STATION_GROUP/$STATION_NAME/outputs
ls *.csv >directory.txt
* edit those directory.txt... re

... I am putting together another git project to drive all of this.
===Extra things to remember and document===
* the parameter generation program misspells ignore
* once the data has been processed need to cd to both we

Current work flow for setting up datapro to run on a new site with integration into the diagnostic web page

2016-11-07T23:07:42Z

69.161.4.229: /* Adding the new site to the diagnostic page */

Current work flow for setting up datapro to run on a new site with integration into the diagnostic web page

2016-11-07T22:36:15Z

69.161.4.229: Created page with "===Initial Stuff=== * download a copy of the data onto your local machine * either by hand or using an automated tool set up the parameter and key files. * transfer the c..."

INE Diagnostic Page

2016-11-07T22:07:35Z

69.161.4.229:

===Source Data===
http://snobold.ine.uaf.edu/loggernet-data/ 

===Status Page===
* http://werc.ine.uaf.edu/data/

=== New Site Set up Process ===
* [[Current work flow for setting up datapro to run on a new site with integration into the diagnostic web page]]

===10 meter towers, met:===
* Imnavait Met-TEON/arp (radio)
* Upper Kuparuk Met-TEON/arp (radio)
* Franklin Bluffs Met -TEON/arp (radio)

===3 m towers, met:===
* Putuligayuk Met-TEON/arp (Cellular)
* Roche Moutonnee Met-TEON/arp (iridium)
* GSM3 Murphy Dome East Met (Goldstream/Liljedahl) (radio)
* GSM4 Murphy Dome West Met (Goldstream/Liljedahl) (radio)
* Green Cabin Lake Met (TEON/Arp) radio CR10x
* Sag R Bridge SR50 (BP/Toniolo)
* ASM1 Accomplishment Cr Met (Alyeska/Toniolo)
* ASM2 Saviyukviayak Met (Alyeska/Toniolo)
* ASM3 Ivishak Spring Met (Alyeska/Toniolo)
* DBM4 Sag-Ivishak Met (ADOT/2016breakup/Toniolo)
* DSM1 Echooka Met (ADOT/2016breakup/Toniolo)
* Jarvis Cr Met (Jarvis/Liljedahl)
* Jarvis Glacier Met (Jarvis/Liljedahl)
* Lower Jarvis Met (Jarvis/Liljedahl)
* Umiat_Airport (BLM/Toniolo)

===3 m towers, met/hydro:===
* GSM1 Dougnut Lake (Goldstream/LIljedahl) (Radio)
* GSM2 Goldstream pond (Goldstream/Liljedahl) (Radio)
* Sag R East Bank DSS5 (ADOT breakup 2016)
* DSS1 Sag R MP414 (ADOT/Sed) radio
* DSS2 Sag R below Ivishak (ADOT/Sed)
* DSS3 Sag R Happy Valley (ADOT/Sed)
* DSS4 Sag R MP318 (ADOT/Sed)
* ASS1 Sag R Alyeska (Alyeska/Toniolo)

===Hydro:===
* Roche Moutonnee Cr TEON/Arp (Iridium)
* Imnavait Cr Weir TEON/Arp (radio)
* Upper Kuparuk R TEON/Arp (radio)
* Putuligayuk R TEON/Arp (cellular)
* GSS1 (Goldstream/Liljedahl) (radio) CR10x-PB
* Jarvis Bridge (Jarvis/Liljedahl)
* Jarvis Canyon (Jarvis/Liljedahl)
* Seabee Cr Gauge (BLM/Toniolo)

===Misc./Other:===
* Beaver Slide Soil (unknown PI)

===Repeaters:===
* Franklin Bluffs (ADOT/Sed)
* Imnavait Mountain TEON/Arp
* Slope Mountain TEON/Arp
* Ice Cut (ADOT/Sed)
* Upper Kuparuk Ridge TEON/Arp
* Sagwon TEON/Arp
* Jarvis Repeater (Jarvis/Liljedahl)

===Scratch pad==
* http://werc.ine.uaf.edu/data/viz_template/
* Need to update this code:
** http://ngeedata.iarc.uaf.edu/data/barrow/intense_A/index.html
** Remove english units and update the dygraph to behave more like I have for the blueberry visualizer (code merge of some sort).
* Need to update javascript for main plot on the data page so it sits properly again.

Initial Debian Server Setup

2016-11-04T03:13:23Z

69.161.4.229: /* Other packages */

[[IARC Server List]]
==Step by step (roughly) procedure for getting a Debian-ARM server up and running for remote deployment as a serial radio base or a number of other things==
Been meaning to put this together for a while.
===Download media===
A lot of these ARM based systems have a microSD or SD card image that can be flashed from an online source. I like to use the most recent stable version of Debian if it's available and also the minimum size image so that less extra cruft is installed (don't need a window manager etc) 
The RaspberryPis work well, too but have less horsepower under the hood: 
https://www.raspberrypi.org/downloads/raspbian/ 
This is more powerful but doesn't come with a vanilla kernel (this means long term updating is harder): 
http://wiki.solid-run.com/doku.php?id=products:imx6:overview:flashsdcard

We also have a few units (the DreamPlug and the GuruPlug) from GlobalScale but the power supplies have become flaky over time and I think the cubox-i is current favorite. 
https://www.globalscaletechnologies.com/ 

I guess in addition, also using intel based NUC which is slightly larger, maybe more powerful but the cpu instruction set is x86 rather than ARM. For Debian, that means nothing but it should also mean we can run loggernet for linux further afield. (this is on my later in the winter list).

===First step===
#Download the current image of the OS available online to your computer
## http://wiki.solid-run.com/doku.php?id=products:imx6:software:os:debian 
## https://www.raspberrypi.org/downloads/raspbian/ 
# Flash it to the appropriate media (microSD or SD card typically). Something like this:
## sudo dd if=~/Downloads/ignition.img of=/dev/rdisk2 bs=4096 ''' (cubox-i example)'''
## sudo dd if=2015-11-21-raspbian-jessie-lite.img of=/dev/rdisk2 bs=4096 '''(Raspberry Pi example)'''
# Look up online the default user / password to get into the system once it is up and running, too.
## A simple search for something like "''Raspberry pi default user name and password''" should get you close.
# Once you're up and running, log in as the default user and then let's do some set up in an interactive sudo session:
<pre> sudo -i</pre>

===For raspberry pi, change keyboard layout to US===
Default is UK English layout... before you change the passwords you should get a US layout keyboard. Here is a good resource 
https://wiki.debian.org/Keyboard 
<pre>dpkg-reconfigure keyboard-configuration</pre>
<pre> service keyboard-setup restart</pre>

=== Create user accounts===
(junk name here not what you might see on our systems)
<pre> adduser scientist</pre>

=== give the user extended permissions===
https://wiki.debian.org/sudo
<pre>adduser scientist sudo</pre>

===Tighten up remote ssh access===
Been having some issues with port scanning and automated log in attempts from all over the world. You can initially lock things down by disabling the default account from ssh login (after creating your first other user in the previous step). First up, edit the ssh server configuration. The file is found in ''/etc/ssh/sshd_config'':
<pre> editor /etc/ssh/sshd_config</pre>
Add these lines to that file (or verify that they are there / double check that you aren't duplicating and creating confusion for the daemon):
<pre>
## Disable root login:
PermitRootLogin no
##specify which users can log in over ssh:
AllowUsers scientist
</pre>
With that out of the way, restart the ssh server and you have taken a step towards better securing the system:
<pre>service ssh restart</pre>
===Update Firewall:===
Firewall is complicated. I'm not 100% I have this correct but it's quite a bit more secure than the defaults and such. It's worth its own entry though:
* [[IP Tables Firewall Example]]

===update apt & all packages===
<pre>
apt-get update
apt-get upgrade
</pre>

===Raspberry Pi specific: Resize Partition===
Run this utility as root:
<pre>raspi-config</pre>
One of the options (the first one) is:
<pre>1 Expand Filesystem Ensures that all of the SD card storage is available to the OS </pre>
Select that and the file system will go from the default, like 1.3 GB and expand to consume the whole card. See here for a bit more discussion: 
http://raspberrypi.stackexchange.com/questions/499/how-can-i-resize-my-root-partition

===Install additional packages===
Great, if you've made it this far then you should have a decently secure system, nearly ready for deployment. There are a few more pacakges it is smart to install... kind of depends on what you are doing whether you add all of them or not but there first are a good move.

<pre> apt-get install nmap ntp less imagemagick python mailutils fail2ban ser2net git telnet screen wget curl</pre>
* '''nmap''' is useful for checking out the internet presence of your device. Kind of optional but nice to have installed if you need it at a later date.
** https://packages.debian.org/stable/nmap
* '''ntp''' is the time server, used to get internet time. It should be installed by default but I have found it isn't alway. So, best to be explicit and install it here.
** https://packages.debian.org/jessie/ntp
* '''less''' is a simple command line utility for reading text files. A gain, it should be installed by default but I have run into it not being installed.
** https://packages.debian.org/jessie/less
* '''imagemagick''' is a great command line based image manipulation library. Used by a few different utilities. I think we use it to resize / slice automatically generated plots, and to do some other stuff in Barrow.
** https://packages.debian.org/jessie/imagemagick
* '''python''' common programming language we use extensively and should be added to the system.
* '''mailutils''' is a simple email package. Used by crontab & the security updates system (as well as others I'm forgetting) to give you status information.
* '''fail2ban''' is used to lockdown ssh a bit further than the default ip tables rules. It blocks IP addresses that try to log in too often with a bad user/password. Intended to reduce workload of these simple computers handling garbage so they can focus on serving as a serial base station etc.
* '''ser2net''' pretty solid utility for getting serial port access (or usb to serial access) via sockets. This is the primary thing we use to handle getting data logger information available over the internet. For multipoint radios, there is also a '''ser2nets''' which can allow more than one LoggerNet instance to access the radio network simultaneously. I haven't used ser2nets but have thought about experimenting with it.
* '''git''' we use this for distributing datapro and other utilities
* ''telnet'' useful for testing ser2net's operation.
* ''screen'' useful for testing the usb to serial device
* ''curl'' utility for grabbing web pages / data from servers
* ''wget'' utility for grabbing web pages / data from servers

====Automatically install security updates:====
The last generally mandatory package to add is the Unattended security updates. There are couple packages to install but there are also some configs to modify It's best to just read this page. 
https://wiki.debian.org/UnattendedUpgrades 
But, I can also give you a hint that you'll need to do at least these three things from the command line:
<pre>apt-get install unattended-upgrades apt-listchanges</pre>
<pre>editor /etc/apt/apt.conf.d/50unattended-upgrades</pre>
<pre>editor /etc/apt/apt.conf.d/20auto-upgrades</pre>

===Other packages===
So, those are typically the base to get you up and running. Then, this next set are slightly more specialized. If you are going to do any data processing or use the WERC data retrieval tools these are good to install:
<pre> apt-get install build-essential libdbi-perl libconfig-yaml-perl python-numpy python-tz python-pandas liblockfile-simple-perl libdbi-perl bc </pre>

===check timezone===
Important to confirm we are using UTC:
<pre>sudo dpkg-reconfigure tzdata </pre>

===Next steps===
So, at this point you have a system that has now been tailored to generic deployment. With that out of the way there are some specific things and tools / scripts that are nice to give you a bit more feedback. For example. it needs to tell you its IP address routinely so there are some things to set up in order to make that happen like:
* set up a private key in the ''scientist'' user account so you can automate the transfer
** follow this: http://troy.jdmz.net/rsync/index.html
** '''ssh-keygen -t rsa -b 2048 '''
** then, add the key to the server you want to push too:
*** '''ssh-copy-id scientist@ngeedata.iarc.uaf.edu'''
* set up the crontab so that the automated stuff is configured to happen... do all of this from your user account rather than your root account.
<pre>
mkdir ~/bin
nano ~/bin/main_cron
</pre>
* add something like this to the file main_cron to transmit the IP address every 10 minutes:
<pre>
*/10 * * * * /sbin/ifconfig > /home/scientist/grayling_ip.txt
*/10 * * * * scp -oport=2222 /home/scientist/grayling_ip.txt user@ngeedata.iarc.uaf.edu:/home/user/remote_ips/
</pre>
* add this to the crontab:
<pre> crontab ~/bin/main_cron</pre>
* confirm that it is in the crontab:
<pre> crontab -l</pre>
Great,
* set up a few more bash scripts... running out of steam here I'm just going to list commands for a bit and then annotate later.
* this gets Ross' csv utilities with datapro plus a number of other things installed
<pre>
cd ~/bin/
git clone https://github.com/rwspicer/csv_utilities.git
python ~/bin/csv_utilities/get_ip.py --infile=/home/scientist/extra/grayling_ip.txt --outfile=/home/scientist/grayling_ip
</pre>

* also need to configure ser2net:
<pre>sudo nano /etc/ser2net.conf</pre>
* remove the existing lines at the bottom of the configuration and add this:
** specifying the port as ''7808''
** data type is '''''raw''''' ''(default is telnet)''
** serial to usb device is ''/dev/ttyUSB0''
** speed and data information (this is standard of most of our serial things though the baud rate may change depending on application): ''115200 8DATABITS NONE 1STOPBIT''
** a few additional parameters to close the connection and reset the radio after each use: '' HANGUP_WHEN_DONE RTSCTS LOCAL''
<pre>
# IARC radio
7808:raw:25:/dev/ttyUSB0:115200 8DATABITS NONE 1STOPBIT HANGUP_WHEN_DONE RTSCTS LOCAL
</pre>
* now, reload ser2net daemon with new configuration:
<pre>service ser2net restart</pre>
* next, test your setup:
<pre> telnet localhost 7808</pre>
** then hit enter a bunch and watch for a reply from the data logger. If you see it, awesome. to close the program, hold down the control key and press the close bracket key. This will bring up a new prompt:
<pre><CONTROL><]></pre>
** then:
<pre>close</pre>
* if that works, fantastic. If that doesn't, try connecting directly to the serial port using ''screen'':
<pre>sudo screen /dev/ttyUSB0 9600</pre>
** If that works, awesome! To quit screen then enter:
<pre>
<CONTROL> - <A> (press and hold control, then press A)
<COLON>
<TYPE> quit </TYPE>
</pre>
* if that works, then you have a good serial connection and may just need to tweak the ser2net configuration.
** I just noticed here that <pre>service ser2net restart</pre> didn't properly reload the configuration file. However this did:
<pre>service ser2net stop
service ser2net start</pre>

===Yet to come===
So, at this point you have a solid serial server. There is often more functionality you can add though depending on how things go. For instance, the speed test bash script may need to be installed, there might be webcam

Initial Debian Server Setup

2016-11-04T03:13:03Z

69.161.4.229: /* Install additional packages */

[[IARC Server List]]
==Step by step (roughly) procedure for getting a Debian-ARM server up and running for remote deployment as a serial radio base or a number of other things==
Been meaning to put this together for a while.
===Download media===
A lot of these ARM based systems have a microSD or SD card image that can be flashed from an online source. I like to use the most recent stable version of Debian if it's available and also the minimum size image so that less extra cruft is installed (don't need a window manager etc) 
The RaspberryPis work well, too but have less horsepower under the hood: 
https://www.raspberrypi.org/downloads/raspbian/ 
This is more powerful but doesn't come with a vanilla kernel (this means long term updating is harder): 
http://wiki.solid-run.com/doku.php?id=products:imx6:overview:flashsdcard

We also have a few units (the DreamPlug and the GuruPlug) from GlobalScale but the power supplies have become flaky over time and I think the cubox-i is current favorite. 
https://www.globalscaletechnologies.com/ 

I guess in addition, also using intel based NUC which is slightly larger, maybe more powerful but the cpu instruction set is x86 rather than ARM. For Debian, that means nothing but it should also mean we can run loggernet for linux further afield. (this is on my later in the winter list).

===First step===
#Download the current image of the OS available online to your computer
## http://wiki.solid-run.com/doku.php?id=products:imx6:software:os:debian 
## https://www.raspberrypi.org/downloads/raspbian/ 
# Flash it to the appropriate media (microSD or SD card typically). Something like this:
## sudo dd if=~/Downloads/ignition.img of=/dev/rdisk2 bs=4096 ''' (cubox-i example)'''
## sudo dd if=2015-11-21-raspbian-jessie-lite.img of=/dev/rdisk2 bs=4096 '''(Raspberry Pi example)'''
# Look up online the default user / password to get into the system once it is up and running, too.
## A simple search for something like "''Raspberry pi default user name and password''" should get you close.
# Once you're up and running, log in as the default user and then let's do some set up in an interactive sudo session:
<pre> sudo -i</pre>

===For raspberry pi, change keyboard layout to US===
Default is UK English layout... before you change the passwords you should get a US layout keyboard. Here is a good resource 
https://wiki.debian.org/Keyboard 
<pre>dpkg-reconfigure keyboard-configuration</pre>
<pre> service keyboard-setup restart</pre>

=== Create user accounts===
(junk name here not what you might see on our systems)
<pre> adduser scientist</pre>

=== give the user extended permissions===
https://wiki.debian.org/sudo
<pre>adduser scientist sudo</pre>

===Tighten up remote ssh access===
Been having some issues with port scanning and automated log in attempts from all over the world. You can initially lock things down by disabling the default account from ssh login (after creating your first other user in the previous step). First up, edit the ssh server configuration. The file is found in ''/etc/ssh/sshd_config'':
<pre> editor /etc/ssh/sshd_config</pre>
Add these lines to that file (or verify that they are there / double check that you aren't duplicating and creating confusion for the daemon):
<pre>
## Disable root login:
PermitRootLogin no
##specify which users can log in over ssh:
AllowUsers scientist
</pre>
With that out of the way, restart the ssh server and you have taken a step towards better securing the system:
<pre>service ssh restart</pre>
===Update Firewall:===
Firewall is complicated. I'm not 100% I have this correct but it's quite a bit more secure than the defaults and such. It's worth its own entry though:
* [[IP Tables Firewall Example]]

===update apt & all packages===
<pre>
apt-get update
apt-get upgrade
</pre>

===Raspberry Pi specific: Resize Partition===
Run this utility as root:
<pre>raspi-config</pre>
One of the options (the first one) is:
<pre>1 Expand Filesystem Ensures that all of the SD card storage is available to the OS </pre>
Select that and the file system will go from the default, like 1.3 GB and expand to consume the whole card. See here for a bit more discussion: 
http://raspberrypi.stackexchange.com/questions/499/how-can-i-resize-my-root-partition

===Install additional packages===
Great, if you've made it this far then you should have a decently secure system, nearly ready for deployment. There are a few more pacakges it is smart to install... kind of depends on what you are doing whether you add all of them or not but there first are a good move.

<pre> apt-get install nmap ntp less imagemagick python mailutils fail2ban ser2net git telnet screen wget curl</pre>
* '''nmap''' is useful for checking out the internet presence of your device. Kind of optional but nice to have installed if you need it at a later date.
** https://packages.debian.org/stable/nmap
* '''ntp''' is the time server, used to get internet time. It should be installed by default but I have found it isn't alway. So, best to be explicit and install it here.
** https://packages.debian.org/jessie/ntp
* '''less''' is a simple command line utility for reading text files. A gain, it should be installed by default but I have run into it not being installed.
** https://packages.debian.org/jessie/less
* '''imagemagick''' is a great command line based image manipulation library. Used by a few different utilities. I think we use it to resize / slice automatically generated plots, and to do some other stuff in Barrow.
** https://packages.debian.org/jessie/imagemagick
* '''python''' common programming language we use extensively and should be added to the system.
* '''mailutils''' is a simple email package. Used by crontab & the security updates system (as well as others I'm forgetting) to give you status information.
* '''fail2ban''' is used to lockdown ssh a bit further than the default ip tables rules. It blocks IP addresses that try to log in too often with a bad user/password. Intended to reduce workload of these simple computers handling garbage so they can focus on serving as a serial base station etc.
* '''ser2net''' pretty solid utility for getting serial port access (or usb to serial access) via sockets. This is the primary thing we use to handle getting data logger information available over the internet. For multipoint radios, there is also a '''ser2nets''' which can allow more than one LoggerNet instance to access the radio network simultaneously. I haven't used ser2nets but have thought about experimenting with it.
* '''git''' we use this for distributing datapro and other utilities
* ''telnet'' useful for testing ser2net's operation.
* ''screen'' useful for testing the usb to serial device
* ''curl'' utility for grabbing web pages / data from servers
* ''wget'' utility for grabbing web pages / data from servers

====Automatically install security updates:====
The last generally mandatory package to add is the Unattended security updates. There are couple packages to install but there are also some configs to modify It's best to just read this page. 
https://wiki.debian.org/UnattendedUpgrades 
But, I can also give you a hint that you'll need to do at least these three things from the command line:
<pre>apt-get install unattended-upgrades apt-listchanges</pre>
<pre>editor /etc/apt/apt.conf.d/50unattended-upgrades</pre>
<pre>editor /etc/apt/apt.conf.d/20auto-upgrades</pre>

===Other packages===
So, those are typically the base to get you up and running. Then, this next set are slightly more specialized. If you are going to do any data processing or use the WERC data retrieval tools these are good to install:
<pre> apt-get install build-essential libdbi-perl libconfig-yaml-perl python-numpy python-tz liblockfile-simple-perl libdbi-perl bc </pre>

===check timezone===
Important to confirm we are using UTC:
<pre>sudo dpkg-reconfigure tzdata </pre>

===Next steps===
So, at this point you have a system that has now been tailored to generic deployment. With that out of the way there are some specific things and tools / scripts that are nice to give you a bit more feedback. For example. it needs to tell you its IP address routinely so there are some things to set up in order to make that happen like:
* set up a private key in the ''scientist'' user account so you can automate the transfer
** follow this: http://troy.jdmz.net/rsync/index.html
** '''ssh-keygen -t rsa -b 2048 '''
** then, add the key to the server you want to push too:
*** '''ssh-copy-id scientist@ngeedata.iarc.uaf.edu'''
* set up the crontab so that the automated stuff is configured to happen... do all of this from your user account rather than your root account.
<pre>
mkdir ~/bin
nano ~/bin/main_cron
</pre>
* add something like this to the file main_cron to transmit the IP address every 10 minutes:
<pre>
*/10 * * * * /sbin/ifconfig > /home/scientist/grayling_ip.txt
*/10 * * * * scp -oport=2222 /home/scientist/grayling_ip.txt user@ngeedata.iarc.uaf.edu:/home/user/remote_ips/
</pre>
* add this to the crontab:
<pre> crontab ~/bin/main_cron</pre>
* confirm that it is in the crontab:
<pre> crontab -l</pre>
Great,
* set up a few more bash scripts... running out of steam here I'm just going to list commands for a bit and then annotate later.
* this gets Ross' csv utilities with datapro plus a number of other things installed
<pre>
cd ~/bin/
git clone https://github.com/rwspicer/csv_utilities.git
python ~/bin/csv_utilities/get_ip.py --infile=/home/scientist/extra/grayling_ip.txt --outfile=/home/scientist/grayling_ip
</pre>

* also need to configure ser2net:
<pre>sudo nano /etc/ser2net.conf</pre>
* remove the existing lines at the bottom of the configuration and add this:
** specifying the port as ''7808''
** data type is '''''raw''''' ''(default is telnet)''
** serial to usb device is ''/dev/ttyUSB0''
** speed and data information (this is standard of most of our serial things though the baud rate may change depending on application): ''115200 8DATABITS NONE 1STOPBIT''
** a few additional parameters to close the connection and reset the radio after each use: '' HANGUP_WHEN_DONE RTSCTS LOCAL''
<pre>
# IARC radio
7808:raw:25:/dev/ttyUSB0:115200 8DATABITS NONE 1STOPBIT HANGUP_WHEN_DONE RTSCTS LOCAL
</pre>
* now, reload ser2net daemon with new configuration:
<pre>service ser2net restart</pre>
* next, test your setup:
<pre> telnet localhost 7808</pre>
** then hit enter a bunch and watch for a reply from the data logger. If you see it, awesome. to close the program, hold down the control key and press the close bracket key. This will bring up a new prompt:
<pre><CONTROL><]></pre>
** then:
<pre>close</pre>
* if that works, fantastic. If that doesn't, try connecting directly to the serial port using ''screen'':
<pre>sudo screen /dev/ttyUSB0 9600</pre>
** If that works, awesome! To quit screen then enter:
<pre>
<CONTROL> - <A> (press and hold control, then press A)
<COLON>
<TYPE> quit </TYPE>
</pre>
* if that works, then you have a good serial connection and may just need to tweak the ser2net configuration.
** I just noticed here that <pre>service ser2net restart</pre> didn't properly reload the configuration file. However this did:
<pre>service ser2net stop
service ser2net start</pre>

===Yet to come===
So, at this point you have a solid serial server. There is often more functionality you can add though depending on how things go. For instance, the speed test bash script may need to be installed, there might be webcam

Initial Debian Server Setup

2016-11-04T03:12:38Z

69.161.4.229: /* Install additional packages */

[[IARC Server List]]
==Step by step (roughly) procedure for getting a Debian-ARM server up and running for remote deployment as a serial radio base or a number of other things==
Been meaning to put this together for a while.
===Download media===
A lot of these ARM based systems have a microSD or SD card image that can be flashed from an online source. I like to use the most recent stable version of Debian if it's available and also the minimum size image so that less extra cruft is installed (don't need a window manager etc) 
The RaspberryPis work well, too but have less horsepower under the hood: 
https://www.raspberrypi.org/downloads/raspbian/ 
This is more powerful but doesn't come with a vanilla kernel (this means long term updating is harder): 
http://wiki.solid-run.com/doku.php?id=products:imx6:overview:flashsdcard

We also have a few units (the DreamPlug and the GuruPlug) from GlobalScale but the power supplies have become flaky over time and I think the cubox-i is current favorite. 
https://www.globalscaletechnologies.com/ 

I guess in addition, also using intel based NUC which is slightly larger, maybe more powerful but the cpu instruction set is x86 rather than ARM. For Debian, that means nothing but it should also mean we can run loggernet for linux further afield. (this is on my later in the winter list).

===First step===
#Download the current image of the OS available online to your computer
## http://wiki.solid-run.com/doku.php?id=products:imx6:software:os:debian 
## https://www.raspberrypi.org/downloads/raspbian/ 
# Flash it to the appropriate media (microSD or SD card typically). Something like this:
## sudo dd if=~/Downloads/ignition.img of=/dev/rdisk2 bs=4096 ''' (cubox-i example)'''
## sudo dd if=2015-11-21-raspbian-jessie-lite.img of=/dev/rdisk2 bs=4096 '''(Raspberry Pi example)'''
# Look up online the default user / password to get into the system once it is up and running, too.
## A simple search for something like "''Raspberry pi default user name and password''" should get you close.
# Once you're up and running, log in as the default user and then let's do some set up in an interactive sudo session:
<pre> sudo -i</pre>

===For raspberry pi, change keyboard layout to US===
Default is UK English layout... before you change the passwords you should get a US layout keyboard. Here is a good resource 
https://wiki.debian.org/Keyboard 
<pre>dpkg-reconfigure keyboard-configuration</pre>
<pre> service keyboard-setup restart</pre>

=== Create user accounts===
(junk name here not what you might see on our systems)
<pre> adduser scientist</pre>

=== give the user extended permissions===
https://wiki.debian.org/sudo
<pre>adduser scientist sudo</pre>

===Tighten up remote ssh access===
Been having some issues with port scanning and automated log in attempts from all over the world. You can initially lock things down by disabling the default account from ssh login (after creating your first other user in the previous step). First up, edit the ssh server configuration. The file is found in ''/etc/ssh/sshd_config'':
<pre> editor /etc/ssh/sshd_config</pre>
Add these lines to that file (or verify that they are there / double check that you aren't duplicating and creating confusion for the daemon):
<pre>
## Disable root login:
PermitRootLogin no
##specify which users can log in over ssh:
AllowUsers scientist
</pre>
With that out of the way, restart the ssh server and you have taken a step towards better securing the system:
<pre>service ssh restart</pre>
===Update Firewall:===
Firewall is complicated. I'm not 100% I have this correct but it's quite a bit more secure than the defaults and such. It's worth its own entry though:
* [[IP Tables Firewall Example]]

===update apt & all packages===
<pre>
apt-get update
apt-get upgrade
</pre>

===Raspberry Pi specific: Resize Partition===
Run this utility as root:
<pre>raspi-config</pre>
One of the options (the first one) is:
<pre>1 Expand Filesystem Ensures that all of the SD card storage is available to the OS </pre>
Select that and the file system will go from the default, like 1.3 GB and expand to consume the whole card. See here for a bit more discussion: 
http://raspberrypi.stackexchange.com/questions/499/how-can-i-resize-my-root-partition

===Install additional packages===
Great, if you've made it this far then you should have a decently secure system, nearly ready for deployment. There are a few more pacakges it is smart to install... kind of depends on what you are doing whether you add all of them or not but there first are a good move.

<pre> apt-get install nmap ntp less imagemagick python mailutils fail2ban ser2net git telnet screen wget curl python-pandas</pre>
* '''nmap''' is useful for checking out the internet presence of your device. Kind of optional but nice to have installed if you need it at a later date.
** https://packages.debian.org/stable/nmap
* '''ntp''' is the time server, used to get internet time. It should be installed by default but I have found it isn't alway. So, best to be explicit and install it here.
** https://packages.debian.org/jessie/ntp
* '''less''' is a simple command line utility for reading text files. A gain, it should be installed by default but I have run into it not being installed.
** https://packages.debian.org/jessie/less
* '''imagemagick''' is a great command line based image manipulation library. Used by a few different utilities. I think we use it to resize / slice automatically generated plots, and to do some other stuff in Barrow.
** https://packages.debian.org/jessie/imagemagick
* '''python''' common programming language we use extensively and should be added to the system.
* '''mailutils''' is a simple email package. Used by crontab & the security updates system (as well as others I'm forgetting) to give you status information.
* '''fail2ban''' is used to lockdown ssh a bit further than the default ip tables rules. It blocks IP addresses that try to log in too often with a bad user/password. Intended to reduce workload of these simple computers handling garbage so they can focus on serving as a serial base station etc.
* '''ser2net''' pretty solid utility for getting serial port access (or usb to serial access) via sockets. This is the primary thing we use to handle getting data logger information available over the internet. For multipoint radios, there is also a '''ser2nets''' which can allow more than one LoggerNet instance to access the radio network simultaneously. I haven't used ser2nets but have thought about experimenting with it.
* '''git''' we use this for distributing datapro and other utilities
* ''telnet'' useful for testing ser2net's operation.
* ''screen'' useful for testing the usb to serial device
* ''python-pandas'' useful python package
* ''curl'' utility for grabbing web pages / data from servers
* ''wget'' utility for grabbing web pages / data from servers

====Automatically install security updates:====
The last generally mandatory package to add is the Unattended security updates. There are couple packages to install but there are also some configs to modify It's best to just read this page. 
https://wiki.debian.org/UnattendedUpgrades 
But, I can also give you a hint that you'll need to do at least these three things from the command line:
<pre>apt-get install unattended-upgrades apt-listchanges</pre>
<pre>editor /etc/apt/apt.conf.d/50unattended-upgrades</pre>
<pre>editor /etc/apt/apt.conf.d/20auto-upgrades</pre>

===Other packages===
So, those are typically the base to get you up and running. Then, this next set are slightly more specialized. If you are going to do any data processing or use the WERC data retrieval tools these are good to install:
<pre> apt-get install build-essential libdbi-perl libconfig-yaml-perl python-numpy python-tz liblockfile-simple-perl libdbi-perl bc </pre>

===check timezone===
Important to confirm we are using UTC:
<pre>sudo dpkg-reconfigure tzdata </pre>

===Next steps===
So, at this point you have a system that has now been tailored to generic deployment. With that out of the way there are some specific things and tools / scripts that are nice to give you a bit more feedback. For example. it needs to tell you its IP address routinely so there are some things to set up in order to make that happen like:
* set up a private key in the ''scientist'' user account so you can automate the transfer
** follow this: http://troy.jdmz.net/rsync/index.html
** '''ssh-keygen -t rsa -b 2048 '''
** then, add the key to the server you want to push too:
*** '''ssh-copy-id scientist@ngeedata.iarc.uaf.edu'''
* set up the crontab so that the automated stuff is configured to happen... do all of this from your user account rather than your root account.
<pre>
mkdir ~/bin
nano ~/bin/main_cron
</pre>
* add something like this to the file main_cron to transmit the IP address every 10 minutes:
<pre>
*/10 * * * * /sbin/ifconfig > /home/scientist/grayling_ip.txt
*/10 * * * * scp -oport=2222 /home/scientist/grayling_ip.txt user@ngeedata.iarc.uaf.edu:/home/user/remote_ips/
</pre>
* add this to the crontab:
<pre> crontab ~/bin/main_cron</pre>
* confirm that it is in the crontab:
<pre> crontab -l</pre>
Great,
* set up a few more bash scripts... running out of steam here I'm just going to list commands for a bit and then annotate later.
* this gets Ross' csv utilities with datapro plus a number of other things installed
<pre>
cd ~/bin/
git clone https://github.com/rwspicer/csv_utilities.git
python ~/bin/csv_utilities/get_ip.py --infile=/home/scientist/extra/grayling_ip.txt --outfile=/home/scientist/grayling_ip
</pre>

* also need to configure ser2net:
<pre>sudo nano /etc/ser2net.conf</pre>
* remove the existing lines at the bottom of the configuration and add this:
** specifying the port as ''7808''
** data type is '''''raw''''' ''(default is telnet)''
** serial to usb device is ''/dev/ttyUSB0''
** speed and data information (this is standard of most of our serial things though the baud rate may change depending on application): ''115200 8DATABITS NONE 1STOPBIT''
** a few additional parameters to close the connection and reset the radio after each use: '' HANGUP_WHEN_DONE RTSCTS LOCAL''
<pre>
# IARC radio
7808:raw:25:/dev/ttyUSB0:115200 8DATABITS NONE 1STOPBIT HANGUP_WHEN_DONE RTSCTS LOCAL
</pre>
* now, reload ser2net daemon with new configuration:
<pre>service ser2net restart</pre>
* next, test your setup:
<pre> telnet localhost 7808</pre>
** then hit enter a bunch and watch for a reply from the data logger. If you see it, awesome. to close the program, hold down the control key and press the close bracket key. This will bring up a new prompt:
<pre><CONTROL><]></pre>
** then:
<pre>close</pre>
* if that works, fantastic. If that doesn't, try connecting directly to the serial port using ''screen'':
<pre>sudo screen /dev/ttyUSB0 9600</pre>
** If that works, awesome! To quit screen then enter:
<pre>
<CONTROL> - <A> (press and hold control, then press A)
<COLON>
<TYPE> quit </TYPE>
</pre>
* if that works, then you have a good serial connection and may just need to tweak the ser2net configuration.
** I just noticed here that <pre>service ser2net restart</pre> didn't properly reload the configuration file. However this did:
<pre>service ser2net stop
service ser2net start</pre>

===Yet to come===
So, at this point you have a solid serial server. There is often more functionality you can add though depending on how things go. For instance, the speed test bash script may need to be installed, there might be webcam

2017 NGEE June Nome Trip

2016-11-02T22:50:48Z

69.161.4.229:

* @ [[Kougraok M64]]
** if we get permission then site at the top perhaps?
** If no permission then maybe a subset of instruments adjacent to the GIPL Pilgrim site and or hold off until September / the spring.
*** soil pit
**** soil moisture @ 5, 10, 20, 30, 40 cm bgs
**** soil thermal conductivity @ 10, 20, 30, 40 cm bgs
**** soil temperature @ 10, 20, 30, 40 cm bgs
**** soil heat flux near base of active layer & plant / soil boundary
*** 1.5m soil temperature profile
*** snow depth @ tripod
*** snow depth away from tripod
*** interval camera
*** 4 component radiation
*** Spectral response sensors: NDVI & PRI up looking and two vegetation communities
*** 1.5m Air temperature, Relative Humidity, & Dew Point Temperature
*** Wind speed and direction at 3m
*** summer precipitation
* @ Teller Bottom
** send one more program to fix 5cm bgs soil moisture out of range array error
* @ C1 Grid
** We have a CNR4 for this site I believe.
* @ C2 Blueberry
** Replace the LW Transmitted cable, will need to bring the whole package from Fairbanks
** 10meter AT/RH has not worked since 2011

Seward Peninsula Sites

2016-11-02T22:49:23Z

69.161.4.229: /* Trip Itineraries etc. */

==== Trip Itineraries etc.====
* [[2007 September Nome Trip]]
* [[2008 June Nome Trip]]
* [[2008 August Nome Trip]]
* [[2009 August Nome Trip]]
* [[2009 September Nome Trip]]
* [[2010 June Nome Trip]]
* [[2010 August Nome Trip]]
* [[2010 November Nome Trip]]
* [[2011 August Nome Trip]]
* [[2012 August Nome Trip]]
* [[2014 July Nome Trip]]
* [[2015 July Nome Trip]]
* [[2015 September Nome Trip]]
* [[2016 June Nome Trip]]
* [[2016 April Nome Trip]]
* [[2016 Summer Nome Trip]]
* [[2016 NGEE August Nome Trip]]
* [[2016 NGEE September Nome Trip]]
* [[2017 NGEE June Nome Trip]]
* [[ General Site Checklist]]
* [[Storage Inventory]]
* Kougarok Storage: Combo changed in 2015. Ask Bob Busey for access if needed.
* [[Base Callbook]]

==== Data ====
Unofficial data location on ngeedata: 
/home/bbusey/working_files/sp_datapro/raw/ 
Internal wiki data: 
https://wiki.alaska.edu/display/iarcsp/

* [[C1-Grid Data Downloads]]
* [[C2-Blueberry Data Downloads]]
* [[C3-Guy Rowe Data Downloads]]
* [[K1 Burn Data Downloads]]
* [[K2 Met Data Downloads]]
* [[K3 Mauze Data Downloads]]
* [[K1 Snow Thermocouples Data Downloads]]
* [[K2 Snow Thermocouples Data Downloads]]
* [[Anvil Mountain Data Downloads]]
* [[Skookum Pass Data Downloads]]
* [[Kigluaiks Repeater Data Downloads]]
* [[Kigluaiks Repeater 2 Data Downloads]]
* [[Online Archive Fixes]]

==== Instrumentation Serial Numbers ====
* [[C1-Grid Instrumentation History]]
* [[C2-Blueberry Instrumentation History]]
* [[C3-Guy Rowe Instrumentation History]]
* [[K1 Burn Instrumentation History]]
* [[K2 Met Instrumentation History]]
* [[K3 Mauze Instrumentation History]]
* [[K1 Snow Thermocouples Instrumentation History]]
* [[K2 Snow Thermocouples Instrumentation History]]
* [[Anvil Mountain Instrumentation History]]
* [[Skookum Pass Instrumentation History]]
* [[Kigluaiks Repeater Instrumentation History]]
* [[Kigluaiks Repeater 2 Instrumentation History]]

====Sites Background====
As carry over from the ATLAS and HARC projects WERC maintains a network of telemetered data stations on the Seward Peninsula. Archived data is currently found on the [http://www.uaf.edu/water/project/atlas/atlas.html UAF WERC site]

Principal Data site, Council Area:
[[C1 Grid]] (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U27]), active
[[C2 Blueberry]], active
C3 Guy Rowe, inactive

Principal Data site, Kougarok Area:
[[K1 Burn]] site, active
[[K2 Met]], active
[[K3 Mauze]] Gulch (near [http://www.udel.edu/Geography/calm/data/data-links.html CALM site U28]), active

We use radio telemetry to communicate with our stations out on the peninsula. So, there are several radio repeaters in that list, too. They are:
[[Anvil]] Mountain (near Nome)
[[Skookum]] Pass (near Council)
[[Kigluaiks]] (in Kigluaik Mountains across the valley to the south of Kougarok)

In addition there are several other sites:
[[Snake]] River (outside Nome on Nome-Teller Highway)
[[Kuzitrin]] River (60+ miles up the Kougarok Road from Nome)
[[Niagara]] Creek (~1 mile from K2 Met)

See most of them on a map here:
[http://data.ine.uaf.edu/seward/ Site Map]

Radios turned off for C1-grid, C2-blueberry, Skookum Pass, Anvil Mountain on 11/4/2009

====Project Background====
Originally there were three Council sites, C1, C2, and C3. Each was selected to be in a different type of landscape but C3 kept getting attacked by bears so we eventually removed it. C1 is called C1-Grid because it is located in the valley bottom near Council along the perimeter of a CALM 1 km x 1 km active layer grid. So, the meteorological station was set up to complement the field data collected on the CALM grid. In terms of importance to the original project it was a secondary station so the data elements collected there are basically: Air Temperature / Relative Humidity (1m & 3m), Wind Speed (1m & 3m) & Wind Direction (3m), Net Radiation (in summer), precipitation (in summer), snow depth, soil moisture (vertically profiled), and soil temperature (vertically profiled). The station is also a shorter 3 meter tower.

===Council Sites===
The blueberry site is located on Blueberry Hill to the SW of Council. It is the primary site for the Council area so has more sensors and the tower is a 10 meter high setup. In addition to the sensors mentioned for C1 there is also up facing shortwave & longwave radiation, and down facing shortwave & longwave radiation sensors. Wind speed & AT/RH (as a single sensor) are also measured at more heights than at C1 (1m, 3m & 10m). In 2007 I made some changes to the station. In the past longwave radiation was measured using a battery. However, it was a lead-mercury battery, which is considered hazardous and has been discontinued so I re-did the wiring. This will be reflected in the data as a couple extra data columns that appear last year. Blueberry is across the Niukluk river (I think that's the river, not 100%) and is sometimes unreachable in fall. So, we weren't able to get across the river to remove the net radiometer in Fall 2007. Data for it shouldn't be trusted much past October 1st though because once the snow falls or frost accumulates on the domes that protect the radiation sensor.

On a similar note, the precipitation gages are out year-round but we only record summer rainfall. So, you should mark with '7777' the winter time of year. For measured precipitation I don't generally count it until the air temperature is consistently above freezing. It's kind of subjective on the shoulder seasons figuring out what is snow vs what is rainfall and if you don't feel comfortable with that, I'm happy to look over the data and define select which points to include and which to exclude. It may sound like we're doing something funky but we aren't. The tipping bucket rain gage is only designed to measure liquid precipitation. So, when snow accumulates in the bucket no measurements are recorded until the bucket temperature gets above freezing to melt the snow in the bucket. Since those gages are just used in the summer then we exclude all solid precip that may melt in and be counted at the data logger.

===Kougarok Sites===
K1-burn is another secondary site. It is located on an area burned by an old tundra fire up on a low sloping bench above the Kougarok river. The vegetation in this area is dry tussock and grasses. Much of the organic soil layer burnt off in the fire so recovery has been slow. This site gets visited fairly frequently by animals so you'll see most of the soil moisture sensors have had their cables eaten, as has also happened to the soil temperature sensors. This site is quite similar to C1 that I talked about earlier in terms of sensor setup. In the fall of 2007 we added a snow depth sensor to the site so that will be a new thing. I'm not sure about its reporting though.

K2-met is the 10 meter tower for the Kougarok area. It is located on very wet tundra 300 meters or so up slope from the Kougarok river, so down in the valley. It is a full site similar to C2 although it also has a snow sensor installed. A broken, older sensor was replaced with a newer one in fall 2007.

K3 mauze is located at the perimeter of the Kougarok area CALM grid. It is another 3 meter tower. This site has been hammered by the bears in recent years so some sensors are in and out in terms of their quality.

===Nome Base===
The base radio and moxa server are located at the UAF Northwest campus in Nome.

[[File:NomeAntenna.jpg|400px]]
[[File:NomeRadio.jpg|400px]]

===Discussion of the Station Data===
So, that's a short bit of background on the stations. The data on the CD I gave you is predominantly archived data or data that has been through some level of processing. You can find the up to the day data here: 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/ 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-blueberry 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-burn 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-k-met 
http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-kmauze 

Those are all quite large at this point text files. Let me know if you find any gaps in the data. Sometimes there is trouble with the back end connection between the UAF webserver and the computer that retrieves this data over the radio network from Nome.

So, when you look at the data there it will probably look like a bunch of not real helpful numbers. Plus you'll see an Array ID at the start of the line which seems to alternate each hour. That is because the meteorological data arrays are mixed together withe subsurface data arrays. It's kind of a by product of the data logger we use. The design for them was done in the early 90s or late 80s so the electronic data storage architecture is robust but perhaps a bit primitive compared to the computers of today. So, to split out all the data I should have included on that cd a directory called 'batch' or 'bin'. If you look in there you should see some .bat files which are dos batch files. I don't know if you're using linux or windows, if you're using linux then pop open .bat file and you'll see 'grep .........' you should be able to change the directory paths there (I can help if you have questions about this) but these little batch files basically read through the data above (e.g. atlas-blueberry) and pull out all the met data arrays and then all the soil data array lines and dump them into different files. So, when you mention seeing different data files in the CD I gave you, the data is in various steps of processing. Oh, also, if you are using a windows computer (as I am for the data processing) then you should have cygwin installed. It's a program that gives you many of the free utilities that come with linux/unix/debian (most helpfully for us today, grep). You can download it here if you need:
http://www.cygwin.com

So, if you're using windows, just modify the batch files. I like to use this program:
[http://www.context.cx Context]
for viewing text files. I find it much more useful than notepad, but if you have a favorite text editing program, feel free to use it. Anyway, if you're using windows you'll need to modify the batch file to grep out the 2007 data I think. The general format for the grep command is:
grep [text string to search for] [input data file] > or >> [output data file]
c:\cygwin\bin\grep 228,3018,2007 "c:\data\atlas\c1-grid\atlas-c1-grid.txt" > "c:\data\atlas\c1-grid\c1-grid_228-2007-soil.csv"
You'll see both '>' and '>>' in the batch files. '>' means put all the data line matches into the output file and create a new file each time. '>>' means put all the data line matches into the output file and append them to the end of the file if it already exists.

So, you'll need to change the paths (e.g. 'c:\data\'), to whatever you're using. I like
c:\data\project\station_name from windows or /home/bob/data/project/station_name from linux but totally your call on how you organize. From linux the line above would like more like this:
grep 228,3018,2007 "/home/bob/data/atlas/c1-grid/atlas-c1-grid.txt" > "/home/bob/data/atlas/c1-grid/c1-grid_228-2007-soil.csv"

So, when you see stuff like this:
CD/atlas/C1-grid/c1-grid_123-2006-met.csv
CD/atlas/C1-grid/c1-grid_123-2007-met.csv
CD/atlas/C1-grid/c1-grid_228-2004-soil.csv
etc. etc.
That is output from the grep program. the 123/228 references the array ID. So, that file contains met data from 2007 from this file:
[http://www.uaf.edu/water/projects/near-real-time/rawdatafiles/atlas-c1-grid C1 Data File]
Here's a sample line:
123,3018,2005,1,100,-0.47,-0.176,97.7,96.1,3.639,162.7,4.414,92.7,0,-2.046,12.35

I see for some flies the '123' has been replaced by the regular date & time. But that's probably the only change in the file. I would have done that if I was just checking things out.

So, moving forwards a bit, my rough method for processing the data is this:
# download data off website
# use GREP to separate out by data line type and year using the batch file for dos.. or if you use linux then you can actually edit the .bat files to be the correct path (like I mentioned above) and then from the command line type this: chmod u+x c1-grid.bat this will allow you to just type './c1-grid.bat' just as I have been doing but you'll do it from linux rather than windows.
# open in excel or use a program I wrote to process the data (more on this below)
# view the data with a data visualization program I used Origin but Excel is semi okay (origin is nice because you can quickly change the time scales) or if you have another program you really like like matlab that's fine, too.
# delete bad data that is apparent through using the visualization program (replace with '6999')
# have someone else do a final check (like me probably)
# post data to the internet

'process the data' is pretty nebulous. I have a program I wrote which converts thermistor resistances (soil temperature data) from resistance to temperature and then formats the data as it should look on the web. That's the main reason you see so many similarly named files. Some are from step 1, some are from step 2 etc. So, that is probably a bit confusing. But for you, probably using a spreadsheet program like excel or whatever you're most comfortable with is better. You can find header information for what is in each column of the data file. The other main bit of 'processing' that happens is to correct the net radiation data for wind. I can dig that equation up for you if you can't find it in those spreadsheets.
So, also along the process the data lines, some things you'll want to look out for:
# missing lines of data... make a note and check with me if you find any
# repeated lines of data... more likely to find due to the rsyncing process (the way the uaf website talks to the server downloading data from the radio network).
There are a couple ways of converting julian dates into regular. You could use these programs I've written for visual basic:
http://werc.engr.uaf.edu/mediawiki/index.php/Excel_Macros
or, another way is this:
a) type into a cell '12/31/06'
b) change the format of the cell to number.
c) if your data is in the format column 1 is the date you'd like, column 2 is the year, column 3 is the day, column 4 is the hour then in column 1 you can print (using data for 2007) '= 39082 + b1 + c1/2400' then format this column as a date & time and you'll be in business. Do some simple boolean stuff to check for gaps of data or, just create another column that has as the first cell '1/1/07 1:00' then the second row in the column '=a1 + 1/24' then look to see if the calculated date (mentioned first) ever varies from the second method. If it doesn't you're golden. If they do vary then you can track down where the problem is. In general, if there is multiple lines of the same data (or so it looks like) I will delete the first instance and keep the last. The reason for this is related to the radio telemetry. Some of the connections are quite weak so we don't always get the data on an hourly basis. Sometimes we'll get half the line so we'll need to get things resent later (with regard to the radio network and data retrieval from the data loggers).

I guess another note on the array IDs. The met data and the radiation data are contained in the same array ID line. The soil temperature and soil moisture data (most of which has broken sensors) is in a separate array ID. That is the 123, 127 etc. that you wonder about below.

Well, I've written a book, maybe I will post this to our wiki and we can go back and forth on there, too. Totally let me know if you have any questions. I'll be in town through Friday (just learned) so I should be able to help more in the short term than the medium term. thanks

===== Part 2 =====
Here's a bit about the radiation. I will get you more calibration values once we're back from the June trip the week of 6/16.

So, in the data files you've looked at off the internet is the output voltage in millivolts as recorded by the data logger. To get from milliVolt output from the data logger to Net Radiation in Watts / m^2 you'll need to do a couple things:
1) correct for wind speed.
2) apply the sensor specific calibration factor (positive or negative depending on sensor output)

Via this manual (pdf page 5 document page 3):
Net Radiometers -- REBS Q-7.1 (Radiation Energy Balance Systems)
http://www.campbellsci.com/documents/manuals/q-7-1.pdf

Are these wind speed correction equations (where wind speed in in meters per second as we have at our stations):

Positive Wind Speed Correction Factor:
Wind_CF_Pos = 1 + (0.066 * 0.2 * 1m_Wind_Speed) / (0.066 + (0.2 * 1m_Wind_Speed) )

Negative Wind Speed Correction Factor:
Wind_CF_Neg = (0.00174 * 1m_Wind_Speed) + 0.99755

So, if Net Radiation (in mVolts as recorded by the data logger) is positive use this equation:
Corrected_Net_Rad = Wind_CF_Pos * Uncor_Net_Rad * Pos_Calib

If Net Radiation (in mVolts as recorded by the data logger) is negative use this equation:
Corrected_Net_Rad = Wind_CF_Neg * Uncor_Net_Rad * Neg_Calib

Here's the equation for C2 as it looks for me in Excel:
=IF(ABS([NetRadiometer Cell])<> 6999,IF(L4087>0,9.15*[Net Radiometer Cell]*(1+(0.066*0.2*[1 meter Wind Speed])/(0.066+0.2*[1 meter Wind Speed])),11.13*L4087*(0.00174*[1 meter Wind Speed]+0.99755)),7777)

So, given all that you need one more thing, the calibration values for positive and negative calibrations. Here's a first batch and I'll get you some more after we come back from out trip in a couple weeks.
C1 Grid:
Positive Calibration: 9.18
Negative Calibration: 11.31

C2 Blueberry:
Positive Calibration: 9.15
Negative Calibration: 11.13

K1 Burn:
[Need to track down yet, after 2008 June visit]
K2 Met:
[Need to track down yet, after 2008 June visit]
K3 Mauze Gulch: Q99012
Positive Calibration: 8.97
Negative Calibration: 10.90

The raw data input column information you should find in the excel files or here, too:

http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/C1Grid/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Councilsites/Blueberry/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-burn/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/k-met/current-raw.html
http://www.uaf.edu/water/projects/atlas/metdata/Kougaroksites/kmauze/current-raw.html

The output column information (how everything should be laid out) is here:

http://www.uaf.edu/water/projects/atlas/council/c1/data/met/c1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/soil/c1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c1/data/rad/c1_rad_2005.csv

http://www.uaf.edu/water/projects/atlas/council/c2/data/met/c2_met_2006.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/soil/c2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/council/c2/data/rad/c2_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/met/k1_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/soil/k1_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k1/data/rad/k1_rad_2006.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/met/k2_met_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/soil/k2_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k2/data/rad/k2_rad_2004.csv

http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/met/Mauze_met_2006.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/soil/Mauze_soil_2004.csv
http://www.uaf.edu/water/projects/atlas/kougarok/k3/data/rad/Mauze_rad_2006.csv

===Datapro processing notes===
On 5/23/16 we wanted to update the datapro processed output files to use the directly downloaded logger files instead of the ones from the radio network. Here is what we did, using C1 as an example:

*A record of all directly downloaded data files is found here http://ngeedata.iarc.uaf.edu/wiki/C1-Grid_Data_Downloads
*Config files such as Key files to run data pro are here: ngeedata /home/bbusey/working_files/sp_datapro/c1-grid/config/
*Copied raw directly downloaded data files copied from owncloud folder(owncloud/Modern Seward Peninsula/ Seward_Data/raw_data_archive) to /home/bbusey/working_files/sp_datapro/raw/c1-grid/raw
*Deleted some data in each output file in /home/bbusey/working_files/sp_datapro/c1-grid/outputs so that data ended on 12/31/12 or before
**note 1: in this folder are some shortcut links called symlinks for the website).
**note 2: Datapro looks to see what the last line of data is so it doesn’t append if the data is already in the file
*Copied the bash script from /home/bbusey/bin/process_sp.sh too /home/bbusey/process_direct_downloads_bin/process_c1grid.sh so we could edit it and run it on the directly downloaded data instead of the radio downloads
**about bash: when you use ssh to bring up a terminal window, that particular instance is bash terminal or bash shell. so, you can use a bash script to string together a bunch of bash commands. in this case here we will use a bash script to process each .dat direct download using datapro
**this line specifies that we are running datapro version 3, the key file for array 27, and the direct data file we downloaded from C1 in 2014
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_array_127_key.txt --alt_data_files=$ROOT_DIR_SP/raw/c1-grid/RAW/c1_grid_2014_07_22.dat
**then we run that line again with the next data file, or next key file, etc
**we run the bash script for every new line and put the old line in a comments
*Now the files in the /home/bbusey/working_files/sp_datapro/c1-grid/outputs/ folder should then be updated to the end of the last data file
**you can check this by outputting the last two lines of each .csv file in the folder by typing
tail -n2 /home/bbusey/working_files/sp_datapro/c1-grid/outputs/*.csv
* the script we paused that runs hourly on the radio downloaded files looks like
python $DATAPRO3 --key_file=$ROOT_DIR_C1/config/c1-met_key.txt

C2-Blueberry Data Downloads

2016-11-02T18:00:46Z

69.161.4.229:

C2 Blueberry Data Data Start - Data Stop Array IDs Present
C2JULY2000.CSV 1999,209,1400 - 2000,208,1600
C2OCT99.DAT 1999,209,1400 - 1999,275,1900
BLUE.DAT 2001,97,1400 - 2001,137,600
C20401.DAT 2001,9,500 - 2001,97,1300
C2102701.DAT 2000,229,1200 - 2000,301,1200
COUN2001.DAT 2000,256,1500 - 2000,301,1200
COUN2002.DAT JUST: 2000,301,1300
COUNT2003.DAT 2000,301,1400 - 2001,27,1400
BLUEBERR.DAT 2001,97,140 - 2001,281,1900
BLUEBR01.DAT 2001,192,1200 - 2001,229,1200
BBDATA1999.XLS 1999,209,1500 - 1999,365,2400
BBDATA2000.XLS 2000,1,300 - 2000,366,2400
BBDATA2001.XLS 2001,1,300 - 2001,281,900
bluedat.txt 2001,277,600 - 2001,297,1900
Blueberry_2005-06-05-a1.dat 2002,70,1700 - 2005,156,2100
Blueberry_2005-07-28-a1.dat 2005,156,2200 - 2005,209,2200 131,230
C2_final_storage_1.dat 2002,231,2400 - 2006,227,1100 130,131,230
C2_2006-10-06.dat 2006,227,1200 - 2006,279,1700 131,230
C2_2007-06-09.dat 2006,279,1800 - 2007,160,1800 131,230
C2_2007-07-26.dat 2007,160,1900 - 2007,207,1100 167,168,230
C2_2008-06-13.dat 2007,207,1200 - 2008,25,2400 168,230
C2_2008-08-26.dat 2008,165,1500 - 2008,239,1900 179,191,237
c2_blueberry_2009_08_05.dat 2008,239,1900 - 2009,217,1200 179,191,237
c2_blueberry_2009_09_27.dat 2009,217,1300 - 2009,270,1000 179,196,237
C2_2010_06_26.dat 2009,217,1200 - 2010,177,2100 179,196,237
C2_2010_08_10.dat 2009,270,1100 - 2010,222,1900 179,196.237
c2_blueberry_2011_08_16.dat 2010,222,2100 - 2011,228,1800 179,182,196,238
c2_blueberry_2012_08_06.dat 2010,282,1500 - 2012,219,1700 179,182,238,196,250
c2_blueberry_2012_radio-only_gap.dat 2012,219,1800 - 2012,288,600 182,238,250
c2_blueberry_2014_07_24.dat 2012,288,600 - 2014,205,1200 182,238,250
c2_blueberry_2014_radio-only_gap.dat 2014,205,100 - 2014,269,2400 182,238,250
c2_blueberry_2016_08_18.dat 2014,249,400 - 2016,231,1000 182,238,250

SHAND RECORD 2001,365,2400 - PRESENT
Ralph now carries out the duties of Shand.

Legit Gaps: 2003 82 2100 to 2003 101 1500
2002 168 2100 to 2002 217 1800
2002 97 900 to 2002 97 1500
2008 26 0000 to 2008 165 1400 (dead battery)
2010 222 2000 (station maintenance)
2014 0124 0200 to 2014 0206 1200 (low battery voltage. There are a few data points in that period, but spotty)
2015 8/24 15:00 -- After shand.engr shut down there is some major funkyness going down with the clock (see note below)
I think this will only be resolved with a site visit.

Incomplete Array ID Identification:
130 = 23 elements, Met
131 = 24 elements, Met
168 = 30 elements, Met
179 = 29 elements, Met
182 = 29 elements, Met
191 = 5 elements, Snow
191 = 7 elements, Snow
196 = 4 elements, Snow
250 = 4 elements, Snow
230 - 22, Soil
230 - 23, Soil
237 - 24, Soil
238 = 25, Soil

==========================================================================
Just generally, in the fall of 2009 the Incoming shortwave radiometer wires at the radiation stand become loose. There are some garbage values interspersed that fall that I got tired of cleaning up. Primarily mid October to Mid December. Maybe all of that data should just be zeroed out.

===Fall 2015 to Spring 2016===
In August 2015 shand.engr.uaf.edu was shut down. Shand had been the remote data retrieval computer from 2002 to 2015. Over the fall of 2015 we had a number of issues transitioning away from shand. It quickly became obvious that LoggerNet could not consistently retrieve data from the tenuous radio links on the SP. I moved the most recent version of Loggerdata.pl to host lognet so that we could take advantage of the older, more stable data retrieval software on the new computer. However, it turns out that there is a major bug specific to CentOS, the operating system running on LogNet. The time zone of the logger specified in the configuration files used by Loggerdata.pl was ignored on centOS so August 2015 to early January 2016 there was some fighting back and forth with the two competing data retrieval programs. I inadvertently fixed this by disabling loggerdata downloads for this site in late December / early January. However, due to poor performance I re-enabled again in March to try to get the sites caught up before our April trip. Around then I figured out the timezone bug present when running things on LogNet / CentOS. So, data retrievals were moved to host Ralph which is a debian OS based computer. Downloading became more consistent at that point and I'm not sure why we haven't heard from the Council sites since early May but could be a bear again. But, in light of this there is quite a bit of data in the Blueberry hill data stream with a mix of timezone (UTC vs Ak Standard at UTC-9hrs). It is only apparent which time zone the data is in if you look at the incoming shortwave radiation data and so I thought I would create this section to delineate as best I can tell. Definitely a bit of an own goal...

Station origination to
"2015-08-24 15:00:00" AKSTD
"2015-08-24 01:00:00" UTC
"2015-08-25 01:00:00" AKSTD
"2015-08-25 00:00:00" From here till the next several months, a low value @ 0:00 or 1:00 = AKSTD, a high value = UTC
If that makes any sense.
Like this sampler:
"2015-09-02 00:00:00",0.00 (AKSTD)
"2015-09-02 00:00:00",0.00 (AKSTD)
"2015-09-02 00:00:00",0.00 (AKSTD)
"2015-09-02 01:00:00",0.00 (AKSTD)
"2015-09-02 01:00:00",11.24 (AKSTD)
"2015-09-02 00:00:00",126.00 (UTC)
"2015-09-02 01:00:00",147.70 (UTC)
"2015-09-02 01:00:00",128.10 (UTC)
"2015-09-02 01:00:00",54.21 (UTC)
"2015-09-02 00:00:00",9.77 (AKSTD)
"2015-09-02 01:00:00",0.07 (AKSTD)
"2015-12-20 00:00:00" UTC
"2015-12-29 00:00:00" AKSTD
"2016-03-28 18:00:00" UTC
"2016-03-29 17:00:00" AKSTD
"2016-03-30 00:00:00" UTC
"2016-03-30 16:00:00" AKSTD

C2-Blueberry Data Downloads

2016-11-02T17:36:22Z

69.161.4.229:

C2 Blueberry Data Data Start - Data Stop Array IDs Present
C2JULY2000.CSV 1999,209,1400 - 2000,208,1600
C2OCT99.DAT 1999,209,1400 - 1999,275,1900
BLUE.DAT 2001,97,1400 - 2001,137,600
C20401.DAT 2001,9,500 - 2001,97,1300
C2102701.DAT 2000,229,1200 - 2000,301,1200
COUN2001.DAT 2000,256,1500 - 2000,301,1200
COUN2002.DAT JUST: 2000,301,1300
COUNT2003.DAT 2000,301,1400 - 2001,27,1400
BLUEBERR.DAT 2001,97,140 - 2001,281,1900
BLUEBR01.DAT 2001,192,1200 - 2001,229,1200
BBDATA1999.XLS 1999,209,1500 - 1999,365,2400
BBDATA2000.XLS 2000,1,300 - 2000,366,2400
BBDATA2001.XLS 2001,1,300 - 2001,281,900
bluedat.txt 2001,277,600 - 2001,297,1900
Blueberry_2005-06-05-a1.dat 2002,70,1700 - 2005,156,2100
Blueberry_2005-07-28-a1.dat 2005,156,2200 - 2005,209,2200 131,230
C2_final_storage_1.dat 2002,231,2400 - 2006,227,1100 130,131,230
C2_2006-10-06.dat 2006,227,1200 - 2006,279,1700 131,230
C2_2007-06-09.dat 2006,279,1800 - 2007,160,1800 131,230
C2_2007-07-26.dat 2007,160,1900 - 2007,207,1100 167,168,230
C2_2008-06-13.dat 2007,207,1200 - 2008,25,2400 168,230
C2_2008-08-26.dat 2008,165,1500 - 2008,239,1900 179,191,237
c2_blueberry_2009_08_05.dat 2008,239,1900 - 2009,217,1200 179,191,237
c2_blueberry_2009_09_27.dat 2009,217,1300 - 2009,270,1000 179,196,237
C2_2010_06_26.dat 2009,217,1200 - 2010,177,2100 179,196,237
C2_2010_08_10.dat 2009,270,1100 - 2010,222,1900 179,196.237
c2_blueberry_2011_08_16.dat 2010,222,2100 - 2011,228,1800 179,182,196,238
c2_blueberry_2012_08_06.dat 2010,282,1500 - 2012,219,1700 179,182,238,196,250
[Find the gap between 2012,219,1700 and 2012,288,600 in the radio record]
c2_blueberry_2014_07_24.dat 2012,288,600 - 2014,205,1200 182,238,250
c2_blueberry_2016_08_18.dat 2014,249,400 - 2016,231,1000 182,238,250

SHAND RECORD 2001,365,2400 - PRESENT
Ralph now carries out the duties of Shand.

Legit Gaps: 2003 82 2100 to 2003 101 1500
2002 168 2100 to 2002 217 1800
2002 97 900 to 2002 97 1500
2008 26 0000 to 2008 165 1400 (dead battery)
2010 222 2000 (station maintenance)
2014 0124 0200 to 2014 0206 1200 (low battery voltage. There are a few data points in that period, but spotty)
2015 8/24 15:00 -- After shand.engr shut down there is some major funkyness going down with the clock (see note below)
I think this will only be resolved with a site visit.

Incomplete Array ID Identification:
130 = 23 elements, Met
131 = 24 elements, Met
168 = 30 elements, Met
179 = 29 elements, Met
182 = 29 elements, Met
191 = 5 elements, Snow
191 = 7 elements, Snow
196 = 4 elements, Snow
250 = 4 elements, Snow
230 - 22, Soil
230 - 23, Soil
237 - 24, Soil
238 = 25, Soil

==========================================================================
Just generally, in the fall of 2009 the Incoming shortwave radiometer wires at the radiation stand become loose. There are some garbage values interspersed that fall that I got tired of cleaning up. Primarily mid October to Mid December. Maybe all of that data should just be zeroed out.

===Fall 2015 to Spring 2016===
In August 2015 shand.engr.uaf.edu was shut down. Shand had been the remote data retrieval computer from 2002 to 2015. Over the fall of 2015 we had a number of issues transitioning away from shand. It quickly became obvious that LoggerNet could not consistently retrieve data from the tenuous radio links on the SP. I moved the most recent version of Loggerdata.pl to host lognet so that we could take advantage of the older, more stable data retrieval software on the new computer. However, it turns out that there is a major bug specific to CentOS, the operating system running on LogNet. The time zone of the logger specified in the configuration files used by Loggerdata.pl was ignored on centOS so August 2015 to early January 2016 there was some fighting back and forth with the two competing data retrieval programs. I inadvertently fixed this by disabling loggerdata downloads for this site in late December / early January. However, due to poor performance I re-enabled again in March to try to get the sites caught up before our April trip. Around then I figured out the timezone bug present when running things on LogNet / CentOS. So, data retrievals were moved to host Ralph which is a debian OS based computer. Downloading became more consistent at that point and I'm not sure why we haven't heard from the Council sites since early May but could be a bear again. But, in light of this there is quite a bit of data in the Blueberry hill data stream with a mix of timezone (UTC vs Ak Standard at UTC-9hrs). It is only apparent which time zone the data is in if you look at the incoming shortwave radiation data and so I thought I would create this section to delineate as best I can tell. Definitely a bit of an own goal...

Station origination to
"2015-08-24 15:00:00" AKSTD
"2015-08-24 01:00:00" UTC
"2015-08-25 01:00:00" AKSTD
"2015-08-25 00:00:00" From here till the next several months, a low value @ 0:00 or 1:00 = AKSTD, a high value = UTC
If that makes any sense.
Like this sampler:
"2015-09-02 00:00:00",0.00 (AKSTD)
"2015-09-02 00:00:00",0.00 (AKSTD)
"2015-09-02 00:00:00",0.00 (AKSTD)
"2015-09-02 01:00:00",0.00 (AKSTD)
"2015-09-02 01:00:00",11.24 (AKSTD)
"2015-09-02 00:00:00",126.00 (UTC)
"2015-09-02 01:00:00",147.70 (UTC)
"2015-09-02 01:00:00",128.10 (UTC)
"2015-09-02 01:00:00",54.21 (UTC)
"2015-09-02 00:00:00",9.77 (AKSTD)
"2015-09-02 01:00:00",0.07 (AKSTD)
"2015-12-20 00:00:00" UTC
"2015-12-29 00:00:00" AKSTD
"2016-03-28 18:00:00" UTC
"2016-03-29 17:00:00" AKSTD
"2016-03-30 00:00:00" UTC
"2016-03-30 16:00:00" AKSTD