Difference between revisions of "NGEE Teller SW Notes"

From IARC 207 Wiki
Jump to navigation Jump to search
 
(15 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
[[NGEE]]
 
[[NGEE]]
 
===Direct Link to Data===
 
===Direct Link to Data===
−
* http://ocotal.iarc.uaf.edu/data/uaf_sp/teller_sw/
+
(Note to myself: 2022-01-18 I still need to update monitors site with data set update from November 2021.  What needs to be done specifically is re-chunk out the data by calendar year).
 +
 
 +
* http://monitors.iarc.uaf.edu/data/uaf_sp/teller_sw/
 
* Summary of Observations:
 
* Summary of Observations:
 
** https://docs.google.com/spreadsheets/d/1gxLKzjoDvJkFmkDe0SVBhRRbokB-rvEpH77nN8cyLo0/edit#gid=1509399423
 
** https://docs.google.com/spreadsheets/d/1gxLKzjoDvJkFmkDe0SVBhRRbokB-rvEpH77nN8cyLo0/edit#gid=1509399423
Line 9: Line 11:
  
 
===General===
 
===General===
−
* Look over bash script... need to make sure that QA corrections take place after stage is corrected to match rebar values...
+
* Winter 2021 Observation: Look over bash script... need to make sure that QA corrections take place after stage is corrected to match rebar values...
 
** a second QA I think.
 
** a second QA I think.
 +
*** Basically, the grouper needs to run once to create the hourly individual time series discharge data.
 +
*** Then the manual QA is applied.
 +
*** then the data grouper runs a second time
 +
*** This is a bit odd and another way to do this would be to split the teller_sw data grouper into two pieces but it works ok for now.
 
* I'm looking at the 2017 April 27 & 28 discharge measurement times and the corrected data looks great but the raw data looks like it has the shift applied too early.  Perhaps that is unavoidable?  I'll think on it.  but currently not fixed 11/7/2018
 
* I'm looking at the 2017 April 27 & 28 discharge measurement times and the corrected data looks great but the raw data looks like it has the shift applied too early.  Perhaps that is unavoidable?  I'll think on it.  but currently not fixed 11/7/2018
 
* '''Basin Size is 1.24 km^2 or 1,243,875 m^2'''* I think we need to double-check this value using the high resolution DSM from LANL. ''2/14/2019''
 
* '''Basin Size is 1.24 km^2 or 1,243,875 m^2'''* I think we need to double-check this value using the high resolution DSM from LANL. ''2/14/2019''
 
* Cross section at salt slug dump is 2.7 meters roughly
 
* Cross section at salt slug dump is 2.7 meters roughly
 
* about 90 meters from there to the measuring point (following left bank surveyed points).
 
* about 90 meters from there to the measuring point (following left bank surveyed points).
 +
* '''Definitely need a low flow second ratings curve.  Linear and kicks in around 11.66cm corrected stage value.'''
 +
===Instrumentation 2021 Refresh===
 +
* Data logger
 +
** Campbell Scientific CR800
 +
* Pressure Transducer 2 (old supplmented in June 2021)
 +
** Instrumentation Northwest AquiStar PT12
 +
** https://www.rshydro.co.uk/PDFs/INW/sdi-12_manual.pdf
 +
** Range Stage:  0 to 3.51 m of water (0 to 5 psig)
 +
** Accuracy Stage: +- 0.1 % Full Scale Range
 +
** Range Temperature: -15 C to 55 C
 +
** Accuracy Temperature: +- 0.5 C
 +
* Pressure Transducer 3 (added in September 2021)
 +
** Campbell Scientific CS451
 +
** https://s.campbellsci.com/documents/us/manuals/cs451-cs456.pdf
 +
** Range Stage: 0 to 2.0 m of water (0 to 2.9 psig)
 +
** Accuracy Stage: +- 0.1 % Full Scale Range
 +
** Range Temperature: 0 C to 60 C
 +
** Accuracy Temperature: +- 0.2 C
 +
Historical notes about this refresh:
 +
* On June 2021 visit deployed the CR10X -> CR800 auto translated program and adjusted it slightly.
 +
** Left original PT in the creek and added the backup that had been in nome storage.
 +
** In the field it looked like the original PT had frozen and wasn't responding properly any longer (or maybe moisture in the vent tube etc)
 +
* In September cut the old PT and added the CSI CS451 which has higher resolution.
 +
* I'm not sure why but when looking at data table fill times it looked like what I had set up in June with outputting all variables every 10 minutes wasn't going to work...
 +
* We also added a snow sensor at this site and perhaps adding the 5 minute data to the 10 minute data shortened the logger fill times too much.  So, on the September visit (meaning over the 14 days we were in Nome & outlying area) I changed battery voltage & panel temperature to be a once per hour output.
 +
 +
===Original Instrumentation ===
 +
* Data logger
 +
** Campbell Scientific CR10X
 +
* Pressure Transducer
 +
** Instrumentation Northwest AquiStar PT12
 +
** https://www.rshydro.co.uk/PDFs/INW/sdi-12_manual.pdf
 +
** Range Stage:  0 to 3.51 m of water (0 to 5 psig)
 +
** Accuracy Stage: +- 0.1 % Full Scale Range
 +
** Range Temperature: -15 C to 55 C
 +
** Accuracy Temperature: +- 0.5 C
  
 
===Latest===
 
===Latest===
−
* Officially processed through 3/2019
+
* Officially processed through 6/2019
−
** there are some early winter 2017 peaks I think should be considered.
+
** there are some early winter 2017 peaks I think should be considered for deletion.
 
** In 2018-2019 winter stage got low enough that curve is off again.  Like lower stage = higher flow.
 
** In 2018-2019 winter stage got low enough that curve is off again.  Like lower stage = higher flow.
 +
* For 2021+ I think going to retire PT1 from record the PT2 (June install) and PT3 (September install)
 +
====May 2022====
 +
* Need to come out in May I think before creek stage has dropped too far from snow melt and make a couple discharge measurements.  I think if we do this it will be easier to tie 2021 to 2016 - 2021 record since stage-disharge curve is pretty tight at the top.
 +
 
===QA===
 
===QA===
 +
 
====Stage 2016====
 
====Stage 2016====
  
Line 56: Line 103:
 
  9/24/2018 18:50 AKDT 3cm above rebar 1; 13cm in front of rebar 1
 
  9/24/2018 18:50 AKDT 3cm above rebar 1; 13cm in front of rebar 1
 
  9/26/2018 10:48 AKDT 5cm above rebar 1; 14.5cm in front of rebar 1
 
  9/26/2018 10:48 AKDT 5cm above rebar 1; 14.5cm in front of rebar 1
 +
 +
====Stage 2019-2020====
 +
* In fall 2019 the solar panel disappeared (after our Sept visit).  Then either in 2019 or 2020 winter the creek froze hard enough to kill the transducer membrane (suspected then confirmed in 2021 data).
 +
====Stage 2021====
 +
* June 2021 trip brought out the back up PT and logged until fall the original and backup
 +
* Processed the original PT that I suspected had frozen is indeed totally broken. 
 +
* To continue the data record for discharge in 2021 I did something ballpark and hope to improve in 2022.  Looked at observed discharge value for 6/23/21 as well as the September wading rod measurements.  Used the Q equation to compute the PT Stage best guess value and then adjusted the current pressure transducers readings to that value.  The catch and why I think 2022 will need a second look is that all of the Q measurements in 2021 were at the low end of the curve where there is much more variability in the observations. 
 +
* Another catch with the 2021 stage are the multiple time steps of NAN in the data record.  I think I need to correct that by adding like I've done elsewhere an if/then / do loop to try measuring several times until the SDI12Recorder function isn't returning NAN.
 +
 
====Temperature 2016====
 
====Temperature 2016====
 
  5/2/2016 3:00
 
  5/2/2016 3:00
Line 65: Line 121:
 
  4/6/2018 2:50
 
  4/6/2018 2:50
  
−
   
+
====Temperature 2021====
 +
* The SDI12Recorder function default treatment of the value 2 in the array is to re-use the previous value if current value is NAN (didn't successfully communicate with sensor). I will change that in the 2022 program but for now need to remove with the manual QA this data artifact.
 +
* This weirdness I think for now must be related to the other pressure transducer causing some electrical issues but I don't know that for certain. The battery voltage was normal and in mid July when this began there wasn't even rainfall.  In the past, such as at break up another manifestation of this would be unsteady choppy stage would get a NAN stage reading but on the CR10X the temperature would still vary (April/May 2017 after the channel was cleared of snow exhibit this on occasion).
 +
* So, going to removed all the flat sections in water temperature for now.
 +
* I'm starting to think it might be solar panel related, like a voltage not showing up in the diagnostic value which is the average. Temperature gaps:
 +
2021-07-11 00:30 to 2021-07-11 03:50
 +
2021-07-11 23:10 to 2021-07-12 03:40
 +
2021-07-12 19:50 to 2021-07-13 03:50
 +
2021-07-13 19:40 to 2021-07-14 03:40
 +
2021-07-14 19:40 to 2021-07-15 04:10
 +
2021-07-15 18:50 to 2021-07-16 03:00
 +
2021-07-17 21:40 to 2021-07-17 23:40
 +
2021-07-18 19:30 to 2021-07-19 04:10
 +
2021-07-21 19:50 to 2021-07-22 05:50
 +
2021-07-22 19:30 to 2021-07-22 22:00
 +
2021-07-23 02:20 to 2021-07-23 04:10
 +
2021-07-23 19:30 to 2021-07-23 23:30
 +
2021-07-24 01:40 to 2021-07-24 03:20
 +
2021-07-25 00:40 to 2021-07-25 02:50
 +
2021-07-25 23:40 to 2021-07-26 00:40
 +
2021-07-31 21:00 to 2021-08-01 00:30
 +
 
 
===Site Downloads / visits:===
 
===Site Downloads / visits:===
 
* Bob 4/21/2016 (CR1000)
 
* Bob 4/21/2016 (CR1000)
Line 78: Line 155:
 
* Bob 9/26/2018
 
* Bob 9/26/2018
 
* Bob 4/7/2019
 
* Bob 4/7/2019
 +
* Bob/Bob 6/11 & 6/14/2019
 +
* Bob Bolton 7/3/2019 (Q measurement only)
 +
* Bob Busey 9/19/2019
 +
* Ken Hughes 11/4/2020 (Data logger Retrieval only)
  
 
===From Computer Notes===
 
===From Computer Notes===
Line 97: Line 178:
  
 
  2/8: So.... including precip helps but there is still a period in september / october 2016 that confuses me with the diurnal signal looking more like air temperature driven.  Emily and I sort of think that the heating of the cable might be playing a role here but I haven't looked too hard yet.
 
  2/8: So.... including precip helps but there is still a period in september / october 2016 that confuses me with the diurnal signal looking more like air temperature driven.  Emily and I sort of think that the heating of the cable might be playing a role here but I haven't looked too hard yet.
 +
 +
===Data Archive File Notes===
 +
====10 minute data files====
 +
* stage_1_fix.csv
 +
* watertemp_1.csv
 +
* discharge_steady.csv
 +
====60 minute data files====
 +
* teller_sw_stage.csv
 +
* teller_sw_temperature.csv
 +
* teller_sw_discharge.csv

Latest revision as of 08:52, 18 January 2022

NGEE

Direct Link to Data

(Note to myself: 2022-01-18 I still need to update monitors site with data set update from November 2021. What needs to be done specifically is re-chunk out the data by calendar year).

General

  • Winter 2021 Observation: Look over bash script... need to make sure that QA corrections take place after stage is corrected to match rebar values...
    • a second QA I think.
      • Basically, the grouper needs to run once to create the hourly individual time series discharge data.
      • Then the manual QA is applied.
      • then the data grouper runs a second time
      • This is a bit odd and another way to do this would be to split the teller_sw data grouper into two pieces but it works ok for now.
  • I'm looking at the 2017 April 27 & 28 discharge measurement times and the corrected data looks great but the raw data looks like it has the shift applied too early. Perhaps that is unavoidable? I'll think on it. but currently not fixed 11/7/2018
  • Basin Size is 1.24 km^2 or 1,243,875 m^2* I think we need to double-check this value using the high resolution DSM from LANL. 2/14/2019
  • Cross section at salt slug dump is 2.7 meters roughly
  • about 90 meters from there to the measuring point (following left bank surveyed points).
  • Definitely need a low flow second ratings curve. Linear and kicks in around 11.66cm corrected stage value.

Instrumentation 2021 Refresh

  • Data logger
    • Campbell Scientific CR800
  • Pressure Transducer 2 (old supplmented in June 2021)
  • Pressure Transducer 3 (added in September 2021)

Historical notes about this refresh:

  • On June 2021 visit deployed the CR10X -> CR800 auto translated program and adjusted it slightly.
    • Left original PT in the creek and added the backup that had been in nome storage.
    • In the field it looked like the original PT had frozen and wasn't responding properly any longer (or maybe moisture in the vent tube etc)
  • In September cut the old PT and added the CSI CS451 which has higher resolution.
  • I'm not sure why but when looking at data table fill times it looked like what I had set up in June with outputting all variables every 10 minutes wasn't going to work...
  • We also added a snow sensor at this site and perhaps adding the 5 minute data to the 10 minute data shortened the logger fill times too much. So, on the September visit (meaning over the 14 days we were in Nome & outlying area) I changed battery voltage & panel temperature to be a once per hour output.

Original Instrumentation

  • Data logger
    • Campbell Scientific CR10X
  • Pressure Transducer

Latest

  • Officially processed through 6/2019
    • there are some early winter 2017 peaks I think should be considered for deletion.
    • In 2018-2019 winter stage got low enough that curve is off again. Like lower stage = higher flow.
  • For 2021+ I think going to retire PT1 from record the PT2 (June install) and PT3 (September install)

May 2022

  • Need to come out in May I think before creek stage has dropped too far from snow melt and make a couple discharge measurements. I think if we do this it will be easier to tie 2021 to 2016 - 2021 record since stage-disharge curve is pretty tight at the top.

QA

Stage 2016

8/13/2016 3:00	PTs offset
9/14/2016 22:00	PT1 Position Reset & PT2 Removed
  • 2016 Offsets for corrected PT 1 Stage:
"2016-08-13 03:00:00",-0.61
"2016-08-15 02:30:00",-0.49
"2016-09-14 22:00:00",-1.83
  • Hourly Processing deletions:
    • 2016-09-15 17:00 to 2016-12-31 23:00 :
      • In fall there is a diurnal component to stage that is driven by air temperature I think due to exposed pressure transducer cable after the leaves fall.
      • Probably good data returns for a bit later in fall but it is hard to say just when the channel shape changes due to ice. So, I lean to remove all data until 2017 April visit

Stage 2017

4/1/2017 3:00	Station Turned back on, PT2 in place further downstream
10/1/2017 20:00	PT2 Removed
4/28/2017 1:50 shift in pt2? then
  • Offset for 4/1/2017+
"2017-04-01 01:30:00",-3.10
  • Hourly processing deletions:
    • 2017-01-01 0:00 to 2017-04-01 2:00
      • Winter / station battery dead
    • 2017-10-24 21:00 to 2018-05-14 20:00
      • Winter / hard to say when channel shape changes.
      • I set this date due to the creek being in recession for winter and the water temperature signal which is a full on guess. For reference, I NaN out the summer precip on 10/15. So I'm not 100% confident this is the right time to the hour but probably it's okay?
      • Ending date in May 2018 was selected as first day with a diurnal signal in water temperature (i.e. snow melt has melted the snow in the channel) and also, the first Q peak in May to me looks snow influenced. But 5/14 is down from that peak.

Stage 2018

6/15/2018 12:40 AKDT 12cm above rebar 1; 20.5 cm in front of rebar 1
9/24/2018 18:50 AKDT 3cm above rebar 1; 13cm in front of rebar 1
9/26/2018 10:48 AKDT 5cm above rebar 1; 14.5cm in front of rebar 1

Stage 2019-2020

  • In fall 2019 the solar panel disappeared (after our Sept visit). Then either in 2019 or 2020 winter the creek froze hard enough to kill the transducer membrane (suspected then confirmed in 2021 data).

Stage 2021

  • June 2021 trip brought out the back up PT and logged until fall the original and backup
  • Processed the original PT that I suspected had frozen is indeed totally broken.
  • To continue the data record for discharge in 2021 I did something ballpark and hope to improve in 2022. Looked at observed discharge value for 6/23/21 as well as the September wading rod measurements. Used the Q equation to compute the PT Stage best guess value and then adjusted the current pressure transducers readings to that value. The catch and why I think 2022 will need a second look is that all of the Q measurements in 2021 were at the low end of the curve where there is much more variability in the observations.
  • Another catch with the 2021 stage are the multiple time steps of NAN in the data record. I think I need to correct that by adding like I've done elsewhere an if/then / do loop to try measuring several times until the SDI12Recorder function isn't returning NAN.

Temperature 2016

5/2/2016 3:00
8/13/2016 3:00

Temperature 2017

10/1/2017 20:50

Temperature 2018

4/6/2018 2:50

Temperature 2021

  • The SDI12Recorder function default treatment of the value 2 in the array is to re-use the previous value if current value is NAN (didn't successfully communicate with sensor). I will change that in the 2022 program but for now need to remove with the manual QA this data artifact.
  • This weirdness I think for now must be related to the other pressure transducer causing some electrical issues but I don't know that for certain. The battery voltage was normal and in mid July when this began there wasn't even rainfall. In the past, such as at break up another manifestation of this would be unsteady choppy stage would get a NAN stage reading but on the CR10X the temperature would still vary (April/May 2017 after the channel was cleared of snow exhibit this on occasion).
  • So, going to removed all the flat sections in water temperature for now.
  • I'm starting to think it might be solar panel related, like a voltage not showing up in the diagnostic value which is the average. Temperature gaps:
2021-07-11 00:30 to 2021-07-11 03:50
2021-07-11 23:10 to 2021-07-12 03:40
2021-07-12 19:50 to 2021-07-13 03:50
2021-07-13 19:40 to 2021-07-14 03:40
2021-07-14 19:40 to 2021-07-15 04:10
2021-07-15 18:50 to 2021-07-16 03:00
2021-07-17 21:40 to 2021-07-17 23:40
2021-07-18 19:30 to 2021-07-19 04:10
2021-07-21 19:50 to 2021-07-22 05:50
2021-07-22 19:30 to 2021-07-22 22:00
2021-07-23 02:20 to 2021-07-23 04:10
2021-07-23 19:30 to 2021-07-23 23:30
2021-07-24 01:40 to 2021-07-24 03:20
2021-07-25 00:40 to 2021-07-25 02:50
2021-07-25 23:40 to 2021-07-26 00:40
2021-07-31 21:00 to 2021-08-01 00:30

Site Downloads / visits:

  • Bob 4/21/2016 (CR1000)
  • Lily 7/13/2016 (CR1000)
  • Bob/Lily 8/12/2016 (CR1000)
  • Bob 9/14/2016
  • Bob 4/1/2017
  • Bob 5/2017
  • Bob/Lily 10/1/2017
  • Bob 3/30/2018
  • Bob 6/13 & 15/2018
  • Bob 9/26/2018
  • Bob 4/7/2019
  • Bob/Bob 6/11 & 6/14/2019
  • Bob Bolton 7/3/2019 (Q measurement only)
  • Bob Busey 9/19/2019
  • Ken Hughes 11/4/2020 (Data logger Retrieval only)

From Computer Notes

Need to put into a spreadsheet pressure transducer value / date / manual stage reading.... need to review notes.

Related: discharge measurements
Related: Get manual stage & discharge measurements into UTC time.

Then once that is complete can look at using 
/var/site/uaf_sp/bin/teller_sw.sh
and the step utility function to correct everything to proper heights.
...should think about gedit plug ins too to make it more like xed.

Original Google Doc:

Summary Google Doc: 


2/8: So.... including precip helps but there is still a period in september / october 2016 that confuses me with the diurnal signal looking more like air temperature driven.  Emily and I sort of think that the heating of the cable might be playing a role here but I haven't looked too hard yet.

Data Archive File Notes

10 minute data files

  • stage_1_fix.csv
  • watertemp_1.csv
  • discharge_steady.csv

60 minute data files

  • teller_sw_stage.csv
  • teller_sw_temperature.csv
  • teller_sw_discharge.csv
Retrieved from "http://ocotal.iarc.uaf.edu/index.php?title=NGEE_Teller_SW_Notes&oldid=4071"