SR50 programming notes for CR1000 - Revision history

imported>Ken: /* APPENDIX: digital filter */

2008-08-28T19:02:38Z

APPENDIX: digital filter

imported>Ken: /* proposed CR1000 programming changes */

2008-08-28T19:01:14Z

proposed CR1000 programming changes

imported>Ken at 18:58, 28 August 2008

2008-08-28T18:58:07Z

imported>Ken at 18:53, 28 August 2008

2008-08-28T18:53:12Z

imported>Ken at 18:45, 28 August 2008

2008-08-28T18:45:58Z

New page

== SR50 sensors ==

= description of sensor =

The SR50 emits an audible pulse of sound and measures the interval of time
until an echo of the pulse is detected. Directed down from a height above
the ground, the SR50 is used here to measure the distance between the sensor
and the surface of the bare ground or the accumulated snow pack.

The SR50 is a ``smart'' sensor with onboard processing, and interfaces to
the datalogger using the serial SDI12 protocol. Various commands are
supported, but for our purposes the ``M'' or ``M1'' commands are key. Both
commands cause the sensor to take several readings and return the average
distance to the target; the latter command additionally returns a ``quality''
value.

The SR50 directly measures time, but returns distance; its onboard
time-to-distance calculation assumes nominal air conditions, including
a temperature of 0C (273.15K). A correction should be made to the
sensor's output distance to account for the actual ambient air temperature.
(Other factors which might affect the speed-of-sound in air are not
corrected for, e.g., atmospheric pressure or relative humidity, but are
presumably negligible...)

= existing CR1000 programming for Bullen and Foothills projects =

The sensor is read once per minute using the ``M'' command, and
including a multiplier of 100 to convert output from m to cm:

SDI12Recorder(DT,SR50_CNTRL,0,"M!",100.0,0)

The distance corrected for ambient air temperature:

TCDT = DT*SQR((AT+273.15)/273.15)

The presumed snow depth is calculated by subtracting the measured and
temperature-corrected distance from the fixed sensor height, which is
measured during site visits and entered into the program:

SD = SD_Ht-TCDT

The SD snow depth value is logged as an average of the 1-minute
measurements every hour in the HrlyAtms output table, and once
per day in the Daily table. In addition, the DT raw distance (not
temperature-corrected) is output as an average in the DailyRaw table.

= proposed CR1000 programming changes =

The sensor is read once per minute using the ``M1'' command:

SDI12Recorder(SR50,SR50_CNTRL,0,"M1!",100.0,0)

This instruction outputs two values into the ``SR50'' array variable,
which is further expanded into ``SR50_cm'' for distance and ``SR50_Q''
for quality. (In addition, an alias is provided for ``DT'' for output
in the DailyRaw table.)

The temperature-corrected distance is also converted to a negative value,
reflecting the fact that it is directed *down*:

SR50_tc = -SR50_cm*SQR((AT+273.15)/273.15)

The previous snow depth calculation and data output tables are not
changed, with the ``SD'' value averaged in the HrlyAtms and Daily
tables:

SD = SD_Ht+SR50_tc

New programming is added, first skipping further processing if either
the raw distance or quality value is zero:

if (SR50_cm <> 0) AND (SR50_Q <> 0) then

The sensor quality value is used to derive a '''quality weighting
factor''' using an ad hoc and arbitrary function, with the goal of placing
higher confidence in the data when the quality is between 162 and 210,
and lower confidence for greater values:

qf = 162 ^ 2 / SR50_Q ^ 2

A '''wind weighting factor''' is defined for accounting for sensor
problems in high winds, but is not yet used:

wf = 1

The above weighting factors are applied to a filter time constant which
is used to implement a simple digital low-pass filter. Three different
time constants are used, 1 hour, 4 hours, and 12 hours, to smooth the
temperature-corrected data over those intervals.

The time constants are hard-coded in the program to go along with the
named filter variables, SR50_f1h, SR50_f4h, and SR50_f12h, which are
output in the new ``SR50'' data table.

The weighted 1-hour time constant is factored by the 1-minute measurement
scan to derive a working fraction x:

x = qf * wf * 60 / 3600 '' 1 hour time constant

'''The filter operation consists of taking 1/x'th of the current measurement
and 1/(1-x) of the previous filtered value and storing it as the new
filtered value:'''

SR50_f1h = SR50_tc * x + SR50_f1h * (1 - x)

The fraction x is adjusted to represent the additional 4- and 12-hour
time constants, and the corresponding filtered values are updated:

x = x / 4 '' 4 hour time constant
SR50_f4h = SR50_tc * x + SR50_f4h * (1 - x)

x = x / 3 '' 12 hour time constant
SR50_f12h = SR50_tc * x + SR50_f12h * (1 - x)

endif

= APPENDIX: information on quality numbers from the SR50 manual =

The SR50 quality numbers seem like they should be useful, but it is
not clear how they are derived (i.e., in the sensor's processor) nor
exactly what they represent.

From section 5.0 (source paragraph expanded):

Target quality numbers can also be recorded when using the SDI-12
or ASCII output option.

The quality numbers provide the user with an indication of the
measurement certainty.

The quality numbers can vary from 162 to 600.

A quality number between 162 and 210 indicates that the SR50 detected
a solid target.

A number between 210 and 300 is an indication that the target is
not definitive ...

... and a number between 300 and 600 is an indication that the target
is of poor quality (see section 3.0 for possible causes).

Though quality numbers of 210 to 600 indicate that the target was
not clearly definitive, it does not mean that the measurement was
inaccurate.

It is only an indication that the SR50 detected a target of poor
quality.

Consideration should be given when interpreting data with high
quality numbers.

From section 3.0:

Numbers lower than 210 are considered to be measurements of good
quality. A value of zero however, indicates a reading was not
obtained. Numbers greater than 300 indicate that there is a degree
of uncertainty in the measurement. Causes of high numbers include:
· the sensor is not perpendicular to the target surface
· the target is small and reflects little sound
· the target surface is rough or uneven
· the target surface is a poor reflector of sound (low density snow)
It is not necessary to make use of the quality numbers but they
can provide additional information such as an indication of surface
density in snow monitoring applications.

= APPENDIX: digital filter =

The digital filter implemented above is the simplest type of '''infinite
impulse response''' filter, and is equivalant to an analog low-pass
filter consisting of a single resistor and a capacitor.

The ``infinite'' part of the name reflects the fact that each measurement
(or ``impulse'') going into the filtered value remains there, albeit
with diminished influence over time.

This simple filter has exponential response, so that the impact of any
individual measurement decays exponentially over time. Since only a
small portion of the noisy input data is ``accumulated'' in the stored
filtered value, even large diversions have little impact. Conversely,
if the noisy input data roughly approximates the ``actual'' or ideal
input value, that ideal value will be reinforced continously and will
dominate the stored value.

A consequence of the filter operation is that a step change in the input
value, e.g., perhaps caused by a physical adjustment to the sensor height,
will appear gradually in the stored filter value. After one time constant
period, 1-e^-1 or about 63% of the step change will be reflected in the
filter value, and it will approach the new value asymptotically over time.

= APPENDIX: weighted time constants =

It is observed that the SR50 can produce ``noisy'' output, rather than the
ideal clean data that would be preferred. To a large extent the noise
signal should be filtered away by the digital filter implemented in the
program, but we're also attempting to make use of the SR50 `quality''
number to influence the data processing and interpretation.

The SR50 quality value as defined is somewhat fuzzy, consisting of some
arbitrary number ranges with different meanings. In the above program,
we've attempted to use the quality to derive a weighting factor which
should be close to 1 when in the ``good'' range, smaller in the ``not
so good'' range, and much smaller in the ``poor'' range. The relation
used is just one that seems to work well enough

We've also allowed for using a similar relation to try to account for
degradation of sensor performance in high wind conditions, perhaps
accompanied by blowing snow. This effect is not well studied, and for
now we've left the wind weighting factor at 1, so that it has no effect,
but it is there to allow for future use and also just to demonstrate
what these weighting factors do and how they are used.

Specifically, the quality (qf) and wind (wf) weighting factors are
divisors to the filter algorithm time constant. A weighting factor of 1
has no effect, and the time constant is not changed. A weighting factor
of 0.5 causes the effective time constant to be twice the nominal value;
a factor of 0.01 would increase the effective time constant 100 times;
and so on.

Note that the weighting factors are applied to the filter time constant,
and not to the data values themselves. With an increased time constant,
each measurement is still used, but with a lesser impact than with the
nominal time constant. Effectively, we're giving more confidence to
data values with good quality (or perhaps when winds are calm) than to
those with lower quality (or when winds are high).

'''NOTE:''' In the process of describing the algorithm and processing,
it now seems better to perhaps drop this whole weighting business and
let the digital filter stand on its own. So long as the noise in the
SR50 data output is ``random'', varying around the desired, ideal value,
the filter alone should be sufficient to produce good results. So we'll
probably be rethinking some of the above...

@@ Line 160: / Line 160: @@
 = APPENDIX: digital filter =
-The digital filter implemented above is the simplest type of '''infinite
+The digital filter implemented above is the simplest type of '''infinite impulse response''' filter, and is equivalant to an analog low-pass filter consisting of a single resistor and a capacitor.
-The ''infinite'' part of the name reflects the fact that each measurement
+The ''infinite'' part of the name reflects the fact that each measurement (or ''impulse'') going into the filtered value remains there, albeit with diminished influence over time.
 This simple filter has exponential response, so that the impact of any

@@ Line 94: / Line 94: @@
            x = qf * wf * 60 / 3600 '' 1 hour time constant
-'''The filter operation consists of taking 1/x'th of the current measurement and 1/(1-x) of the previous filtered value and storing it as the new filtered value:'''
+'''The filter operation consists of taking 1 x'th of the current measurement and 1-x of the previous filtered value and storing it as the new filtered value:'''
            SR50_f1h = SR50_tc * x + SR50_f1h * (1 - x)

@@ Line 73: / Line 73: @@
        if (SR50_cm <> 0) AND (SR50_Q <> 0) then
-The sensor quality value is used to derive a '''quality weighting
+The sensor quality value is used to derive a '''quality weighting factor''' using an ad hoc and arbitrary function, with the goal of placing higher confidence in the data when the quality is between 162 and 210, and lower confidence for greater values:
            qf = 162 ^ 2 / SR50_Q ^ 2
-A '''wind weighting factor''' is defined for accounting for sensor
+A '''wind weighting factor''' is defined for accounting for sensor problems in high winds, but is not yet used:
            wf = 1
@@ Line 94: / Line 90: @@
 output in the new ''SR50'' data table.
-The weighted 1-hour time constant is factored by the 1-minute measurement
+The weighted 1-hour time constant is factored by the 1-minute measurement scan to derive a working fraction x:
            x = qf * wf * 60 / 3600 '' 1 hour time constant
-'''The filter operation consists of taking 1/x'th of the current measurement
+'''The filter operation consists of taking 1/x'th of the current measurement and 1/(1-x) of the previous filtered value and storing it as the new filtered value:'''
            SR50_f1h = SR50_tc * x + SR50_f1h * (1 - x)

@@ Line 8: / Line 8: @@
 and the surface of the bare ground or the accumulated snow pack.
-The SR50 is a ``smart'' sensor with onboard processing, and interfaces to
+The SR50 is a ''smart'' sensor with onboard processing, and interfaces to
 the datalogger using the serial SDI12 protocol.  Various commands are
-supported, but for our purposes the ``M'' or ``M1'' commands are key.  Both
+supported, but for our purposes the ''M'' or ''M1'' commands are key.  Both
 commands cause the sensor to take several readings and return the average
-distance to the target; the latter command additionally returns a ``quality''
+distance to the target; the latter command additionally returns a ''quality''
 value.
@@ Line 26: / Line 26: @@
 = existing CR1000 programming for Bullen and Foothills projects =
-The sensor is read once per minute using the ``M'' command, and
+The sensor is read once per minute using the ''M'' command, and
 including a multiplier of 100 to convert output from m to cm:
@@ Line 48: / Line 48: @@
 = proposed CR1000 programming changes =
-The sensor is read once per minute using the ``M1'' command:
+The sensor is read once per minute using the ''M1'' command:
        SDI12Recorder(SR50,SR50_CNTRL,0,"M1!",100.0,0)
-This instruction outputs two values into the ``SR50'' array variable,
+This instruction outputs two values into the ''SR50'' array variable,
-which is further expanded into ``SR50_cm'' for distance and ``SR50_Q''
+which is further expanded into ''SR50_cm'' for distance and ''SR50_Q''
-for quality.  (In addition, an alias is provided for ``DT'' for output
+for quality.  (In addition, an alias is provided for ''DT'' for output
 in the DailyRaw table.)
@@ Line 63: / Line 63: @@
 The previous snow depth calculation and data output tables are not
-changed, with the ``SD'' value averaged in the HrlyAtms and Daily
+changed, with the ''SD'' value averaged in the HrlyAtms and Daily
 tables:
@@ Line 92: / Line 92: @@
 The time constants are hard-coded in the program to go along with the
 named filter variables, SR50_f1h, SR50_f4h, and SR50_f12h, which are
-output in the new ``SR50'' data table.
+output in the new ''SR50'' data table.
 The weighted 1-hour time constant is factored by the 1-minute measurement
@@ Line 171: / Line 171: @@
 filter consisting of a single resistor and a capacitor.
-The ``infinite'' part of the name reflects the fact that each measurement
+The ''infinite'' part of the name reflects the fact that each measurement
-(or ``impulse'') going into the filtered value remains there, albeit
+(or ''impulse'') going into the filtered value remains there, albeit
 with diminished influence over time.
 This simple filter has exponential response, so that the impact of any
 individual measurement decays exponentially over time.  Since only a
-small portion of the noisy input data is ``accumulated'' in the stored
+small portion of the noisy input data is ''accumulated'' in the stored
 filtered value, even large diversions have little impact.  Conversely,
-if the noisy input data roughly approximates the ``actual'' or ideal
+if the noisy input data roughly approximates the ''actual'' or ideal
 input value, that ideal value will be reinforced continously and will
 dominate the stored value.
@@ Line 191: / Line 191: @@
 = APPENDIX: weighted time constants =
-It is observed that the SR50 can produce ``noisy'' output, rather than the
+It is observed that the SR50 can produce ''noisy'' output, rather than the
 ideal clean data that would be preferred.  To a large extent the noise
 signal should be filtered away by the digital filter implemented in the
-program, but we're also attempting to make use of the SR50 `quality''
+program, but we're also attempting to make use of the SR50 ''quality''
 number to influence the data processing and interpretation.
@@ Line 200: / Line 200: @@
 arbitrary number ranges with different meanings.  In the above program,
 we've attempted to use the quality to derive a weighting factor which
-should be close to 1 when in the ``good'' range, smaller in the ``not
+should be close to 1 when in the ''good'' range, smaller in the ''not
-so good'' range, and much smaller in the ``poor'' range.  The relation
+so good'' range, and much smaller in the ''poor'' range.  The relation
 used is just one that seems to work well enough
@@ Line 228: / Line 228: @@
 it now seems better to perhaps drop this whole weighting business and
 let the digital filter stand on its own.  So long as the noise in the
-SR50 data output is ``random'', varying around the desired, ideal value,
+SR50 data output is ''random'', varying around the desired, ideal value,
 the filter alone should be sufficient to produce good results.  So we'll
 probably be rethinking some of the above...