|
|
| (One intermediate revision by the same user not shown) |
| Line 32: |
Line 32: |
| | | | |
| | ==Campbell/Comm.pm== | | ==Campbell/Comm.pm== |
| | + | |
| | | | |
| | This module becomes part of a Perl program via the '''use''' or '''require''' | | This module becomes part of a Perl program via the '''use''' or '''require''' |
| Line 70: |
Line 71: |
| | | | |
| | ==querylogger.pl== | | ==querylogger.pl== |
| − | This perl script was written to exercise and test the Campbell::Comm | + | This perl script was written to exercise and test the Campbell::Comm |
| − | module, but may be useful in its own right as a way to interface with
| |
| − | dataloggers in batch (i.e., non-interactive) form. Each method in the
| |
| − | Campbell::Comm module should be represented by a querylogger command
| |
| − | as specified on the command line.
| |
| − | | |
| − | The interface is described if run with no arguments:
| |
| − | | |
| − | $ querylogger
| |
| − | syntax: "$ /home/ken/bin/querylogger OPTIONS SITE COMMAND ...
| |
| − | where OPTIONS are
| |
| − | --quiet suppress output to STDOUT
| |
| − | -q same as above
| |
| − | --logfile=f print everything to file f
| |
| − | -l f same as above
| |
| − | --comm=s use rc file section Campbell::Comm-s, or
| |
| − | -c s use rc file section Campbell::Comm-s
| |
| − | --identify identify site in output (default)
| |
| − | --noidentify don't identify site in output
| |
| − | where SITE is one of: site, slope_mtn,
| |
| − | and COMMANDs may include:
| |
| − | status list status strings
| |
| − | time show time (setting not yet supported)
| |
| − | backup=n n=number of output arrays to spew (no limit)
| |
| − | MPTR=loc loc=output storage location (no limit)
| |
| − | data=n n=number of data records to get
| |
| − | binary=n n=number of data locations to get
| |
| − | program print all logger programming
| |
| − | flags optionally toggle specified flag(s), show flags
| |
| − | ports show ports (toggle not yet supported)
| |
| − | memory show mode A memory settings
| |
| − | signatures show mode B signature settings
| |
| − | input=n[,m][,r-s] show input storage locations
| |
| − | capture=[f] end previous capture, capture output to file f
| |
| − | get_time_flags experimental using F command
| |
| − | | |
| − | The querylogger (and loggerdata) scripts assume a specific system
| |
| − | configuration of sites and ''rc'' files; see below...
| |
| − | | |
| − | | |
| − | ==loggerdata.pl==
| |
| − | | |
| − | The '''loggerdata.pl''' script is the workhorse of WERC's data
| |
| − | gathering system from CR10X loggers, and besides the '''querylogger.pl'''
| |
| − | script (see above) is the only script we've written around the Campbell::Comm
| |
| − | module.
| |
| − | | |
| − | See below for the [[#data site architecture]] assumed by the system.
| |
| − | | |
| − | Like [[#querylogger.pl]], '''loggerdata.pl''' runs in
| |
| − | a non-interactive or ''batch'' mode. But where the former program is
| |
| − | driven according to the set of commands provided on the command line, the
| |
| − | latter is hard-wired to follow a set sequence of commands.
| |
| − | | |
| − | As with any computer program, the details of what this program does are
| |
| − | spelled out in its source code. The following steps are from memory, and
| |
| − | should be confirmed...
| |
| − | | |
| − | * a connection is established to the datalogger
| |
| − | * several ''escape characters'' are sent in case the datalogger has been left in an edit or other state
| |
| − | * the datalogger command prompt, '''"*"''' is obtained
| |
| − | * the "next data memory location" ('''DSP''') is retrieved from the site state on disk (not on the logger)
| |
| − | * the datalogger's current data memory position ('''MPTR''') is obtained
| |
| − | * the '''G''' logger telecommunications command is used to set the '''MPTR''' to the last '''DSP''' value
| |
| − | * data is requested in either ASCII (the '''D''' command) or binary (the '''F''' command) mode
| |
| − | * incoming data is integrity-checked using checksums or ''signatures''
| |
| − | * confirmed-good data is stored on disk
| |
| − | * update the stored '''DSP''' state information
| |
| − | * repeat the above data query steps until:
| |
| − | ** the '''DSP''' value reaches the logger's '''MPTR'''
| |
| − | ** the query times out
| |
| − | ** the allowed processing time for the session elapses
| |
| − | * query the datalogger's time
| |
| − | ** if the logger time is off by more than a specified amount (e.g., 10 seconds), set the time
| |
| − | * if the logger program hasn't been downloaded for greater than a specified time (e.g., 10 days), download the program
| |
| − | ** store the logger program to a '''version control system''' (RCS)
| |
| − | * query and store the datalogger ''flags'' settings
| |
| − | * disconnect from the remote logger
| |
| − | * compose a ''log message'' summarizing the session
| |
| − | ** print the log message if the session has been run by a user
| |
| − | ** append the log message to the site log file
| |
| − | | |
| − | In summary, the '''loggerdata.pl''' program:
| |
| − | | |
| − | * tries to download data from the last available location
| |
| − | ** all data retrieved is integrity-checked, then stored
| |
| − | * checks (& maybe sets) the time
| |
| − | * maybe downloads the datalogger's current program
| |
| − | | |
| − | In general, we make every attempt to ensure that data is downloaded as
| |
| − | it exists on the datalogger, and safely stored on the data-gathering
| |
| − | computer. Data is not skipped; in particular, if a remote site has "fallen
| |
| − | behind", the program will patiently and persistently try to bring the
| |
| − | local (on-computer) data store in line with the remote logger.
| |
| − | | |
| − | The datalogger's time setting is of particular importance mainly so that
| |
| − | connection times can be kept ''in synch'', assuming that the radios are
| |
| − | powered on only at certain times. Of course, another benefit in checking/setting
| |
| − | the logger's time is that the stored data timestamps are meaningful in an
| |
| − | absolute sense -- but note that this could also be achieved by only checking
| |
| − | (and not setting) the time.
| |
| − | | |
| − | | |
| − | | |
| − | | |
| − | | |
| − | | |
| − | ==data site architecture==
| |
| − | | |
| − | THIS IS A STUB .... NEEDS WORK!
| |
Campbell Scientific's CR10 family of dataloggers has been heavily used by WERC
and many other research groups for years. Though now superseded by the CR1000
line of loggers, the CR10 line will likely be in service for years to come.
The CR10 loggers were provided with complete documentation, both for developing
the datalogging programs themselves and for interfacing with other systems via
serial communications. At WERC, we used this documentation to develop code in
the Perl programming language for talking to these datalogging systems, with
connections via serial radios (FreeWave) and the Internet.
Note that Campbell provides the LoggerNet (formerly PC208 and other)
programs to interface dataloggers to PCs; WERC's Campbell::Comm and related
programs serve a more limited function than LoggerNet/PC208, being primarily
concerned with reliably downloading data and checking/setting the time.
WERC's code is posted online at http://www.uaf.edu/water/staff/irving/csi-code/,
including a README file.
This system has been in use for several years, downloading data from up to 50
remote loggers hourly in several different radio networks.
This code has not been packaged for distribution, or automatic
installation, but it is intended to be available under the GNU Public
License (GPL).
Technical descriptions of the primary components follow...
Note that this Perl code has been written, tested, and run only on Linux
computers; Perl itself is very portable across platforms, but porting
problems (e.g,. to Windows OS family) might be encountered in such areas
as process management (fork, exec), alarm timing, and possibly others.
Campbell/Comm.pm
This module becomes part of a Perl program via the use or require
keywords. It provides the machinery to connect and send/receive commands
to/from a remote datalogger using Perl's object-oriented programming
methodology. Individual methods are provided corresponding to some/most
of the telecommunications commands documented in the Campbell Scientific
CR10 family manuals.
Another Perl module is needed in order to interact with remote dataloggers,
and that is Net::Telnet, which is included in the standard Perl distribution.
This module allows automating the telnet protocol, but boils down to being
able to send commands to the remote system and then wait for and handle
responses -- or no response, if that's necessary.
Besides the nominal TCP/IP connections, Net::Telnet is capable of supporting
interfaces using a direct serial or modem connection. This usage is
implemented in the (or various...) open() methods.
A bug in Net::Telnet (this is my view; it may be considered a feature
by the authors...) requires a patch; this should be represented in the
Net::Telnet version included in the above tree. Specifically, Net::Telnet
goes out of its way to handle timeouts using wall time (i.e., absolute
time as measured on a wall clock) rather than relative inactivity time
as used in the underlying select() system function. In our case the latter
form is required due to the many levels and sources of latency in the
communication system.
The Perl Expect module might provide a good alternative to Net::Telnet if
Campbell::Comm were to be rewritten. Both provide the key functionality
of being able to wait for and selectivly match on responses from the
remote system.
Due to the specific need to communicate with dataloggers via FreeWave
radios, the Campbell::Comm module was written with this capability perhaps
too implicitly. A rewrite (anticipated) should remove any code not
directly concerned with the Campbell dataloggers.
querylogger.pl
This perl script was written to exercise and test the Campbell::Comm
