WERC datasite architecture

WERC datasite architecture

WERC uses the Linux operating system for running downloads and storing the data from the Campbell Scientific CR10 family of dataloggers.

The following setup and conventions might be viewed as a type of ad-hoc database, but the implementation is straightforward, using ordinary directories and (flat) files.

site root

A site root directory is established on the data-gathering host computer, e.g., /home/site/.

Individual sites could be found directly under this location, e.g., as:

   /home/site/SITE1/
   /home/site/SITE2/

but since we have so many different projects and locations, it makes sense to provide subdirectories for those entities to help keep things organized, and so the sites are usually defined under "area" directories:

   /home/site/AREA1/SITE1/
   /home/site/AREA1/SITE2/
   /home/site/AREA2/SITE1/
   /home/site/AREA2/SITE5/

Here we're showing two "areas" or "projects", but some of the sites contained therein have the same names. In practice, it might be preferrable to keep the site names unique, for example to allow utility programs to refer to the sites without having to specify the area.

In any case, the "area" directories don't necessarily have any assumed meaning or importance other than organization -- though they could do...

site directory

A site -- here assumed to be a local or remote location with a data logger and attached sensors -- is defined by a directory. What that directory contains depends on how things are configured; a site could just have files in its site directory, or it might include other subdirectories, e.g., var/, etc/, web/ tmp/, whatever makes sense for the intended use.

What really defines a site for the purposes here is the existence of a site rc file.

site rc file

In Unix-speak, rc files are plain text configuration files, very similar to INI files found on (older? versions of) the Windows operating system. ('rc itself might stand for "resource configuration", but other definitions can likewise be found.)

For this WERC datasite system, we chose to use the filename siterc or it's dotted (and so, "hidden") version, .siterc, for storing configuration settings.

Further, we defined a simple syntax for the site rc files:

   section:
      entry=value
      ...

where the rc file can contain any number of sections, and any number of entry=value entries under each section.

In practice, our convention has been to have an rc file section corresponding to -- and sharing the name of -- the programs that are used to gather, process, and move data on the site's behalf. Any program settings specific to the site can then be viewed and configured directly in the rc file, e.g., using a text editor -- subject to the normal user & group permissions inherent to the Linux operating system.

Here's an example siterc file from one of our active sites (with key settings changed).