Go to file
Patrick Tapping 9aa31c8ca4 Add a sinktimer debugging filter 2023-08-22 19:13:33 +09:30
arduino/datalog Reset default arduino sensors, fix keepalive 2022-11-17 17:45:31 +10:30
datalogd Add a sinktimer debugging filter 2023-08-22 19:13:33 +09:30
docs Bump version to 0.3.7 2023-03-29 16:27:15 +10:30
misc Add Thorlabs PM16-xxx support 2023-06-05 14:38:45 +09:30
plugin_demos Initial version 0.1.0 2020-04-18 01:56:12 +09:30
recipes Change sensorpanel to a datafilter 2023-08-03 21:41:50 +09:30
.gitignore Add Pipfile 2022-09-21 11:53:53 +09:30
LICENSE.txt Initial version 0.1.0 2020-04-18 01:56:12 +09:30
Makefile Timestamp in isoformat. 2022-09-26 17:26:52 +09:30
Pipfile pyqtgraph sink documentation 2022-12-23 17:25:03 +10:30
README.rst Fix readme 2023-03-29 17:12:38 +10:30
change_version.sh Script to automate change of version number. 2020-05-06 16:11:36 +09:30
qtdesigner.sh Begin watchdog widget 2023-06-01 20:53:27 +09:30
readthedocs.yml Try to get api docs to rtd build... 2020-04-18 03:00:27 +09:30
registersensorwidget.py Change sensorpanel to a datafilter 2023-08-03 21:41:50 +09:30
setup.py Change systemd unit target to default (note also enable lingering to start without login) 2023-08-07 14:50:16 +09:30

README.rst

datalogd - A Data Logging Daemon
================================

datalogd is a data logging daemon service which uses a source/filter/sink plugin architecture to
allow extensive customisation and maximum flexibility. There are no strict specifications or
requirements for data types, but typical examples would be readings from environmental sensors such
as temperature, humidity, voltage or the like.

The user guide and API documentation can be read online at `Read the Docs
<https://datalogd.readthedocs.io/>`_. Source code is available on `GitLab
<https://gitlab.com/ptapping/datalogd>`_.

Custom data sources, filters, or sinks can be created simply by extending an existing
``DataSource``, ``DataFilter``, or ``DataSink`` python class and placing it in a plugin directory.

Data sources, filters, and sinks can be arbitrarily connected together with a connection digraph
described using the `DOT graph description language
<https://en.wikipedia.org/wiki/DOT_(graph_description_language)>`_.

Provided data source plugins include:
  * ``LibSensorsDataSource`` - (Linux) computer motherboard sensors for temperature, fan speed,
    voltage etc.
  * ``SerialDataSource`` - generic data received through a serial port device. Arduino code for
    acquiring and sending data through its USB serial connection is also included.
  * ``RandomWalkDataSource`` - testing or demonstration data source using a random walk algorithm.
  * ``ThorlabsPMDataSource`` - laser or light power measurement using the Thorlabs PM100 or PM400
    power meter.
  * ``PicoTC08DataSource`` - thermocouple or other sensor measurements using the Pico Technologies
    TC-08 USB data logger.

Provided data sink plugins include:
  * ``PrintDataSink`` - print to standard out or standard error streams.
  * ``FileDataSink`` - write to a file.
  * ``LoggingDataSink`` - simple output to console using python logging system.
  * ``InfluxDB2DataSink`` - InfluxDB 2.x database system specialising in time-series data.
  * ``MatplotlibDataSink`` - create a plot of data using matplotlib.
  * ``PyqtgraphDataSink`` - plot incoming data in realtime in a pyqtgraph window.

Provided data filter plugins include:
  * ``SocketDataFilter`` - bridge a connection over a network socket.
  * ``KeyValDataFilter`` - selecting or discarding data entries based on key-value pairs.
  * ``TimeStampDataFilter`` - adding timestamps to data.
  * ``AggregatorDataFilter`` - aggregating multiple data readings into a fixed-size buffer.
  * ``CSVDataFilter`` - format data as a table of comma separated values.
  * ``PolynomialFunctionDataFilter`` - apply a polynomial function to a value.
  * ``FlowSensorCalibrationDataFilter`` - convert a pulse rate into liquid flow rate.
  * ``CoolingPowerDataFilter`` - calculate power dissipation into a liquid using temperatures and
    flow rate.

See the Data Logging Recipes section in the documentation for examples of how to link various data
sources, filters, and sinks to make something useful.