Garden133 Firmware

Design and operation of the C++ firmware that runs the Garden133 devices.

Firmware design  

This post describes the firmware for the Garden133 system, which monitors garden soil moisture levels for automated watering control. The key components of the system are the satellite sensor units and the base station. Both of these have custom PCBAs with ESP32 microprocessors, and custom firmware written in C++.

Aspects of the firmware discussed here which might interest readers include:

• How to perform ESP32 deep sleep, and
• The module-based organization of C++ firmware using the og3¹ framework. This framework makes it (somewhat) easy to build devices that can be configured via a web interface and which work well with Home Assistant.

The previous post about radio communication of the Garden133 system discusses design of the radio packet format and use of Protocol buffers encoding. These might also be of interest.

The satellite units live in the garden to monitor soil moisture, and are powered by batteries and solar panels. Depending on the position of the state of the “debug toggle” button, the firmware for these devices either:

• Supports low-power operation, or
• Allows initial setup and configuration, and firmware update.

The satellite unit firmware is part of the Garden133 Github repository, along with circuit designs and 3D-printable designs.

The base station receives LoRa² packets from the multiple satellite units in the yard. Its firmware listens for LoRa packets from the satellite units, and automatically integrates them into Home Assistant³ using the MQTT⁴ protocol.

The base station firmware is part of the LoRa133 Github repository, also alongside circuit designs and 3D-printable designs.

Satellite unit firmware  

Low power operation  

The satellite units preserve battery power because they:

• Run only for a short time,
• Perform minimal LoRa communication,
• Keep the WiFi radio powered off,
• Put the LoRa module into sleep mode after sending data to the base station, then
• Set a timer to wake in in 5 minutes, and put the ESP32 system into “deep sleep” mode.

When the ESP32 system wakes from deep sleep, the firmware starts as if it was booting for the first time. It calls the function esp_sleep_get_wakeup_cause() to determine whether it is actually booting or whether it has been awakened from deep sleep by a timer, an external signal, or some other mechanism. This article from Random Nerd Tutorials is a good explanation.

Firmware can mark some program memory as RTC_DATA_ATTR to allow it to be preserved during deep sleep. This section of program memory is used by the satellite unit to:

• Track the number of times it has awakened since the last “real” reboot.
• Remember the WiFi MAC address used for generating the unique device-id code for the unit.
• Bookkeep an estimated real elapsed time since the last real reboot, including time the unit has been in deep sleep.
• Remember the state of the kernel filter that smooths moisture readings over time.
• Remember the state of the LoRa packet sending code, including the latest packet sequence number to use. The packet sequence number helps the base station to track packets that are dropped between the satellite and base station.

These are in addition to data stored in flash memory, which is remembered even when the unit is fully rebooted and when power is removed and then restored.

Here is the current declaration of the data preserved in RTC memory:

// This data should persist during deep sleep.
// DO NOT SET INITIALIZERS ON THESE VALUES!
RTC_DATA_ATTR struct {
  unsigned bootCount;
  byte mac[6];
  KernelFilter::State<kFilterSize> filter_state;
  unsigned code;
  satellite::PacketSender::Rtc packet_sender;
  unsigned last_wake_secs;
  unsigned expected_wake_secs;
} s_rtc;

I’m so used to making sure that all values in my code are initialized to known values that I reflexively added initializers to these RTC memory values in my first version of the code (e.g., unsigned bootCount = 0). It took me an embarrassing amount of time to realize this is why it seemed that the values were not remembered during deep sleep. They were remembered, but the initializer values overwrote them as soon as the board woke.

Things like WiFi configuration, MQTT configuration, and moisture sensor calibration are kept in flash memory, where they are remembered even when the device is power-cycled. These are configured via a web interface in “Debug mode,” as described below.

Initial configuration  

Pressing a toggle button on the Garden133 board puts it into “Debug mode.” This mode allows initial setup and configuration of the unit.

When the debug toggle is on, the board enables WiFi, MQTT, and Over-the-Air (OTA) programming. The firmware runs normally rather than going into periodic deep sleep.

If the unit has not previously had its WiFi configured (access point name, password, etc..), it becomes its own access point. If you connect your computer or phone to its WiFi network, you should automatically see the device’s web interface. You can connect the device to your WiFi network.

Here is a home page interface for a Garden133 device. There are different sections of information about the device, and a set of buttons that can be used to configure different aspects of the device.

Through its web interface, you can connect the unit directly to an MQTT server. You can calibrate the ADC⁵ readings of the moisture sensor, the solar voltage sensor, the battery voltage sensor, and the “board power” voltage sensor. You can also configure the settings used for the LoRa radio module.

Below is the web interface for configuring the translation from ADC readings to soil moisture percentages.

Once configuration is finished the device is ready to be deployed. Switch the “debug” toggle button back off, reboot the board, fasten the enclosure lid to keep the weather out, and install the unit in the garden.

Base station code  

The base station translates LoRa messages into MQTT messages, which are in turn sent via WiFi to Home Assistant via an MQTT broker. The LoRa messages use the packet format presented in Garden133 radio communications

These messages have two purposes, to:

1. Describe the satellite units and the sensor readings they provide, and
2. Send the actual sensor readings to Home Assistant.

For the first purpose, the base station generates special Home Assistant MQTT Discovery messages to describe hardware to Home Assistant. The og3x-satellite library handles this task, automatically sending messages to the MQTT broker that describe the satellite unit and all the sensor readings it provides. An example from the soil moisture sensor reading of the sensor for the satellite unit monitoring our “Raised bed #1” is shown below. It is sent to MQTT topic homeassistant/sensor/lora133b/raised-bed-1_soil_moisture/config. With an entry like this for each sensor in the device, the device is automatically added to Home Assistant.

{
  "~": "og3/lora133b",
  "avty_t": "~/connection",
  "pl_avail": "online",
  "pl_not_avail": "offline",
  "device": {
    "name": "raised-bed-1",
    "ids": "raised-bed-1_5db6",
    "mf": "c133 org",
    "mdl": "LoRa133",
    "sw": "LoRa133 v0.5.0",
    "via_device": "8813BF012878_LoRa133",
  },
  "unit_of_meas": "%",
  "stat_t": "~/raised-bed-1_5db6",
  "val_tpl": "{{value_json.soil_moisture|float|round(1)}}",
  "dev_cla": "moisture",
  "name": "soil_moisture",
  "uniq_id": "raised-bed-1_5db6_soil_moisture",
  "state_class": "measurement"
}

When a LoRa packet from this satellite unit with sensor reading values is received by the base station, it is translated to a packet like this sent to topic og3/lora133b/raised-bed-1_5db6:

{
  "dropped_packets": 87,
  "RSSI": -102,
  "temperature": 23.85,
  "humidity": 27.06,
  "soil_moisture": 92.09239,
  "fivev_voltage": 3.992871,
  "solar_voltage": 3.876855,
  "soil_ADC_counts": 1408,
  "status": 4,
  "charge": 1,
  "standby": 0,
  "battery_voltage": 4.100391
}

Here is the Home Assistant device view after this message was parsed.

Code organization  

og3 modules  

The firmware for both the satellite units and the base station are based on the og3¹ library. Here, “og” stands for original gangsta because it is an old-school C++ library and application framework, not one of those trendy YAML-based frameworks like ESPHome that the kids these days are using. This is partly because I want the control that writing firmware directly in C++ provides, and partly because I’m a programmer so my hobby projects involve real programming, dammit!

In og3, code is organized as a set of modules. A module is a unit of code that performs a specific function, and which has a defined life-cycle. A module can depend on other modules. For example, these are some of the dependencies for modules in both the Garden133 satellite unit and base station firmware:

graph LR
   B[Config] --> A[Flash]
   C[Wifi manager] --> B
   D[MQTT manager] --> C
   E[HA discovery] --> D
   F[Web server] --> C
   G[OTA manager] --> C

Module dependencies determine the order in which modules are started.

When the firmware boots, it first constructs all the modules and registers them with the ModuleSystem. Firmware typically has a single ModuleSystem.

Next, the link() method of each module is called in the order that the modules were constructed. In link(), each module grabs a reference to all other modules in the system it needs to use or that it depends on. A module obtains pointers to other modules by name (e.g., "mqtt"). In link(), each module also declares which other modules it depends on.

After link() has been called for each module, the ModuleSystem uses these module dependencies to construct the module order. For example, since everything eventually depends on configuration data in flash memory when the firmware starts, the flash module is first in the order.

The module order from the dependency graph above might be:

1. Flash. File-based storage that persists between boots and when not powered.
2. Config. This module loads and saves values for variables and parameters. These are stored as JSON in files in flash memory.
3. Wifi manager. Code that when configured connects to a given access point, or which creates its own access point if not configured or the configured values fail to work. The WiFi manager allows its configuration to be updated via a web page.
4. MQTT manager. Code that connects to an MQTT broker, and allows the configuration to the broker to be set up via a web page.
5. Web server. Code that manages HTTP requests to the web pages supported by the firmware.
6. OTA manager. Code that manages the server that supports over-the-air updates of the firmware.
7. HA discovery. Code that helps other modules register devices using the Home Assistant MQTT discovery protocol.

After the link() stage and the module are sorted, the init() method of each module is called in sorted order. Next, the start() for each module is called in sorted order.

At this point, the firmware should be ready for its normal operation. The firmware repeatedly calls update() on the ModuleSystem to run the modules. ModuleSystem::update() calls the update() function for each module, in sorted order.

The current loop() function of the base station firmware is shown below. The first line is og3::s_app.loop(), which internally calls ModuleSystem::update(). The object s_app is a HAApp, which is a wrapper for core og3 objects such as the module system, and also the basic set of modules necessary for interfacing with Home Assistant via MQTT.

void loop() {
  og3::s_app.loop();

  if (!og3::s_lora.is_ok()) {
    return;
  }

  // Try to parse a packet.
  const int packetSize = LoRa.parsePacket();
  if (!packetSize) {
    return;
  }

  // Received a packet.
  og3::s_app.log().logf("Got packet: %d bytes.", packetSize);

  // read packet
  while (LoRa.available()) {
    og3::process_lora_packets();
  }
}

Web interfaces  

The og3 framework helps automatically build web interfaces for inspecting and configuring the device. Configuration values and things like sensor readings are declared as og3::variable objects, with a set of flags which indicate whether they should be configurable via the web, saved to flash storage, and/or published via MQTT.

The top of the main web page for the base station has basic information about the device, and then a summary of the status of each satellite unit it has heard from.

Below the information about the satellite is a set of buttons for configuring different modules such as WiFi, the MQTT connection, and the LoRa radio. The configuration page for the LoRa radio is shown below.

Satellite unit main loop  

In the satellite unit, the same modules are constructed both in debug mode and in low-power mode. But in debug mode the WiFi module is set as “disabled” before it is started. This causes the WiFi manager not to turn on the WiFi hardware, and this causes all the code that depends on WiFi not to get activated. The web server, OTA server, and MQTT manager are only activated when WiFi connects, so they remain dormant in low-power mode.

The main loop waits for the LoRa radio to be ready, reads the device sensors, sends the sensor readings in a LoRa packet (and sends additional device description packets if this is the initial boot), waits an extra second for transmission to finish, then goes to sleep.

The current version of Garden133’s main loop is shown below.

void loop() {
  og3::s_app.loop();

  // Check the debug-mode toggle button again, so we can switch away from debug mode
  //  while the board is running.
  og3::s_is_debug_mode = digitalRead(og3::kDebugSwitchPin);

  if (og3::s_is_debug_mode) {
    return;  // In debug mode, all the work happens in og3::s_app.loop().
  }

  // Here, we are in "normal mode", where we normally take readings, send them via LoRa,
  //  then go into deep sleep for a while to preserve power.

  // Wait up to 10 seconds for LoRa to start-up at boot.
  // After that, if LoRa isn't running, just sleep.
  if (!og3::s_lora.is_ok()) {
    if (millis() < og3::kMsecInSec * 10) {
      return;
    }
    og3::start_sleep();
  }

  // Here, we are in "normal mode" and the LoRa radio is running.

  // Just once, read the sensors, and send a LoRa packet.
  static bool s_sent = false;
  if (!s_sent) {
    og3::s_packet_sender.update();
    s_sent = true;
  }

  // Here, we have sent a packet and may send further ones before sleeping.
  // Wait for is_sending() to be unset, meaning that all packets have been sent,
  //  then wait for 1 second and then go to sleep.
  if (og3::s_packet_sender.is_sending()) {
    return;
  }

  // Here, all packets have been sent.
  // Schedule a sleep in 1 second if not yet started.
  static bool s_sleep_started = false;
  if (!s_sleep_started) {
    s_sleep_started = true;
    og3::s_app.tasks().runIn(og3::kMsecInSec, og3::start_sleep);
    og3::s_led.blink(1);
  }
}

If you found that exciting, you will love the Github repository.

Conclusion  

This post gave an overview of the C++ firmware for the Garden133 base station and satellite devices. This firmware is similar the og3-based code for my other projects such as Plant133, but it adds support for LoRa radio and uses ESP32 deep sleep to save power. The code is MIT licensed and available in Github, including the libraries it is based on such as og3, og3x-satellite, and og3x-lora.

References