Optimization scenarios

The nature of IoT projects implies that devices report on collected data at certain time intervals, sometimes these are short intervals that assume a relatively high frequency of reports, but usually there is no need to send frequent reports so we can afford to report once every few hours.

If we cite the example of the device developed as part of the tutorial "Uploading sensor readings to Google Sheets using the Micromis Base V1 board," we realize that the real time required to retrieve information from a sensor and then upload it to Google Sheets takes only 15 seconds. By default, the upload is followed by a minute of idle time for the device, which waits for the next data download and upload. The device cycle thus takes 75 seconds of which 80% is idle time.
Let's remember that this device by default works in very short cycles. What about if we assume that data is sent after an hour of inactivity? Then downloading data from the sensor and sending it to the sheet is 0.04% of the total cycle, and the remaining 99.6% of the time the device is idle. Because of such a large difference between the time required for downloading and uploading data and the time of idleness, it is worth focusing on optimizing electricity consumption during the time the device is in idle mode.

Current power consumption in the project

At the very beginning, we need to address the measurement of the current that the Micromis Base V1 board consumes during operation. Due to the fact that the power consumption is optimized during idleness, we will not focus on the elements of the graph that relate to when the data is taken from the sensors and sent to the sheet.

The chart below shows the current consumption per minute, what is important for us is the data presented after 20 seconds of operation of the device. They inform us that the device in idle mode consumes a current of 35 mA.

To visualize the current consumption of the device, we can compare the average capacity of an AA battery cell, which is 2500 mAh. Of course, one AA cell will not allow us to power the Micromis Base V1 board, so we must temporarily forget about the voltage. If the AA battery voltage was 5V and would allow the Micromis Base V1 board to be powered, then the device could operate for 71 hours in idle mode without making any optimization. Assuming that the device we designed is to send reports periodically the time of 71 hours is extremely short.

What can we do to extend that time?

Optimization of ESP32 chip operation

The ESP32 chip used in the Micromis Base V1 board has numerous features to reduce power consumption during its operation. We are talking about power modes, there are as many as 5 modes available:

• Active mode - all components of the ESP32 chip are enabled, including WiFi, Bluetooth and radio circuits - depending on the protocols running, the chip consumes from 90 to as much as 250 mA
• Modem Sleep mode - all components except WiFi, Bluetooth and radio circuits are enabled - the chip consumes from 3 to 20 mA - in this mode the chip is currently operating
• Light Sleep mode - only the CPU, ULP coprocessor and RTC circuits remain on - in this configuration the ESP32 consumes between 0.4 and 1mA
• Deep Sleep mode - only the ULP coprocessor and RTC circuits remain on, we have access to RTC memory - the ESP32 consumes only 10uA
• Hibernation mode - only the main RTC circuit, which is responsible for timekeeping, is working, we have no access to RTC memory - the ESP32 consumes only 2.5uA in this mode

Knowing the different power modes of the ESP32, we can consider which mode will be most suitable for the device's idle time in our project.

The most obvious seems to be to select hibernation mode, but it is worth remembering that by using it we do not have access to the RTC memory in which we can store information. What data can we store in RTC memory? In the case of a project using a GSM modem, it is worth storing in the memory the fact of performing the modem configuration, which needs to be done only once due to the storage of settings in the modem's internal memory.

In order to be able to use the RTC memory and save the information about the performed configuration in it, we will use the Deep Sleep mode. How to use it?

A few additional lines will be added to the code from the tutorial "Transferring sensor readings to Google Sheets using the Micromis Base V1 board." To begin with, we need to add time conversion statements from microseconds to seconds to the software, as well as set the duration of Deep Sleep mode. We use two lines to set the time conversion and the mode running time:

#define CONVERT_US 1000000 //Conversion microseconds to seconds
#define DEEP_SLEEP_TIME  60 //Deep Sleep time in seconds

The value of the CONVERT_US variable should remain the same in each case, the duration of Deep Sleep mode can be managed in the DEEP_SLEEP_TIME variable. This variable stores information about the duration of the mode, expressed in seconds. If we want there to be a 10-minute break between readings and uploads during which Deep Sleep mode will be activated, then the variable should be:

#define DEEP_SLEEP_TIME  600 //Deep Sleep time in seconds

And if we want to download and upload data at two-hour intervals then it should be:

#define DEEP_SLEEP_TIME  7200 //Deep Sleep time in seconds

Having determined the duration of Deep Sleep mode, we can move on to the creation of a variable that will store information about the performed modem configuration. In order for the variable to be stored in RTC memory, the structure of its declaration must look as follows:

RTC_DATA_ATTR typ_zmiennej nazwa_zmiennej = wartość_zmiennej

For example, the declaration of a bool Boolean variable that will store information about the modem configuration should be:

RTC_DATA_ATTR bool isModemConfiguration = false; //Variable to storage information about modem configuration

Of course, we can also include a variable of other types in the RTC memory, for a variable of type String, the variable declaration will be:

RTC_DATA_ATTR String modemIMEI = “112233445566778”;

Once we know what the variables presented above are used for, we can move on to modifying the code. To do this, simply paste the following variables at the beginning of our code:

#define CONVERT_US 1000000 //Conversion microseconds to seconds
#define DEEP_SLEEP_TIME  60 //Deep Sleep time in seconds

RTC_DATA_ATTR bool isModemConfiguration = false; //Variable to storage information about modem configuration

So that the first lines of code are:

//Include library of temperature sensor
#include <Temperature_LM75_Derived.h>
Generic_LM75 temperature; //Creating object for library

#define CONVERT_US 1000000 //Conversion microseconds to seconds
#define DEEP_SLEEP_TIME  60 //Deep Sleep time in seconds

RTC_DATA_ATTR bool isModemConfiguration = false; //Variable to storage information about modem configuration

//Defines the RX and TX pins for the cellular modem
#define RXD2 16
#define TXD2 17

And not:

//Include library of temperature sensor
#include <Temperature_LM75_Derived.h>
Generic_LM75 temperature; //Creating object for library

//Defines the RX and TX pins for the cellular modem
#define RXD2 16
#define TXD2 17

After adding new variables, we need to modify the setup and loop functions. Due to the fact that the ESP32 performs a reset after exiting Deep Sleep mode, and our device only performs a one-time reading of data and sending it to Google Sheets, we should not use the loop function, and put all instructions in the setup function.

At the very beginning we need to make the modem configuration be executed only once. To do this, we need to add a simple conditional statement that will check whether the value of the variable isModemConfiguration is false. If the value of the variable indicates that there is no modem configuration then the program must execute it and change the status of the isModemConfiguration variable to true. Each time you wake up from Deep Sleep, the modem configuration will be skipped. In order for the program to execute this instruction we need to add the following lines of code to it:

  if(!isModemConfiguration)
  {
    configurationModem(); //Configuration GSM modem
    isModemConfiguration = true; //The modem has been configured
  }

Once we are sure that the modem will be configured only once, we can move on to adding Deep Sleep mode startup at the end of the setup function, to do this we need to add two lines of code:

esp_sleep_enable_timer_wakeup(DEEP_SLEEP_TIME * CONVERT_US); //Set time to wake up
esp_deep_sleep_start(); //Start Deep Sleep

The esp_sleep_enable_timer_wakeup function is responsible for determining the time after which the ESP32 chip will exit Deep Sleep mode and return to standard operation. The function uses the DEEP_SLEEP_TIME and CONVERT_US variables specified earlier to calculate the time. Of course, we could enter the time in microseconds into the function, but using the variables is much simpler. Once you have determined the duration of Deep Sleep mode, you need to call its startup using the esp_deep_sleep_start function. It is worth remembering to put this function at the very end of the instruction, if we add any instructions in the following lines they will not be executed.

After adding the above functions, we need to move the sendUpdate function call from the loop function to the setup function. We need to add the sendUpdate call between the networkCheck function call and the start of Deep Sleep mode. Now the setup and loop functions should be:

void setup() 
{
  Serial.begin(115200); //Start main UART interface
  Serial.setTimeout(100); //Set timeout for main UART interface
  Serial2.begin(115200, SERIAL_8N1, RXD2, TXD2); //Configure and start Serial2 UART interface
  pinMode(MODEM_PWRKEY, OUTPUT); //Set MODEM_PWRKEY as output
  Wire.begin(); //Run resources for I2C temperature sensor
  Serial2.setTimeout(100); //Set timeout for Serial2 UART interface
  setupModem(); //Run GSM modem
  if(!isModemConfiguration)
  {
    configurationModem(); //Configuration GSM modem
    isModemConfiguration = true; //The modem has been configured
  }
  networkCheck(); //Check status of cellular connection
  sendUpdate(temperature.readTemperatureC(), getModemTime()); //Send data to Google Sheet endpoint
  Serial.print("ESP32 Deep Sleep for "); //Printing information in Serial Monitor
  Serial.print(DEEP_SLEEP_TIME); //Printing Deep Sleep time in Serial Monitor
  Serial.println(" seconds"); //Printing information in Serial Monitor
  esp_sleep_enable_timer_wakeup(DEEP_SLEEP_TIME * CONVERT_US); //Set time to wake up
  esp_deep_sleep_start(); //Start Deep Sleep
}

void loop() 
{
}

After making all the changes to the code, we can upload it to the Micromis Base V1 board and check its operation. During the first execution cycle of the program, the modem should be configured after startup. After that, the device takes data from the sensor and sends it to the sheet, and then puts the ESP32 chip into Deep Sleep mode. At each subsequent cycle, the program should skip configuring the modem and go straight to checking the connection to the network after turning it on.

When we are sure that the program is working properly we can move on to check the power consumption of the device during the interval between reports. As with the previous graph, we observe the data after 20 seconds of measurement.

Currently, the device's idle power consumption is 10 mA, which is 3.5x less than a device that does not use Deep Sleep mode. Analyzing the consumption using the AA battery capacity as an example, the working time of the device increases from 71 to 250 hours. Unfortunately, this is still not a satisfactory result because it's not much more than 10 days of device operation in idle mode. To further extend the operating time we need to look into optimizing the modem's power consumption.

Quectel M65 modem performance optimization

During Deep Sleep mode in the ESP32 chip most of the components are turned off, this eliminates power consumption to a large extent, but we must remember that during this mode only ESP32 componentsand not the whole deviceare turned off. It is easy to see that the program does not have any function that changes the mode of operation of the modem. What can we do to reduce the modem's power consumption when the device is idle? We have three options available to us:

• Turning off the modem
• Modem sleep
• Disabling the network functions of the modem

Each of these possibilities is characterized by different effects and different modem operation during the next reporting.low effectiveness ofthe network functions of GSM modules, we will not discuss this possibility. We will focus only on disabling and putting the modem to sleep, adding the modem disabling function is extremely simple and in the case of our project requires adding only one line of code to the setup function, but what are the differences between disabling the modem and putting it to sleep

When the modem is turned off, it disconnectsconnection to the mobile network and the active GPRS and TCP/IP threads. This means that when we turn the modem back on, we will have to wait until it connects to the network and reactivates the threads we use for data transfer.

If the modem goes to sleep, it temporarilysuspends active threads and connections, so it can quickly restore them. In addition, the modem can be woken up by an SMS message or information from the WEB, which gives you a lot of control over its operation.

To compare the power consumption of the entire board at idle in both scenarios, we need to modify the software and measure the power consumption in both cases.

At the very beginning we will add to the program the function of disabling the modem, to do this we need to add only one line of code that disables the modem:

Serial2.println("AT+QPOWD=1"); //Turn off modem

This command should be added to the setup function, between the call to the sendUpdate function and the call to the function that puts the ESP32 into Deep Sleep mode. The setup function with the added command should be:

void setup() 
{
  Serial.begin(115200); //Start main UART interface
  Serial.setTimeout(100); //Set timeout for main UART interface
  Serial2.begin(115200, SERIAL_8N1, RXD2, TXD2); //Configure and start Serial2 UART interface
  pinMode(MODEM_PWRKEY, OUTPUT); //Set MODEM_PWRKEY as output
  Wire.begin(); //Run resources for I2C temperature sensor
  Serial2.setTimeout(100); //Set timeout for Serial2 UART interface
  setupModem(); //Run GSM modem
  if(!isModemConfiguration)
  {
    configurationModem(); //Configuration GSM modem
    isModemConfiguration = true; //The modem has been configured
  }
  networkCheck(); //Check status of cellular connection
  sendUpdate(temperature.readTemperatureC(), getModemTime()); //Send data to Google Sheet endpoint
  Serial2.println("AT+QPOWD=1"); //Turn off modem
  Serial.print("ESP32 Deep Sleep for "); //Printing information in Serial Monitor
  Serial.print(DEEP_SLEEP_TIME); //Printing Deep Sleep time in Serial Monitor
  Serial.println(" seconds"); //Printing information in Serial Monitor
  esp_sleep_enable_timer_wakeup(DEEP_SLEEP_TIME * CONVERT_US); //Set time to wake up
  esp_deep_sleep_start(); //Start Deep Sleep
}

After uploading the code, we can test it, unlike the software version, which optimizes power consumption only by putting the ESP32 chip into Deep Sleep mode here we should receive a message every cycle that the modem is turned on via the PWR_KEY pin.

After measuring the power consumption, we can see that the device in the idle state with the modem turned off and ESP32 in Deep Sleep mode consumes an average of 3 mA, which, when converted to the consumption of the 2500 mAh battery cell, gives us a result of almost 35 days of operation. This is already a far better result than the 10 mA, which allows for 10 days of operation.

Currently, the modem is completely turned off for idle time, but due to the loss of connections, threading activity and the inability of an external connection to wake up the modem, this will not always be a good solution. In such a situation, putting the modem to sleep will work much better.

To implement the sleep function we will need to use the MAIN_DTR pin of the modem, which allows it to wake up, we can control it through the 26 pin of ESP32. At the very beginning, we need to go to the first lines of the code and under the line defining the ESP32 pin responsible for controlling the modem's PWR_KEY pin, then paste the following line:

const int MODEM_DTR = 26; //Define pin for modem DTR pin

After adding the line of code responsible for defining the pin, we need to go to the setup function, if there is a command responsible for disabling the modem, then remove it. In place of the command to disable the modem, add the following line:

Serial2.println("AT+QSCLK=1"); //Turn on sleep mode on modem

And between the lines with Serial2.setTimeout(100) and modemWakeup() commands, add:

pinMode(MODEM_DTR, OUTPUT); //Set MODEM_DTR as output
modemWakeup(); //Wakeup modem from sleep mode

The first of the above lines of code declares the pin declared as MODEM_DTR as output, and the second runs the modemWakeup function, which we will add in a moment. Currently, the setup function should be:

void setup() 
{
  Serial.begin(115200); //Start main UART interface
  Serial.setTimeout(100); //Set timeout for main UART interface
  Serial2.begin(115200, SERIAL_8N1, RXD2, TXD2); //Configure and start Serial2 UART interface
  pinMode(MODEM_PWRKEY, OUTPUT); //Set MODEM_PWRKEY as output
  Wire.begin(); //Run resources for I2C temperature sensor
  Serial2.setTimeout(100); //Set timeout for Serial2 UART interface
  pinMode(MODEM_DTR, OUTPUT); //Set MODEM_DTR as output
  modemWakeup(); //Wakeup modem from sleep mode
  setupModem(); //Run GSM modem
  if(!isModemConfiguration)
  {
    configurationModem(); //Configuration GSM modem
    isModemConfiguration = true; //The modem has been configured
  }
  networkCheck(); //Check status of cellular connection
  sendUpdate(temperature.readTemperatureC(), getModemTime()); //Send data to Google Sheet endpoint
  Serial2.println("AT+QSCLK=1"); //Turn on sleep mode on modem
  Serial.print("ESP32 Deep Sleep for "); //Printing information in Serial Monitor
  Serial.print(DEEP_SLEEP_TIME); //Printing Deep Sleep time in Serial Monitor
  Serial.println(" seconds"); //Printing information in Serial Monitor
  esp_sleep_enable_timer_wakeup(DEEP_SLEEP_TIME * CONVERT_US); //Set time to wake up
  esp_deep_sleep_start(); //Start Deep Sleep
}

The last item to add is a short function that will be responsible for waking up the modem by applying a high state to the MODEM_DTR pin for 20 ms and disabling sleep mode. We can add it anywhere above the setup function.

//Function to wakeup modem from sleep mode
void modemWakeup()
{
  digitalWrite(MODEM_DTR, HIGH); //Turning on MAIN_DTR pin for modem wakeup
  delay(20); //20 ms pause
  digitalWrite(MODEM_DTR, LOW); //Turning off MAIN_DTR pin for modem wakeup
  delay(100); //100 ms pause
  Serial2.println("AT+QSCLK=0"); //Disable sleep mode
}

Now we can play the program again on the Micromis Base V1 board and check the current consumption again. The measurements show that the current consumption when the device is idle is 3.2 mA, which is only 0.2 mA more, with the current consumption the capacity of the AA cell we used earlier for comparison will allow us to keep the device idle for 32 days.

32 days is quite a satisfying time! Can it be extended even further? Yes, we have one more possibility!

Disconnection of LEDs indicating the operation of the device

The LEDs on the Micromis Base V1 board allow you to check whether the board is powered and whether the modem is on and processing data. These are useful elements in day-to-day operation, however, they are not necessary in a device whose only task is to download data and then transmit it to a specific access point. To completely eliminate the current consumption generated by the LEDs embedded in the Micromis Base V1 board, it is enough to solder out the resistor marked J1 on the board.

After soldering out the resistor labeled J1, we can again take a measurement of the current consumption, now it is 0.2 mA in the idle state of the device, calculating the operating time on the example of AA batteries with a capacity of 2500 mAh, we get 520 days of operation of the device in the idle state.

This result shows us how much we can tailor the operation of the Micromis Base V1 board to our needs. Have you already created your own IoT devices that required very low power consumption? If so, how did you manage it?