Arduino ESP32 stops posting to ThingSpeak after random times (error -301)

138 views (last 30 days)
David Evans
David Evans on 3 Feb 2021
Edited: LUCA DI NUZZO on 17 Jan 2022 at 21:34
The following code runs on an Adafruit ESP32 Feather that connects to the internet via my ASUS RT-N66U router. The ESP32 is "remote" and is accessible only via wifi.
It posts to ThingSpeak every 10 minutes and works fine for a day, sometimes a few days, but then it stops posting and returns error -301 ("failed to connect") with every attempt. It only starts posting again after a hard reboot.
I suspected heap fragmentation, but free heap is constant at 247k (after an initial quick decline from 250k) and max allocatable heap is constant at 114k from the start.
ESP32 hasn't lost wifi connectivity, since I can access the ESP32 via the router (and run the "server.on" commands).
I also have an ESP8266 posting to ThingSpeak every five minutes and it has been online for months, so the problem probably isn't with the router or ISP.
Even after the ESP32 stops posting, I can successfully manually post from a browser with https://api.thingspeak.com/update.json?api_key=xyz&field5=199, so it seems the problem is with the code.
I'm running the latest ThingSpeak library, ESP library and core (but not developmental version), and Arduino IDE.
Would appreciate suggestions on things to try or monitor.
#include <WiFi.h>
#include <WiFiClient.h>
#include <WebServer.h>
#include <ArduinoOTA.h>
#include <ESP_Mail_Client.h>
#include <esp_int_wdt.h> // for hard reboot
#include <esp_task_wdt.h>// ditto
#include "ThingSpeak.h" // "always put this last in the list of includes"
WebServer server(80); // OTA and server.on
WiFiClient client; // TS only
//**** definitions etc ****
#define SMTP_HOST "smtp.gmail.com"
#define SMTP_PORT 465
#define AUTHOR_EMAIL "xyz@gmail.com"
#define AUTHOR_PASSWORD "abc"
SMTPSession smtp;
void smtpCallback(SMTP_Status status);
ESP_Mail_Session session;
SMTP_Message message;
const char * myWriteAPIKey = "efg"; // TS
const byte deltaDecreaseCM = 30; // threshold in cm... 12" = 30.48 cm
const int distAvg = 1060; // average distance
const unsigned long myChannelNumber = 123; // TS
bool paused = false;
bool savedPaused;
bool intruder = false;
bool alarmSounded = false;
bool snowing = false;
bool snowTriggeredOnce = false;
bool distSaving = true;
byte reqdNumBreaks = 6;
byte lastTSalarmFlag;
byte snowFactor = 1;
byte savedSnowFactor;
byte snowCount;
byte saveDist[100];
byte saveIdx = 99;
int distCurrent;
int savedDistance;
int lastTScode = 200;
int wiFiFailsTS;
unsigned long numIntruders; // can be very large if beam is blocked for a long time (eg. by parked car)
unsigned long alarmTriggeredTime;
unsigned long prevTSfailTime = 0;
unsigned long startSnowingTime;
unsigned long firstSnowTrigger;
unsigned long pauseStartTime;
unsigned long pauseDuration;
//**** setup ****
void setup()
{
Serial1.begin(115200); // TF03 default rate = 115200
WiFi.begin();
while (WiFi.waitForConnectResult() != WL_CONNECTED) {
delay(5000);
ESP.restart();
}
setupMail();
server.on("/", handleRoot);
server.on("/reboot", reBootMe);
server.on("/postTS", doTSpost);
server.on("/showTS", showTScode);
server.onNotFound(handleNotFound);
ArduinoOTA.begin();
server.begin();
ThingSpeak.begin(client);
readTFxTimes(50); // clear serial1 buffer
}
//***************************************************************************************
//**** loop ****
//***************************************************************************************
void loop() {
ArduinoOTA.handle(); // this works even if posting to TS does not work
server.handleClient(); // ditto
unsigned long currTime = millis();
const unsigned long writeTSinterval = 600000UL; // post to TS every 10 min (and upon sounding alarm)
static unsigned long prevTSwriteTime = 0;
const unsigned long maxAlertInterval = 600000UL; // no duplicate alarms for 10 min after an alarm
// reset pause flag if time is up
if (paused && (currTime - pauseStartTime > pauseDuration)) {
paused = false;
}
// reset alarm flag if time is up
if (alarmSounded && (currTime - alarmTriggeredTime > maxAlertInterval)) {
alarmSounded = false;
}
readTFxTimes(1); // read TF03 once every loop
if (! paused && ! alarmSounded) { // chk for intruder, but only if not paused and not w/in 10 min of an alarm
chkForIntruder();
if (intruder && (numIntruders == reqdNumBreaks * snowFactor)) soundAlarm(); // sound alarm if sufficient number of sequential brks
}
// post to thingSpeak
if (prevTSfailTime) { // if an alarmFlag=1 write failed (posted too soon after an alarmFlag=0 post)
if (currTime - prevTSfailTime > 20000UL) { // try again after 20 sec (15.1 sec didn't seem to work on 1/27 when there was a collision)
prevTSfailTime = 0;
prevTSwriteTime = currTime;
writeThingSpeak(1, savedDistance, savedSnowFactor, savedPaused);
//this will only do one re-try. If this fails again with -401 (for whatever reason)
//it will just continue on with normal (alarmFlag=0) posts after 10 minutes.
}
} else if ((currTime - prevTSwriteTime > writeTSinterval) && (! intruder)) {
prevTSwriteTime = currTime;
writeThingSpeak(0, distCurrent, snowFactor, paused); // zero indicates no alarmFlag
}
}
//***************************************************************************************
//**** writeThingSpeak ****
//***************************************************************************************
void writeThingSpeak(byte alarmF, int distC, byte snowF, bool pausD) {
if (WiFi.status() != WL_CONNECTED) { // should already be connected, but check again anyway
wiFiFailsTS++; //this has never been > 1
while (WiFi.status() != WL_CONNECTED) {
WiFi.begin();
delay(5000);
}
}
int freeHeap = ESP.getFreeHeap();
int maxAllocatable = ESP.getMaxAllocHeap();
ThingSpeak.setField(1, distC);
ThingSpeak.setField(2, alarmF); // 0 = no intruder; 1 = intruder; 4 = manual test post
ThingSpeak.setField(3, snowF); // 1 = no snow; other = snowing
ThingSpeak.setField(4, pausD);
ThingSpeak.setField(5, lastTScode);
ThingSpeak.setField(6, freeHeap);
ThingSpeak.setField(7, maxAllocatable);
ThingSpeak.setField(8, wiFiFailsTS);
lastTScode = ThingSpeak.writeFields(myChannelNumber, myWriteAPIKey);
readTFxTimes(50); // in case the above takes "a while". 100 = about one second of reads, so 50 is about half a second
/*
https://github.com/mathworks/thingspeak-arduino
Return Codes
Value Meaning
200 OK / Success
404 Incorrect API key (or invalid ThingSpeak server address)
-101 Value is out of range or string is too long (> 255 characters)
-201 Invalid field number specified
-210 setField() was not called before writeFields()
-301 Failed to connect to ThingSpeak <-------------------------------
-302 Unexpected failure during write to ThingSpeak
-303 Unable to parse response
-304 Timeout waiting for server to respond
-401 Point was not inserted (most probable cause is the rate limit of once every 15 seconds)
0 Other error
*/
}
//**** chkForIntruder ****
void chkForIntruder() {
int deltaDist = distAvg - distCurrent;
if (distSaving) { // not currently accessible (deleted the associated server.on)
saveIdx = (saveIdx + 1) % 100;
if (deltaDist < 0) {
saveDist[saveIdx] = 0;
} else {
saveDist[saveIdx] = deltaDist;
}
}
if (deltaDist > deltaDecreaseCM) { // if distance descreases more than the limit, then there's an intruder
intruder = true;
numIntruders++; // number of sequential breaks, actually
} else {
if (snowing) {
if (millis() - startSnowingTime < 1800000UL) {
if ((reqdNumBreaks / 2 < numIntruders) && (numIntruders < reqdNumBreaks)) snowCount++;
} else { // time is up
if (! snowCount) { // if snowCount == 0, reset flag and factor
snowing = false;
snowFactor = 1;
} else { // snowCount was > 0, so need to keep checking...
startSnowingTime = millis(); // reset time, so check again later
snowCount = 0; // restart count for this new period
} // end "else" (snow count > 0)
} // end "else" (time is up)
} else { // end "if snowing"
if (snowTriggeredOnce) {
if (millis() - firstSnowTrigger > 300000UL) { // triggered once, but time expired, so re-set flag
snowTriggeredOnce = false;
} else if ((reqdNumBreaks / 2 < numIntruders) && (numIntruders < reqdNumBreaks)) { // triggered once, time not expired, meets criteria...set snowing flag, etc.
startSnowingTime = millis();
snowing = true;
snowFactor = 4;
snowTriggeredOnce = false;
distSaving = false;
} //end snowTriggeredOnce
} else if ((reqdNumBreaks / 2 < numIntruders) && (numIntruders < reqdNumBreaks)) { // not triggered yet, but meets criteria, so set triggered once flag, etc.
snowTriggeredOnce = true;
firstSnowTrigger = millis();
} // end not triggered yet but meets criteria
} // end "not snowing"
intruder = false;
numIntruders = 0;
} // end "else" distance not decreased...so no intruder, and numIntruders reset to zero
}
//**** soundAlarm ****
void soundAlarm() {
alarmTriggeredTime = millis();
alarmSounded = true;
sendMyMailNow(); //send an alert
if (snowing && (startSnowingTime - alarmTriggeredTime < 5000)) {
snowing = false;
snowFactor = 1;
}
writeThingSpeak(1, distCurrent, snowFactor, paused); // 1 indicates intruder
if (lastTScode == -401) {
prevTSfailTime = millis();
savedDistance = distCurrent;
savedSnowFactor = snowFactor;
savedPaused = paused;
}
}
//**** readTFxTimes ****
void readTFxTimes(byte numOfReads) {
for (byte i = 0; i < numOfReads; i++) {
while (! readTF03once()) { //read until a number is obtained
}
}
}
//**** readTF03once ****
bool readTF03once() {
int check; // checksum
byte uart[9]; // stores each byte of data returned by LiDAR (was int... I changed to byte)
const byte HEADER = 0x59; // data package frame header...the letter "Y" in ASCII (was int... I changed to byte)
if (Serial1.available()) { //check whether the serial port has data input
if (Serial1.read() == HEADER) { // determine data package frame header = 0x59
uart[0] = HEADER;
if (Serial1.read() == HEADER) { // determine data package frame header = 0x59
uart[1] = HEADER;
for (byte i = 2; i < 9; i++) { // store rest of data to array
uart[i] = Serial1.read();
}
check = uart[0] + uart[1] + uart[2] + uart[3] + uart[4] + uart[5] + uart[6] + uart[7];
if (uart[8] == (check & 0xff)) { // check the received data as per protocols 0xff = 0b11111111
// Not sure why bitwise and (&) is used.
distCurrent = uart[2] + uart[3] * 256; // calculate distance value
return true; //got a reading
}
}
}
}
distCurrent = 0;
return false; //didn't get a reading
}
void handleRoot() {
if (server.arg("pause") != "") { // i.e., if not zero, then user entered ...?pause=(a number)
paused = true;
pauseDuration = (unsigned long) server.arg("pause").toInt(); // in minutes
pauseStartTime = millis();
if (pauseDuration <= 0) { // if neg, do nothing
paused = false;
} else if (pauseDuration > 1200) { // if large, limit to 1200 minutes = 20 hours
pauseDuration = 1200UL;
intruder = false; // so posting to TS continues during pause
numIntruders = 0;
} else { // otherwise, use received value
intruder = false; // so posting to TS continues during pause
numIntruders = 0;
}
pauseDuration *= 60000UL; // convert minutes to milliseconds
server.send(200, "text/plain", "pausing");
} else { // not break or pause
server.send(200, "text/plain", "ESP32 eye .151");
}
}
void reBootMe() { // run with /reboot
// see e32hardReset in test_espB folder for basis of this
server.send(200, "text/plain", "reboot in 2");
delay(2000);
esp_task_wdt_init(1, true);
esp_task_wdt_add(NULL);
while (true);
}
void doTSpost() { // run with /postTS
server.send(200, "text/plain", "posting a 2 to TS");
writeThingSpeak(2, distCurrent, snowFactor, paused);
}
void showTScode() { // run with /showTS
char myCstr[15];
snprintf(myCstr, 15, "TScode=%d", lastTScode);
server.send(200, "text/plain", myCstr);
}
void handleNotFound() {
server.send(404, "text/plain", "404: Not found");
}
void smtpCallback(SMTP_Status status) {
Serial.println(status.info());
if (status.success())
{
Serial.println("----------------");
Serial.printf("Message sent success: %d\n", status.completedCount());
Serial.printf("Message sent failled: %d\n", status.failedCount());
Serial.println("----------------\n");
struct tm dt;
for (size_t i = 0; i < smtp.sendingResult.size(); i++)
{
SMTP_Result result = smtp.sendingResult.getItem(i);
localtime_r(&result.timesstamp, &dt);
Serial.printf("Message No: %d\n", i + 1);
Serial.printf("Status: %s\n", result.completed ? "success" : "failed");
Serial.printf("Date/Time: %d/%d/%d %d:%d:%d\n", dt.tm_year + 1900, dt.tm_mon + 1, dt.tm_mday, dt.tm_hour, dt.tm_min, dt.tm_sec);
Serial.printf("Recipient: %s\n", result.recipients);
Serial.printf("Subject: %s\n", result.subject);
}
Serial.println("----------------\n");
}
}
void setupMail() {
smtp.debug(0); // 0 = none
smtp.callback(smtpCallback);
session.server.host_name = SMTP_HOST;
session.server.port = SMTP_PORT;
session.login.email = AUTHOR_EMAIL;
session.login.password = AUTHOR_PASSWORD;
session.login.user_domain = "mydomain.net";
message.sender.name = "ESP Mail";
message.sender.email = AUTHOR_EMAIL;
message.subject = "Test sending plain text Email";
message.addRecipient("Someone", "phoneNum@mms.cricketwireless.net");
message.text.content = "This is simple plain text message";
message.text.charSet = "us-ascii";
message.text.transfer_encoding = Content_Transfer_Encoding::enc_7bit;
message.priority = esp_mail_smtp_priority::esp_mail_smtp_priority_normal;
message.response.notify = esp_mail_smtp_notify_success | esp_mail_smtp_notify_failure | esp_mail_smtp_notify_delay;
message.addHeader("Message-ID: <abcde.fghij@gmail.com>");
}
void sendMyMailNow() {
if (!smtp.connect(&session)) {
Serial.println("failed to connec to smtp sesh");
return;
} else if (!MailClient.sendMail(&smtp, &message)) { /* Start sending Email and close the session */
//Serial.println("Error sending Email, " + smtp.errorReason());
}
}
  10 Comments
J_R_6_0
J_R_6_0 on 14 Feb 2021
When I was getting 301 errors and WiFi connection issues, I added an ESP.restart if I had lost my WiFi connection, and often after this, the problems just continued. I then changed the WiFi connection check (in the loop() function) to just disconnect and reconnect. So far the system has been stable.
if (WiFi.status() != WL_CONNECTED) {
Serial.println("Reconnecting to WiFi...");
WiFi.disconnect();
WiFi.reconnect();
}

Sign in to comment.

Answers (1)

Vinod
Vinod on 4 Feb 2021
Thank you for trying to eliminate some of the potential issues and posting what you've tried before posting. That certainly helps me suggest some things you may not have tried.
Can you try slimming down your code to remove the WebServer and ArduinoOTA libraries from your sketch? My suspicion is there is possibly some conflict between those libraries and the WiFiClient's ability to open outgoing network connections. Maybe a resource leak on Network port connections which ultimately makes it impossible to open outgoing connections.
  40 Comments
LUCA DI NUZZO
LUCA DI NUZZO on 17 Jan 2022 at 20:55
Hi everyone,
sorry for resuming this old post. I'm stuck on exactly the same problem: after a random number of times, my ESP32 stops connecting to my WiFi and stucks on "Connecting...". Only an hard reset is able to restore the correct functioning. If I undestrand correctly this bug should have been resolved in older version of ESP core so I have updated the ESP core boards to version 2.0.2 and Thingspeak library to 2.0.0. I don't understand why my board is not working.. I have tried also using the HTTPClient but result are the same. I tried two different modems, so they are not the problem. The only different thing to the situation of David is that I send my board to light sleep after every cycle but I think that this is not the problem. Another difference is that the problem jumps out very often, like every 3 or 4 cycles.
Any idea of what is going on and how to solve this?
Thanks in advance.

Sign in to comment.

Communities

More Answers in the  ThingSpeak Community

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!