ESP32 : only able to send "field1:null" ? | ThingSpeak API | Forum

Avatar

Please consider registering
Guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —






— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed sp_TopicIcon
ESP32 : only able to send "field1:null" ?
No permission to create posts
March 13, 2018
5:59 am
Avatar
tochinet

Silver
Members
Forum Posts: 9
Member Since:
February 3, 2015
sp_UserOfflineSmall Offline

I want to send some data to thingspeak, but whatever I try, I'm only able to get logged and receive the entry-id back, but nothing gets written.

The code is 

void HTTPSend(String field1, String field2) { // Send 1 or 2 string to ThingSpeak
if (client.connect(TSserver, 80)) {
Serial.println("Connected to Thingspeak");
// Construct API request body
String body = "field1=";
body += field1;
if (field2.length()){
body += "&field2=";
body += field2;
}
body +="

"; // added because it didn't work. No effect
client.println("POST /update HTTP/1.1");
client.println("Host: api.thingspeak.com");
client.println("User-Agent: ESP8266 (nothans)/1.0");
client.println("Connection: close");
client.println("X-THINGSPEAKAPIKEY: " + writeAPIKey);
client.println("Content-Type: application/x-www-form-urlencoded");
client.println("Content-Length: " + body.length());
client.println("");
client.print(body); // No println in the original sketch?
Serial.print(body);
client.println("");  // added because it didn't work. No effect

unsigned long timeout = millis(); // added a response reception to detect errors. Get the entry_id

while (client.available() == 0) {
if (millis() - timeout > 5000) {
Serial.println(">>> Client Connection Timeout...Stopping");
client.stop();
}
}
Serial.println("Receiving Thingspeak response");
while(client.available()) {
body = client.readStringUntil('
');
Serial.print(".");
}
Serial.print(body);
}
client.stop(); // Whether connection was successful or not
}

Any idea what can be rong ? Nothing is shown in the dashboard, and the json export is a proof that the packet was sent and received, but it says nothing else than "field1":null

March 13, 2018
9:12 am
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 365
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

The format of your HTTP POST command seems to be fine, you might want to remove the (nothans) and change it to (tochicnet), but that wont change how anything works. Here are some troubleshooting steps:

Does the entry ID increase each time you write?

Can you write to the channel using a browser GET command? Here is the format, make sure you use the channel write API key.

https://api.thingspeak.com/update.json?api_key=<your_write_api_key>&field1=123

Can you output the variable you call field1 to the serial monitor just before you append it to the body to make sure something is there? I see there is already a print command for the body variable. What do you see in the serial monitor after Serial.println(body); ?  

March 13, 2018
2:19 pm
Avatar
tochinet

Silver
Members
Forum Posts: 9
Member Since:
February 3, 2015
sp_UserOfflineSmall Offline

Does the entry ID increase each time you write?

Definitely. And the server replies that number as body. 

Can you write to the channel using a browser GET command? Here is the format, make sure you use the channel write API key.

https://api.thingspeak.com/update.json?api_key=<your_write_api_key>&field1=123

Yes, it works.

Can you output the variable you call field1 to the serial monitor just before you append it to the body to make sure something is there? I see there is already a print command for the body variable. What do you see in the serial monitor after Serial.println(body); ? 

 

Depending on the values, the body is  for example field1=5.0&field2=6.0

March 14, 2018
10:29 am
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 365
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

You seem to be doing everything correct.  If your channel is public, can you share the channel number?  Otherwise can you show what the last update looks like? You can use this in your browser:

https://api.thingspeak.com/channels/<your_channelID>/feeds/last.json 

March 16, 2018
9:52 am
Avatar
tochinet

Silver
Members
Forum Posts: 9
Member Since:
February 3, 2015
sp_UserOfflineSmall Offline

Sure, its public, https://thingspeak.com/channels/160089 

Last post is (after my manual try on your request) 

{"created_at":"2018-03-13T19:17:04+01:00","entry_id":262,"field1":"123","field2":null,"field3":null}

Can it be linked with the decimal dot in the values ?

I see that the entry_id or nulls have no quotes but the field1 value has.

March 16, 2018
11:01 am
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 365
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

ThingSpeak can understand decimal values.  If you want you can convince yourself with this command in your browser

https://api.thingspeak.com/update.json?api_key=<your_write_api_key>&field1=12.3

Here are two more small things you can try.

The network strength example in the documentation does not send a line feed at the end of the post.  I see some comments in your code that might indicate you already tried this.

Also,the string for body may not be printing to the client as expected.  

Instead of client.print(body); 

You cold try hard coding the output:

client.println("Content-Length: " + length("field1=1.23));

client.print("field1=1.23"); 

And then if that works, you can try it in two parts:

client.print("field1=");

client.print(string(field1));  

March 18, 2018
12:06 pm
Avatar
tochinet

Silver
Members
Forum Posts: 9
Member Since:
February 3, 2015
sp_UserOfflineSmall Offline

Thanks for the proposal, but it won't work, length is only a method of string. It says undefined.

I tried many other variations in no particular order, and when I focused on the content type

client.println("Content-Type: application/x-www-form-urlencoded");

I tried to put the values in the URL, and it finally worked.

Got as reply (yes, changed as json in the meantime as well

Connected to Thingspeak
field1=123.23&field2=6.0Receiving Thingspeak response
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Status: 200 OK
X-Frame-Options: ALLOWALL
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, OPTIONS, DELETE, PATCH
Access-Control-Allow-Headers: origin, content-type, X-Requested-With
Access-Control-Max-Age: 1800
ETag: "e78f0xxxxxxxxxxxxxxxxxx0203"
Cache-Control: max-age=0, private, must-revalidate
Set-Cookie: request_method=POST; path=/
X-Request-Id: c4cf.....................4d
X-Runtime: 0.067653
X-Powered-By: Phusion Passenger 4.0.57
Date: Sun, 18 Mar 2018 16:04:46 GMT
Server: nginx/1.9.3 + Phusion Passenger 4.0.57

102
{"channel_id":160089,"field1":"123.23","field2":"6.0","field3":null,"field4":null,"field5":null,"field6":null,"field7":null,"field8":null,"created_at":"2018-03-18T16:04:46+00:00","entry_id":357,"status":null,"latitude":null,"longitude":null,"elevation":null}
0

 Any reason why there is a number before and after the JSON in the body ?

March 18, 2018
2:13 pm
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 365
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

Thanks for catching my bad syntax. I should have written

String fieldData ="field1=12.3";

client.println("Content-Length: " + fieldData.length());

client.print("field1=1.23"); 

I'm glad you got it working. Can you show the line(s) where you 'put the values in the URL'? 

I will look into the numbers you are getting before the JSON response, I do not see those with an update from POSTMAN. I can try it with an ESP32 later.  These are the other headers I see:

access-control-allow-headers →origin, content-type, X-Requested-With
access-control-allow-methods →GET, POST, PUT, OPTIONS, DELETE, PATCH
access-control-allow-origin →*
access-control-max-age →1800
cache-control →max-age=0, private, must-revalidate
connection →close
content-type →application/json; charset=utf-8
date →Sun, 18 Mar 2018 17:57:51 GMT
etag →"25ddfff65cd7e90df008cb339d7a0d11"
server →nginx/1.9.3 + Phusion Passenger 4.0.57
status →200 OK
transfer-encoding →chunked
x-frame-options →ALLOWALL
x-powered-by →Phusion Passenger 4.0.57
x-request-id →bbefaa13-c868-4a87-9870-813d94d89ae7
x-runtime →0.037343
 
and the JSON response is 
 
{"channel_id":nnnnnn,"field1":"2","field2":null,"field3":null,"field4":null,"field5":null,"field6":null,"field7":null,"field8":null,"created_at":"2018-03-18T17:57:51+00:00","entry_id":8,"status":null,"latitude":null,"longitude":null,"elevation":null}
 
Is there anything else you are writing the serial monitor?
 
March 19, 2018
8:05 am
Avatar
tochinet

Silver
Members
Forum Posts: 9
Member Since:
February 3, 2015
sp_UserOfflineSmall Offline

Thanks for your great and fast support. The code as I changed it to post was now as below.

I'm still considering moving back the "body" part to the request body, it's cleaner.

I'm also wondering if the
vs
vs. println could have an impact. I read somewhere that the HTTP protocol specified
but I don't think Arduino Serial does.

I may be mistaken, but at least for the first number,  it couldn't be due to some other Serial.print etc because it's part of the same loop. However, it could be some bogus content-length. For the second ("0"), there could always be another unspotted Serial.print somewhere.

void HTTPSend2TS(String field1, String field2) { // Send 1 or 2 strings to ThingSpeak
if (client.connect(TSserver, 80)) {
Serial.println("Connected to Thingspeak");
// Construct API request body
String body = "field1=";
body += field1;
if (field2.length()){
body += "&field2=";
body += field2;
}
client.print("POST /update.json?");
Serial.print("POST /update.json?");
client.print(body); // Here as parameters
client.println(" HTTP/1.1"); // End of the URL line in HTTP specs
Serial.print(body); // Here as parameters
Serial.println(" HTTP/1.1"); // End of the URL line in HTTP specs
client.println("Host: api.thingspeak.com");
client.println("User-Agent: ESP8266 (d-duino)/0.666");
client.println("Connection: close");
client.println("X-THINGSPEAKAPIKEY: " + writeAPIKey);
client.println("Content-Type: application/x-www-form-urlencoded");
// client.println("Content-Length: " + body.length());
client.println(""); // Empty line between Headers: and body
// client.println(body); // No println?
// Serial.print(body);
client.print("

");

unsigned long timeout = millis();
while (client.available() == 0) {
if (millis() - timeout > 5000) {
Serial.println(">>> Client Connection Timeout...Stopping");
client.stop();
}
}
Serial.println("
Receiving Thingspeak response");
while(client.available()) {
body = client.readStringUntil('
');
// Serial.print("."); // One dot per line replied. Replaced by a real print for debug.
Serial.print(body);
}
// Serial.print(body);
}
client.stop(); // Whether connection was successful or not
}

March 19, 2018
8:09 am
Avatar
tochinet

Silver
Members
Forum Posts: 9
Member Since:
February 3, 2015
sp_UserOfflineSmall Offline

Error in the post above, the <backslash>-r and <backslash>-n were not automatically escaped. Trying to repeat here

I'm also wondering if the "
" vs "
" vs println could have an impact.

March 19, 2018
8:14 am
Avatar
tochinet

Silver
Members
Forum Posts: 9
Member Since:
February 3, 2015
sp_UserOfflineSmall Offline

Can't apparently edit the postings, I obviously meant I'm wondering if the "<backslach>-r <backslash>-n" vs "<backslash>-n" vs "println" (instead of print) would have an impact.

March 19, 2018
4:35 pm
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 365
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

The code below works to post values.  I think the difference is that you need to convert the output of body.length() to a string.  You can also include the API key in the body, and that saves one extra header and some lines of code.

String postData="api_key=XXXXXXXXXXXXXXXX&field1=2"; //I build this programatically, see the example code in the soil moisture example.

client.println( "POST /update.json HTTP/1.1" );
client.println( "Host: api.thingspeak.com" );
client.println( "Connection: close" );
client.println( "Content-Type: application/x-www-form-urlencoded" );
client.println( "Content-Length: " + String( postData.length() ) );
client.println();
client.println( postData );

I have also observed the extra characters before and after the output for a device (esp8266), we are looking into that, and the ability to edit posts here.

March 19, 2018
5:20 pm
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 365
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

The numbers before and after the response are the size of the remaining chunks in hex.  The response includes the header:

Transfer-Encoding: chunked (wiki link)

In your case, there were 258 bytes in the response, or 102 in hex.  After the data, there were 0 bytes remaining.  This may be a device or library dependent issue, POSTMAN does not generally show the bytes in the response.  I did see it on an ESP8266 with <ESP8266WiFi.h> library.

You should consider using the ThingSpeak Communication Library.  It is compatible with ESP32, and takes care of a lot of these details for you.  There are several examples in the Arduino section of the documentation examples that show how to use the library, including the Sonar Example.  The process for ESP32 will be very similar to Arduino.  If you aren't already programming with Arduino, the ESP32 network strength example in the documentation demonstrates how to set it up.   There are also Arduino examples that come with the ThingSpeak library.

March 21, 2018
4:27 am
Avatar
tochinet

Silver
Members
Forum Posts: 9
Member Since:
February 3, 2015
sp_UserOfflineSmall Offline

Thanks for the content length tip, I'll try tonight.

I looked at the library, but in general (Arduino, I have less experience with ESP) I'm reluctant to use libraries, because each one brings extra and often hidden dependencies. My worst experience was with "SWRTC" from Leornardo Miliani, that was really nice, ... until there were issues in the use of timer 2, then differences between 3.0 and 3.1, then lower level of support, ... and after 2 years of use I had to drop it and re-engineer whay I needed myself. Of course I learned a lot in the process though.

The thingspeak library looks nice, but I didn't complete my analysis. It's  obviously pretty complex because of the support of several CPUs. One concern is whether it could interact with my other uses of the WiFi client (currently fetching a Weather Underground prediction, but I plan to add other functions). In addition, the closest example to what I want to do (Writing multiple voltages) is not compatible with ESP, and also looks bloated with compiler directives.This seriously reduces first sight attractivity.

Forum Timezone: America/New_York

Most Users Ever Online: 114

Currently Online:
34 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

rw950431: 261

Vinod: 196

piajola: 85

turgo: 70

vespapierre: 63

Adarsh_Murthy: 62

Member Stats:

Guest Posters: 1

Members: 5703

Moderators: 0

Admins: 2

Forum Stats:

Groups: 4

Forums: 17

Topics: 1313

Posts: 4565

Newest Members:

JudestAboth, concettazx4, lyndarf60, caseybo16, iodiree, emiliacz4

Administrators: Hans: 387, lee: 457