NEW FEATURE: Bulk-Update a Channel Feed | Announcements | Forum

Avatar

Please consider registering
Guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —






— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed sp_TopicIcon
NEW FEATURE: Bulk-Update a Channel Feed
No permission to create posts
June 20, 2017
1:32 pm
Avatar
Hans
Natick, MA

Admin
Forum Posts: 407
Member Since:
January 18, 2011
sp_UserOfflineSmall Offline

We just released a new feature that allows you to bulk-update a channel feed with one request.

You can use the bulk-update API in certain scenarios when you want to conserve your device power. When using the bulk-update API, you can collect data over a period of time and then upload the data to ThingSpeak. Bulk-update API takes JSON or CSV encoded data and updates the channel.

Documentation: https://www.mathworks.com/help/thingspeak/bulk-update-a-channel-feed-1.html

Examples:

Let us know what you make with this capability.

Thanks,
Hans

January 16, 2018
1:14 pm
Avatar
turgo

Gold
Members
Forum Posts: 70
Member Since:
June 16, 2013
sp_UserOfflineSmall Offline

Does use of the bulk update feature reduce the number of messages billed to an account?

January 16, 2018
3:17 pm
Avatar
Hans
Natick, MA

Admin
Forum Posts: 407
Member Since:
January 18, 2011
sp_UserOfflineSmall Offline

It does not. If you send one bulk update with 10 messages, it counts as 10 messages. The idea for this feature is to allow you to save on the device side and send data all at once. This might save battery and connectivity costs.

June 18, 2018
6:16 am
Avatar
mnaveed1

Gold
Members
Forum Posts: 25
Member Since:
June 15, 2018
sp_UserOfflineSmall Offline

Hello,
I am using this example:
https://www.mathworks.com/help/thingspeak/continuously-collect-data-and-bulk-update-a-thingspeak-channel-using-an-arduino-mkr1000-board-or-an-esp8266-board.html

My serial monitor displays this output:
Attempting to connect to SSID: Xperia Z_a769
Connected to wifi
SSID: Xperia Z_a769
IP Address: 192.168.43.238
signal strength (RSSI):-47 dBm
Response code:0

Why am I getting response code 0 ? Why am I not getting anything on thingspeak? Help please

June 18, 2018
11:39 am
Avatar
Hans
Natick, MA

Admin
Forum Posts: 407
Member Since:
January 18, 2011
sp_UserOfflineSmall Offline

Are you getting any data inserted? Usually, the "0" response means that you are sending data faster than the rate limit. "0" can also mean that the data format that you sent is not valid so "0" data points were added to your channel.

June 19, 2018
9:32 am
Avatar
mnaveed1

Gold
Members
Forum Posts: 25
Member Since:
June 15, 2018
sp_UserOfflineSmall Offline

NO I am not getting any data.
This is what I have in the code:

strcat(jsonBuffer,"[{\"delta_t\":0,\"field1\":-70},{\"delta_t\":15,\"field1\":-66}]");

char data[500] = "{\"write_api_key\":\"YOUR-CHANNEL-WRITEAPIKEY\",\"updates\":[{\"delta_t\":0,\"field1\":-60},{\"delta_t\":15,\"field1\":200},{\"delta_t\":15,\"field1\":-66}]"; //Replace YOUR-CHANNEL-WRITEAPIKEY with your ThingSpeak channel write API key

As far as sending data is concerned, I have exactly copied and pasted the code in the example.
Help please..??

June 19, 2018
9:53 am
Avatar
Vinod

MathWorks
Members
Forum Posts: 242
Member Since:
May 1, 2016
sp_UserOfflineSmall Offline

What is your channel ID? Have you modified your code to include your channel's write API key?

June 19, 2018
12:50 pm
Avatar
mnaveed1

Gold
Members
Forum Posts: 25
Member Since:
June 15, 2018
sp_UserOfflineSmall Offline

yes I did include my write API key.

June 20, 2018
7:38 am
Avatar
mnaveed1

Gold
Members
Forum Posts: 25
Member Since:
June 15, 2018
sp_UserOfflineSmall Offline

Hello, So this is my complete code. I still get response code: 0. Can you please tell me what is wrong in this code ? I am using ESP8266 thing dev.

/* This example shows how to bulk update a ThingSpeak channel using an Arduino MKR1000 or an ESP8266*/

#include //Uncomment this library to work with ESP8266
#include //Uncomment this library to work with ESP8266

//#include // Comment this to work with ESP8266 board
//#include // Comment this to work with ESP8266 board

char jsonBuffer[500] = "["; // Initialize the jsonBuffer to hold data

char ssid[] = "Xperia Z_a769"; // Your network SSID (name)
char pass[] = "7c6761b56ed2"; // Your network password
WiFiClient client; // Initialize the WiFi client library

char server[] = "api.thingspeak.com"; // ThingSpeak Server

/* Collect data once every 15 seconds and post data to ThingSpeak channel once every 2 minutes */
unsigned long lastConnectionTime = 0; // Track the last connection time
unsigned long lastUpdateTime = 0; // Track the last update time
const unsigned long postingInterval = 120L * 1000L; // Post data every 2 minutes
const unsigned long updateInterval = 15L * 1000L; // Update once every 15 seconds

void setup() {
Serial.begin(9600);
// Attempt to connect to WiFi network
while (WiFi.status() != WL_CONNECTED) {
Serial.print("Attempting to connect to SSID: ");
Serial.println(ssid);
WiFi.begin(ssid, pass); // Connect to WPA/WPA2 network. Change this line if using open or WEP network
delay(10000); // Wait 10 seconds to connect
}
Serial.println("Connected to wifi");
printWiFiStatus(); // Print WiFi connection information
}

void loop() {
// If update time has reached 15 seconds, then update the jsonBuffer
if (millis() - lastUpdateTime >= updateInterval) {
updatesJson(jsonBuffer);
}

}

// Updates the jsonBuffer with data
void updatesJson(char* jsonBuffer){
/* JSON format for updates paramter in the API
* This example uses the relative timestamp as it uses the "delta_t". If your device has a real-time clock, you can provide the absolute timestamp using the "created_at" parameter
* instead of "delta_t".
* "[{\"delta_t\":0,\"field1\":-70},{\"delta_t\":15,\"field1\":-66}]"
*/
// Format the jsonBuffer as noted above
strcat(jsonBuffer,"{\"delta_t\":");
unsigned long deltaT = (millis() - lastUpdateTime)/1000;
size_t lengthT = String(deltaT).length();
char temp[4];
String(deltaT).toCharArray(temp,lengthT+1);
strcat(jsonBuffer,temp);
strcat(jsonBuffer,",");
long rssi = WiFi.RSSI();
strcat(jsonBuffer, "\"field1\":");
lengthT = String(rssi).length();
String(rssi).toCharArray(temp,lengthT+1);
strcat(jsonBuffer,temp);
strcat(jsonBuffer,"},");
// If posting interval time has reached 2 minutes, then update the ThingSpeak channel with your data
if (millis() - lastConnectionTime >= postingInterval) {
size_t len = strlen(jsonBuffer);
jsonBuffer[len-1] = ']';
httpRequest(jsonBuffer);
}
lastUpdateTime = millis(); // Update the last update time
}

// Updates the ThingSpeakchannel with data
void httpRequest(char* jsonBuffer) {
/* JSON format for data buffer in the API
* This example uses the relative timestamp as it uses the "delta_t". If your device has a real-time clock, you can also provide the absolute timestamp using the "created_at" parameter
* instead of "delta_t".
* "{\"write_api_key\":\"YOUR-CHANNEL-WRITEAPIKEY\",\"updates\":[{\"delta_t\":0,\"field1\":-60},{\"delta_t\":15,\"field1\":200},{\"delta_t\":15,\"field1\":-66}]
*/
// Format the data buffer as noted above
char data[500] = "{\"write_api_key\":\"6OF022OLH2QM50TP\",\"updates\":"; //Replace YOUR-CHANNEL-WRITEAPIKEY with your ThingSpeak channel write API key
strcat(data,jsonBuffer);
strcat(data,"}");
// Close any connection before sending a new request
client.stop();
String data_length = String(strlen(data)+1); //Compute the data buffer length
// POST data to ThingSpeak
if (client.connect(server, 80)) {
client.println("POST /channels/522490/bulk_update.json HTTP/1.1"); //Replace YOUR-CHANNEL-ID with your ThingSpeak channel ID
client.println("Host: api.thingspeak.com");
client.println("User-Agent: mw.doc.bulk-update (Arduino ESP8266)");
client.println("Connection: close");
client.println("Content-Type: application/json");
client.println("Content-Length: "+data_length);
client.println();
client.println(data);
}
else {
Serial.println("Failure: Failed to connect to ThingSpeak");
}
delay(15000); //Wait to receive the response
client.parseFloat();
String resp = String(client.parseInt());
Serial.println("Response code:"+resp); // Print the response code. 202 indicates that the server has accepted the response
jsonBuffer[0] = '['; // Reinitialize the jsonBuffer for next batch of data
jsonBuffer[1] = '\0';
lastConnectionTime = millis(); // Update the last connection time
}

void printWiFiStatus() {
// Print the SSID of the network you're attached to:
Serial.print("SSID: ");
Serial.println(WiFi.SSID());

// Print your device IP address:
IPAddress ip = WiFi.localIP();
Serial.print("IP Address: ");
Serial.println(ip);

// Print the received signal strength:
long rssi = WiFi.RSSI();
Serial.print("signal strength (RSSI):");
Serial.print(rssi);
Serial.println(" dBm");
}

This is my output:

Attempting to connect to SSID: Xperia Z_a769
Connected to wifi
SSID: Xperia Z_a769
IP Address: *************
signal strength (RSSI):-44 dBm
Response code:0

June 21, 2018
7:25 pm
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 478
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

I copied your code and changed the API keys and channel numbers to mine and it worked on a NODE MCU ESP8266. You don't have the libraries included at the top, but I assume you have that correct or it wouldn't program.

I would double check that you are using the write API key and not your user API key.

Additionally, I would suggest you try forming the post request in something like POSTMAN and see if you can get it to work there. Please let us know if you get it working.

June 22, 2018
3:49 am
Avatar
mnaveed1

Gold
Members
Forum Posts: 25
Member Since:
June 15, 2018
sp_UserOfflineSmall Offline

Hello,
Yes I did include libraries while uploading it to my board. Did you include these lines as it is except you changed write api key and channel id ?

char data[500] = "{\"write_api_key\":\"6OF022OLH2QM50TP\",\"updates\":"; //Replace YOUR-CHANNEL-WRITEAPIKEY with your ThingSpeak channel write API key

client.println("POST /channels/522490/bulk_update.json HTTP/1.1"); //Replace YOUR-CHANNEL-ID with your ThingSpeak channel ID

June 22, 2018
6:09 am
Avatar
mnaveed1

Gold
Members
Forum Posts: 25
Member Since:
June 15, 2018
sp_UserOfflineSmall Offline

Also, can you tell me how to use POSTMAN and what lines of code should I change ?

June 22, 2018
8:08 am
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 478
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

Since you gave your channel ID and API key, can I try to write to your channel with my hardware? Then I can send you the exact working code except for the wifi info.

POSTMAN is free software to let you test HTTP requests. You can do GET in your browser window, but you need something else for POST and others.
https://www.getpostman.com/
There is a POSTMAN example here https://www.mathworks.com/help/thingspeak/bulkwritejsondata.html

All the lines of code after if (client.connect(server, 80)) {
are the HTTP address and headers for the POST command.

Here is a more simple test than POSTMAN: Are you able to update your channel with a GET request? On your channel view, click on the API keys tab and copy the update channel GET on the right hand side.
Paste the address into your browser window and see if it updates the channel. (don't put the GET)

June 22, 2018
8:16 am
Avatar
mnaveed1

Gold
Members
Forum Posts: 25
Member Since:
June 15, 2018
sp_UserOfflineSmall Offline

Hello,
Yes please go ahead and write to my channel with your hardware.
And yes my channel get updated when I paste the address on my browser.

June 22, 2018
11:03 am
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 478
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

I am able to update your channel using your code.
Serial monitor:
Connected to wifi
SSID: w-iot
IP Address: 172.31.172.175
signal strength (RSSI):-73 dBm
75951: Response code:202
151705: Response code:202
228096: Response code:202

And I can see the points in your channel.

This code does not do a good job of parsing the response.
You might try this other code to get the full response:
https://www.mathworks.com/help/thingspeak/MoistureMonitor.html

Let us know what you see for the full response, if you can adapt the response function in the moisture code to the bulk code.

The only remaining thing left is the libraries.
#include

I have version 1.0.0. How about you?

June 22, 2018
12:53 pm
Avatar
mnaveed1

Gold
Members
Forum Posts: 25
Member Since:
June 15, 2018
sp_UserOfflineSmall Offline

I have included these two libraries:
#include //Uncomment this library to work with ESP8266
#include //Uncomment this library to work with ESP8266

I have arduino version 1.8.5. Can you please also check if it also works on Arduino version 1.8.5 ? Because I am still getting response code 0 no matter what.
Also the board I am using is ESP8266 thing dev.

June 22, 2018
5:22 pm
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 478
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

Yes, I am in 1.8.5.
Can you try adding the function to get to more detailed HTTP response?

July 8, 2018
9:55 am
Avatar
mnaveed1

Gold
Members
Forum Posts: 25
Member Since:
June 15, 2018
sp_UserOfflineSmall Offline

Hello,
I tried to modify the code. But I am unable to get detailed HTTP response. Here is my modified code:

/* This example shows how to bulk update a ThingSpeak channel using an Arduino MKR1000 or an ESP8266*/

#define TIMEOUT 5000
#include //Uncomment this library to work with ESP8266
#include

char jsonBuffer[500] = "["; // Initialize the jsonBuffer to hold data

char ssid[] = "Xperia Z_a769"; // Your network SSID (name)
char pass[] = "7c6761b56ed2"; // Your network password
WiFiClient client; // Initialize the WiFi client library

char server[] = "api.thingspeak.com"; // ThingSpeak Server

/* Collect data once every 15 seconds and post data to ThingSpeak channel once every 2 minutes */
unsigned long lastConnectionTime = 0; // Track the last connection time
unsigned long lastUpdateTime = 0; // Track the last update time
const unsigned long postingInterval = 120L * 1000L; // Post data every 2 minutes
const unsigned long updateInterval = 15L * 1000L; // Update once every 15 seconds

void setup() {
Serial.begin(9600);
// Attempt to connect to WiFi network
while (WiFi.status() != WL_CONNECTED) {
Serial.print("Attempting to connect to SSID: ");
Serial.println(ssid);
WiFi.begin(ssid, pass); // Connect to WPA/WPA2 network. Change this line if using open or WEP network
delay(10000); // Wait 10 seconds to connect
}
Serial.println("Connected to wifi");
printWiFiStatus(); // Print WiFi connection information
}

void loop() {
// If update time has reached 15 seconds, then update the jsonBuffer
if (millis() - lastUpdateTime >= updateInterval) {
updatesJson(jsonBuffer);
}

}

// Updates the jsonBuffer with data
void updatesJson(char* jsonBuffer){
/* JSON format for updates paramter in the API
* This example uses the relative timestamp as it uses the "delta_t". If your device has a real-time clock, you can provide the absolute timestamp using the "created_at" parameter
* instead of "delta_t".
* "[{\"delta_t\":0,\"field1\":-70},{\"delta_t\":15,\"field1\":-66}]"
*/
// Format the jsonBuffer as noted above
strcat(jsonBuffer,"{\"delta_t\":");
unsigned long deltaT = (millis() - lastUpdateTime)/1000;
size_t lengthT = String(deltaT).length();
char temp[4];
String(deltaT).toCharArray(temp,lengthT+1);
strcat(jsonBuffer,temp);
strcat(jsonBuffer,",");
long rssi = WiFi.RSSI();
strcat(jsonBuffer, "\"field1\":");
lengthT = String(rssi).length();
String(rssi).toCharArray(temp,lengthT+1);
strcat(jsonBuffer,temp);
strcat(jsonBuffer,"},");
// If posting interval time has reached 2 minutes, then update the ThingSpeak channel with your data
if (millis() - lastConnectionTime >= postingInterval) {
size_t len = strlen(jsonBuffer);
jsonBuffer[len-1] = ']';
httpRequest(jsonBuffer);
}
lastUpdateTime = millis(); // Update the last update time
}

// Updates the ThingSpeakchannel with data
void httpRequest(char* jsonBuffer) {
/* JSON format for data buffer in the API
* This example uses the relative timestamp as it uses the "delta_t". If your device has a real-time clock, you can also provide the absolute timestamp using the "created_at" parameter
* instead of "delta_t".
* "{\"write_api_key\":\"YOUR-CHANNEL-WRITEAPIKEY\",\"updates\":[{\"delta_t\":0,\"field1\":-60},{\"delta_t\":15,\"field1\":200},{\"delta_t\":15,\"field1\":-66}]
*/
// Format the data buffer as noted above
char data[500] = "{\"write_api_key\":\"Z1SA22ZDPQR37U90\",\"updates\":"; //Replace YOUR-CHANNEL-WRITEAPIKEY with your ThingSpeak channel write API key
strcat(data,jsonBuffer);
strcat(data,"}");
// Close any connection before sending a new request
client.stop();
String data_length = String(strlen(data)+1); //Compute the data buffer length
// POST data to ThingSpeak
if (client.connect(server, 80)) {
client.println("POST /channels/532528/bulk_update.json HTTP/1.1"); //Replace YOUR-CHANNEL-ID with your ThingSpeak channel ID
client.println("Host: api.thingspeak.com");
client.println("User-Agent: mw.doc.bulk-update (Arduino ESP8266)");
client.println("Connection: close");
client.println("Content-Type: application/json");
client.println("Content-Length: "+data_length);
client.println();
client.println(data);
}
else {
Serial.println("Failure: Failed to connect to ThingSpeak");
}
//delay(250); //Wait to receive the response
delay(15000);
client.parseFloat();
String resp = String(client.parseInt());
Serial.println("Response code:"+resp); // Print the response code. 202 indicates that the server has accepted the response
String answer=getResponse();
Serial.println( answer );
Serial.println("end");
jsonBuffer[0] = '['; // Reinitialize the jsonBuffer for next batch of data
jsonBuffer[1] = '\0';
lastConnectionTime = millis(); // Update the last connection time
}

void printWiFiStatus() {
// Print the SSID of the network you're attached to:
Serial.print("SSID: ");
Serial.println(WiFi.SSID());

// Print your device IP address:
IPAddress ip = WiFi.localIP();
Serial.print("IP Address: ");
Serial.println(ip);

// Print the received signal strength:
long rssi = WiFi.RSSI();
Serial.print("signal strength (RSSI):");
Serial.print(rssi);
Serial.println(" dBm");
}

String getResponse(){
String response;
long startTime = millis();

delay( 200 );
while ( client.available() 0 );
// }
client.stop();

return response;
}

Here is the output:

Attempting to connect to SSID: Xperia Z_a769
Connected to wifi
SSID: Xperia Z_a769
IP Address: ***************
signal strength (RSSI):-53 dBm
Response code:0

end

Forum Timezone: America/New_York

Most Users Ever Online: 114

Currently Online:
45 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

rw950431: 272

Vinod: 240

piajola: 95

turgo: 70

vespapierre: 63

Adarsh_Murthy: 62

Member Stats:

Guest Posters: 1

Members: 8665

Moderators: 1

Admins: 2

Forum Stats:

Groups: 3

Forums: 14

Topics: 1600

Posts: 5760

Newest Members:

ChadelelDernnida, MiromiceDernnida, BonnecDernnida, Kabinaimmakly, jeanierw69, ArambatDernnida

Moderators: cstapels: 460

Administrators: Hans: 405, lee: 457