Testing bulk update feature with Arduino MKR GSM | Arduino | Forum

Avatar

Please consider registering
Guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —






— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed sp_TopicIcon
Testing bulk update feature with Arduino MKR GSM
No permission to create posts
April 13, 2018
4:35 pm
Avatar
eplantes

Silver
Members
Forum Posts: 6
Member Since:
April 12, 2018
sp_UserOfflineSmall Offline

Hello.

I have been testing the bulk update feature using Arduino MKR 1400 GSM and I have experimented some problems. For some strange reason, I only can send bulk updates when the packet size is about 256 bytes or less, but not longer. Every time I send my packet with size longer than 256 bytes, I receive an error. I don't know if this error is caused by the library (MKRGSM) or is a limitation on the GSM chip. This supposes a very serious inconvenience for me, because I need to send about 180 measurements (2,2Kbytes); and the between consecutive bulk updates must be 15 seconds or more.

Can someone tell me what could be the reason of this limitation?

This is my arduino code. (For requests with packet size of 256 bytes works fine, but with longer size not).

#####################################################

 // include the GSM library
#include

// PIN number if necessary
#define PINNUMBER ""

// APN information obrained from your network provider
#define GPRS_APN "?????????" // replace with your GPRS APN
#define GPRS_LOGIN "" // replace with your GPRS login
#define GPRS_PASSWORD "" // replace with your GPRS password

// initialize the library instances
GSMClient client;
GPRS gprs;
GSM gsmAccess(true);

 

// ThingSpeak Settings
char thingSpeakAddress[] = "api.thingspeak.com";
String writeAPIKey = "MY_API_KEY_WR"; //replace with your API Key
const int updateThingSpeakInterval = 15 * 1000; // Time interval in milliseconds to update ThingSpeak (number of seconds * 1000 = interval)

// Variable Setup
long lastConnectionTime = 0;
boolean lastConnected = false;
int failedCounter = 0;

char superstring[] = "{\"write_api_key\":\"MY_API_KEY_WR\",\"updates\":[{\"delta_t\":0,\"field8\":-1000000},{\"delta_t\":1,\"field8\":-1000000},{\"delta_t\":1,\"field8\":-1000000},{\"delta_t\":1,\"field8\":-1000000},{\"delta_t\":1,\"field8\":-1000000},{\"delta_t\":1,\"field8\":-1000000},{\"delta_t\":1,\"field8\":-9999999},{\"delta_t\":1,\"field8\":-1000000},{\"delta_t\":1,\"field8\":-9999999}]}";

 

void setup() {
Serial.setTimeout(5);
Serial.begin(57600); // opens serial port, sets data rate to 9600 bps
StartGSM();
}

 

void loop() {
if (client.available()) {
char c = client.read();
Serial.print(c);
}

// Disconnect from ThingSpeak
if (!client.connected() && lastConnected){
Serial.println("...disconnected");
Serial.println();

client.stop();
}

// Update ThingSpeak
if(!client.connected() && (millis() - lastConnectionTime > updateThingSpeakInterval)) {
String ToThingSpeak = superstring;
Serial.println(ToThingSpeak);
updateThingSpeak(ToThingSpeak);
}

if (failedCounter > 3 ) {
StartGSM();
}

lastConnected = client.connected();
}

void updateThingSpeak(String tsData)
{
if (client.connect(thingSpeakAddress, 80))
{
Serial.println("[-] Enviando datos !!!! ");
client.println("POST /channels/MY_CHANNEL/bulk_update.json HTTP/1.1");
client.println("Host: api.thingspeak.com");
client.println("User-Agent: mw.doc.bulk-update (Arduino ESP8266)");
client.println("Connection: close");
client.println("Content-Type: application/json");
client.print("Content-Length: ");
client.print(tsData.length());
client.print("

");

client.print(tsData);

lastConnectionTime = millis();

if (client.connected()) {
Serial.println("Connecting to ThingSpeak...");
Serial.println();

failedCounter = 0;
} else {
failedCounter++;

Serial.println("Connection to ThingSpeak failed ("+String(failedCounter, DEC)+")");
Serial.println();
}

} else {
failedCounter++;

Serial.println("Connection to ThingSpeak Failed ("+String(failedCounter, DEC)+")");
Serial.println();

lastConnectionTime = millis();
}
}

void StartGSM() {
char server[] = "api.thingspeak.com"; // the base URL
int port = 80; // the port, 80 for HTTP

Serial.println("Starting Arduino web client.");
// connection state
boolean notConnected = true;

// Start GSM shield
// pass the PIN of your SIM as a parameter of gsmAccess.begin()
while(notConnected) {
if((gsmAccess.begin(PINNUMBER)==GSM_READY) &
(gprs.attachGPRS(GPRS_APN, GPRS_LOGIN, GPRS_PASSWORD)==GPRS_READY)) {
notConnected = false;
} else {
Serial.println("Not connected");
delay(1000);
}
}

Serial.println("connecting...");

// if you get a connection, report back via serial:
if (client.connect(server, port)) {
Serial.println("connected");
} else {
// if you didn't get a connection to the server:
Serial.println("connection failed");
}

}

################################################

 Thank you
Best regards

April 15, 2018
10:28 pm
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 218
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

I have run into buffer size limitations with Arduino for HTTP posts.  Instead of building ‘superstring ‘ in advance, I would store your field data in an array and pass the array to your posting function.  Then send one field at a time so you don’t have a big string.

 

“client.println("Content-Type: application/json");
client.print("Content-Length: ");
client.print(tsData.length());
client.print("”

 

client.print(“&field1=”+String(fieldArray[1]);

client.print(“&field2=”+String(fieldArray[2]);

client.print(“&field3=”+String(fieldArray[3]);

client.print(“&field4=”+String(fieldArray[4]);

client.println();

…and so on.  You can obviously use a loop here.

April 16, 2018
8:21 am
Avatar
eplantes

Silver
Members
Forum Posts: 6
Member Since:
April 12, 2018
sp_UserOfflineSmall Offline

Hello cstapels.

Thanks for your response but I have a doubt. According to the documentation, the time between consecutive bulk updates must be 15 seconds or more. So if I have to send 180 measurenments, I need a lot of time to send all data. In my case I don't need to send eight flieds every time, only 1 field; for this reason I thought creating a superstring it would be the best option. 

On another hand, looking at the MKRGSM library I have noticed in the GSMClient.cpp file, in the write method, that the max size packet is 256. I suposse it must be some reason of that, so I'm not sure if is not possible increase it. This could be the cause of my problem.

This is the part of the code affected:

**************************************

size_t GSMClient::write(const uint8_t* buf, size_t size)
{
  if (_writeSync) {
    while (ready() == 0);
  } else if (ready() == 0) {
    return 0;
  }

  if (_socket == -1) {
    return 0;
  }

  size_t written = 0;
  String command;

  while (size) {
    size_t chunkSize = size;

    if (chunkSize > 256) { # <----------------------------------------------------------- !!!!
      chunkSize = 256;
    }

    command.reserve(19 + chunkSize * 2);

    command += "AT+USOWR=";
    command += _socket;
    command += ",";
    command += chunkSize;
    command += ",\"";

    for (size_t i = 0; i < chunkSize; i++) {
      byte b = buf[i + written];

      byte n1 = (b >> 4) & 0x0f;
      byte n2 = (b & 0x0f);

      command += (char)(n1 > 9 ? 'A' + n1 - 10 : '0' + n1);
      command += (char)(n2 > 9 ? 'A' + n2 - 10 : '0' + n2);
    }

    command += "\"";

    MODEM.send(command);
    if (_writeSync) {
      if (MODEM.waitForResponse(10000) != 1) {
        break;
      }
    }

    written += chunkSize;
    size -= chunkSize;
  }

  return written;
}

 

******************************

thank you

April 16, 2018
11:09 am
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 218
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

I think you have got the problem figured out now.  You don't need to increase the buffer size though,  Just send each measurement separately.

It the same thing for one field.  

Send an array of measurements and times to your posting functions. Then you will never exceed the 256 byte limit. Here is pseudo code, I have not checked all the syntax- you really have to be careful with all the quotes and slashes:

int deltaTs[measurements];

float values[measurements];

posting_function(deltaTs,values){

..headers and other stuff...

client.print("{\"write_api_key\":\"+String(MY_API_KEY_WR);

client.print("\",\"updates\":[");

for (int i=0;i<measurements;i++){

client.print("{\"delta_t\":"+String(deltaTs[i])+",\"field8\":-"+String(values[i])+"},");

}

}

April 16, 2018
11:49 am
Avatar
eplantes

Silver
Members
Forum Posts: 6
Member Since:
April 12, 2018
sp_UserOfflineSmall Offline

Thanks Cstapels.

I'm going to test it!!

Thank you very much.

April 16, 2018
2:01 pm
Avatar
eplantes

Silver
Members
Forum Posts: 6
Member Since:
April 12, 2018
sp_UserOfflineSmall Offline

Hei Cstapels!

I've tested it and it works!!

At the moment I can't send all data al the same time, I don't know the reason, but I can send over 80 measurements every time. This is fantastic for me, because I can send all data in three requests. I just need to wait the precise time between requests. Once I've finished my code and I've tested it, I'll post it in this thread; maybe it can be useful for someone.

Thank you very much!

April 20, 2018
5:48 pm
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 218
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

Great news! Definitely post your finished code. And please share the final results of your project too if you can.

Forum Timezone: America/New_York

Most Users Ever Online: 114

Currently Online:
17 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

rw950431: 261

Vinod: 196

piajola: 85

turgo: 70

vespapierre: 63

Adarsh_Murthy: 62

Member Stats:

Guest Posters: 1

Members: 5703

Moderators: 0

Admins: 2

Forum Stats:

Groups: 4

Forums: 17

Topics: 1313

Posts: 4565

Newest Members:

frenk0, Jassonchulp, MatGame, Domnesich, saxenadj, concepcionoa3

Administrators: Hans: 387, lee: 457