Problems sending data with Arduino MKR1400 GSM and MQTT | Arduino | Forum

Avatar

Please consider registering
Guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —






— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed sp_TopicIcon
Problems sending data with Arduino MKR1400 GSM and MQTT
No permission to create posts
April 12, 2018
11:38 am
Avatar
eplantes

Silver
Members
Forum Posts: 18
Member Since:
April 12, 2018
sp_UserOfflineSmall Offline

Hello.

Once the purchase of the commercial license has been made, we have begun to carry out the first tests with unexpected results.

Our sensor network is made up of Arduino MKR 1400 GSM type devices. The fact of choosing as a means of GSM transmission and not Wifi is simple. As the crops are far from urbanization and the distances between crops can be considerable, (because they are in the middle of the field), we did not want them to depend on a Wifi network or the need to put repeaters due to the coverage. In that sense, GSM technology, although old, seemed the best option.

However, the choice of this has certain limitations. Due to the large number of samples that have to be sent during each cycle (1 per second, 3 minutes, total 180 samples), we have used the MQTT protocol for the implementation of the sample shipment. From what I understand, this protocol being light, is the most appropriate to send data in real time. Therefore our arduino program establishes a first connection with the server, and once established, it starts sending data once per second for three minutes.

However, due to the graphics obtained in our channels, the results are not as expected. Of 180 samples, about 130 are received more or less, the rest seems to be lost. To others, when downloading the data of the channel we see that many of them arrive with several seconds of difference, instead of a second that is as it should be.

It gives the sensation that the packages do not arrive by what is the own means of transmission. The problem is that we need these 180 samples to do later an optimization calculation of non-linear functions in Matlab. Doing it at the device level (at the Arduino level), would bring many complications, such as accuracy, calculation capacity and processing speed.

I understand that the solution is not simple. Nevertheless I am waiting for your comments.

This is my arduino program

#####################################

// This example uses an Arduino MKR GSM 1400 board
// to connect to shiftr.io.
//
// IMPORTANT: This example uses the new MKRGSM library.
//
// You can check on your device after a successful
// connection here: https://shiftr.io/try.
//
// by Sandeep Mistry
// https://github.com/256dpi/arduino-mqtt

#include <MKRGSM.h>
#include <MQTT.h>
#include <ArduinoLowPower.h>

const char pin[] = "NULL";
const char apn[] = "MY_APN";
const char login[] = "NULL";
const char password[] = "NULL";

const char* serverMQTT = "mqtt.thingspeak.com";
unsigned long lastConnectionTime = 0;
char mqttUserName[] = "GSM1"; // Can be any name.
char mqttPass[] = "######MYPASS######"; // Change this your MQTT API Key from Account > MyProfile.
char writeAPIKey[] = "########MYWRAPIKEY########"; // Change to your channel Write API Key.
long channelID = MYCHANNEL;

static const char alphanum[] ="0123456789"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"abcdefghijklmnopqrstuvwxyz"; // For random generation of client ID.

 

GSMSSLClient net;
GPRS gprs;
GSM gsmAccess;
MQTTClient client;

unsigned long lastMillis = 0;
static bool toSleep = false;
int counter = 0;
const int TIME_SLEEP = 15 * 60 * 1000; // 15 * 60 * 1000

void connect() {
// connection state
bool connected = false;
char clientID[10];

Serial.print("connecting to cellular network ...");

// After starting the modem with gsmAccess.begin()
// attach to the GPRS network with the APN, login and password
while (!connected) {
  if ((gsmAccess.begin(pin) == GSM_READY) &&
    (gprs.attachGPRS(apn, login, password) == GPRS_READY)) {
    connected = true;
 } else {
   Serial.print(".");
   delay(1000);
 }

 }

Serial.print("Attempting MQTT connection...");
// Generate ClientID
 for (int i = 0; i < 8; i++) {
   clientID[i] = alphanum[random(51)];
 }
Serial.print("
Conectando al servidor Mqtt...");
while (!client.connect(clientID, mqttUserName, mqttPass)) {
Serial.print(".");
delay(5000);

}

 Serial.println("
connected!");

 //client.subscribe("/hello");
 // client.unsubscribe("/hello");
}

void messageReceived(String &topic, String &payload) {
  Serial.println("incoming: " + topic + " - " + payload);
}

void setup() {
  Serial.begin(115200);

  // Note: Local domain names (e.g. "Computer.local" on OSX) are not supported by Arduino.
  // You need to set the IP address directly.
  client.begin(serverMQTT, 8883, net);
  client.onMessage(messageReceived);

  connect();
}

void loop() {
  client.loop();

  if (!client.connected()) {
    connect();
  }

  // publish a message roughly every second.
  if (millis() - lastMillis >= 1000) {
     mqttpublishSingle();
     counter++;
     if (counter >= 180) {
       toSleep = true;
       counter = 0;
     }
    lastMillis = millis();
  }

  if (toSleep) {
    LowPower.sleep(TIME_SLEEP);
    toSleep = false;
  }
}

 

void mqttpublishSingle() {

int t = counter;
String data = String(t, DEC);
int length = data.length();
char msgBuffer[length];
data.toCharArray(msgBuffer,length+1);
Serial.println(msgBuffer);

// Create a topic string and publish data to ThingSpeak channel feed.
//String topicString ="channels/" + String( channelID ) + "/publish/"+String(writeAPIKey);
String topicString ="channels/" + String( channelID ) + "/publish/fields/field8/"+String(writeAPIKey);
length=topicString.length();
char topicBuffer[length];
topicString.toCharArray(topicBuffer,length+1);

client.publish( topicBuffer, msgBuffer);

lastConnectionTime = millis();

}

#################################

 

Thank you very much

April 12, 2018
3:39 pm
Avatar
JasonW

MathWorks
Members
Forum Posts: 54
Member Since:
October 21, 2016
sp_UserOfflineSmall Offline

I suspect your application is being affected by the 1 second update limit.  Because you are publishing at exactly 1000ms intervals, there is no room for error when a communication delay occurs.  There are few layers between your Arduino and your ThingSpeak channel.  If one of those layers introduces a slight delay, then the next data point may get rejected because it hits the ThingSpeak api before one second has elapsed.  Try changing the publish interval to 1500ms and see if it makes a difference.  I realize this is counter productive if your device is running from a battery as the GSM radio will be on longer.  However it will be something to try.

I would also like to mention the bulk update feature which seems like a perfect fit for your application.  With a bulk update, data can be collected over time, then uploaded with a single api call.  The transaction is only available using the REST interface. 

Here is a link to the bulk update documentation.  Be sure to check out the Arduino MKR 1000 example link near the end of the page

https://www.mathworks.com/help/thingspeak/bulkwritejsondata.html 

   

April 13, 2018
1:52 pm
Avatar
eplantes

Silver
Members
Forum Posts: 18
Member Since:
April 12, 2018
sp_UserOfflineSmall Offline

Hello JasonW.

Thank you very much for your comment. I will try to change the publish interval between request, for example from 1 second to 1,5 or 2 seconds and see if some change happens.

On another hand, I was being testing the bulk update feature in this same scenario (arduino over GSM network) a few weeks ago, but unfortunalety I experimented some problems too. For some strange reason, I only can send bulk updates when the size packet is about 256 bytes or less, but not longer. Every time I send I packet with size longer than 256 bytes, I receive an error. I don't know if this error is caused by the library (MKRGSM) or is a limitation of the GSM chip. This supose a serious inconvenience for me, because I need wait 15 seconds or more to send the next bulk update.

I can attach some logs with the erros showed, but I supose it will be better open a new topic for explain that.

Thank you very much for your comments

April 24, 2018
9:34 am
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 861
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

I'm pretty sure there is at least a limitation in the library, if not in the chip as well.
I see you opened a new thread here: https://community.thingspeak.com/forum/arduino/testing-bulk-update-feature-with-arduino-mkr-gsm/#p5752
Sorry for the repeat.

You can send multiple packets in one request.
For example if your POST data was:
{
"write_api_key": "WRITE_API_KEY",
"updates": [{
"created_at": "2018-01-30 10:26:2 -0500",
"field1": "1.0",
"field2": "2.0"
},
{
"created_at": "2018-02-02 11:27:27 -0500",
"field1": "1.1",
"field2": "2.2",
"status": "well done"
}
]
}

You can do the following (syntax not tested - psuedocode):
client.print("{\"write_api_key\": "WRITE_API_KEY\",\"updates\": [");
client.print("{\"created_at\": \"2018-01-30 10:26:2 -0500\",\"field1\": "1.0\",\"field2\": \"2.0\"},");
client.print("{\"created_at\": \"2018-02-02 11:27:27 -0500\",\"field1\": \"1.1\",\"field2\": \"2.2\",\"status\": \"well done\" }");
client.println("]}");

...which avoids the 265 character limit. Remember to be careful with .print and .println, the protocol is sensitive to line endings.

Forum Timezone: America/New_York

Most Users Ever Online: 166

Currently Online:
27 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

rw950431: 272

Vinod: 240

piajola: 95

turgo: 70

vespapierre: 63

Adarsh_Murthy: 62

Member Stats:

Guest Posters: 1

Members: 8665

Moderators: 1

Admins: 2

Forum Stats:

Groups: 3

Forums: 14

Topics: 1600

Posts: 5760

Newest Members:

briskpanda, Dejw0089, Yassine Ben Salah, anis, 43994554AKS, suda

Moderators: cstapels: 460

Administrators: Hans: 405, lee: 457