Continuous updating of a processed data channel from a raw data channel that receives IoT data | MATLAB IoT Analytics | Forum

Avatar

Please consider registering
Guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —






— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed sp_TopicIcon
Continuous updating of a processed data channel from a raw data channel that receives IoT data
No permission to create posts
November 5, 2018
5:11 pm
Avatar
jd
New Member
Members
Forum Posts: 4
Member Since:
November 5, 2018
sp_UserOfflineSmall Offline

I have two channels. Channel 1 continuously (e.g. every 15 minutes) received raw data from an IoT application. I developed a simple Matlab analysis app that reads the data from the first channel, does some unit conversions and removal of records with missing data, and next writes the data to a second channel for visualisation. As data are continuously coming in to channel 1, also the Matlab analysis app needs to run continuously (or whenever new data have come in). How to do that, and how to avoid that all data are again written from channel 1 to channel 2?

I now did that by scheduling the Matlab code to run every 15 minutes. But that is not efficient (all data are written again every 15 minutes, while only the new data need to be added).
The problem I furthermore have is that the Matlab Analysis app gives me an error message for the ThingSpeakWrite command (“Error using NAME_OF_MY_Matlab_APP_HERE (line 56)
URL is incorrectly formed, or the requested feature is not supported in this version of ThingSpeak.”). This error does not appear when channel 2 is first cleared (data removed). In that case the Matlab analysis app works fine. But there must be a better way than each 15 minutes clearing all data in channel 2 and calculating and writing all data again. And I don’t know how to automate that from the Matlab app. Please note that the measurement interval may change later. The Matlab code should be able to handle that such that the code does not need to be adapted when the measurement interval changes.

November 6, 2018
11:34 am
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 477
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

You can use a React on Channel 1 set with the frequency 'On data Insertion'. Assuming your IoT application only updates one field at a time, you could make your MATLAB analysis app read a single feed from Channel 1 (just the last one) instead of all of the data. Or you can read only the last few minutes of data.
Then you could conditionally write the data to channel 2 only if it passes your tests.

Can you share the exact syntax for your thingSpeakWrite command where you get the error? You can to delete the API keys or hash them.

November 6, 2018
1:25 pm
Avatar
jd
New Member
Members
Forum Posts: 4
Member Since:
November 5, 2018
sp_UserOfflineSmall Offline

Thanks for the suggestion of the React.

The syntax of the line that gives the error is as follows:
thingSpeakWrite(writeChannelID,analyzedData,'WriteKey', writeAPIKey,'TimeStamp',cleanTimestamps);

The strange thing is that it works when the 'writeChannelID' channel is cleared. It gives an error when there are data in the channel.

November 6, 2018
2:23 pm
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 477
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

With some trial and error, I figured it out. I will file a request to improve that error message.
See the very very bottom of the reference page help: https://www.mathworks.com/help/thingspeak/thingspeakwrite.html
In the limitations, it describes how entries in ThingSpeak channels must have unique timestamps. So I tried writing the same time stamp twice, and didn't get your error message. Then I tried a timestamp array with two of the same timestamps in the array and I got it.
*edited*
Any nonunique timestamps in an array of timestamps will give this error, not just two repeated timestamps in the same array.

So replace the timestamps with unique timestamps (or none, and let ThingSpeak create the timestamp from when the entry is written.)
Then you wont have to clear the channel.

Thanks for pointing this one out to us, let me know if you need help creating unique timestamps or if you get more errors.

November 6, 2018
5:40 pm
Avatar
jd
New Member
Members
Forum Posts: 4
Member Since:
November 5, 2018
sp_UserOfflineSmall Offline

Thanks. That must be the reason. The limitation mentioned at the very bottom of the reference page says"All timestamps must be unique. You cannot write new data with timestamps that match existing data in the channel." But that was exactly what I do: Every 15 minutes (or whenever new raw data come in into channel 1) I rewrite all data onto channel 2, that already contains most of these data. I will need to modify my Matlab code to ensure that only data NEWER than the data already written are added. It is good to make the error message more clear.

November 6, 2018
6:45 pm
Avatar
jd
New Member
Members
Forum Posts: 4
Member Since:
November 5, 2018
sp_UserOfflineSmall Offline

Thanks. Problem solved now by making sure only new data are added to the channel. This assures the timestamps are unique.

Forum Timezone: America/New_York

Most Users Ever Online: 114

Currently Online:
34 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

rw950431: 272

Vinod: 240

piajola: 95

turgo: 70

vespapierre: 63

Adarsh_Murthy: 62

Member Stats:

Guest Posters: 1

Members: 8665

Moderators: 1

Admins: 2

Forum Stats:

Groups: 3

Forums: 14

Topics: 1600

Posts: 5760

Newest Members:

AlisaMiz, AriurinDernnida, KabinaDernnida, lakishakm3, CoreenDernnida, yeseniarowe

Moderators: cstapels: 460

Administrators: Hans: 405, lee: 457