Data disappears from chart and database | ThingSpeak API | Forum

Avatar

Please consider registering
Guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —






— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed sp_TopicIcon
Data disappears from chart and database
No permission to create posts
January 19, 2018
11:44 am
Avatar
bbruno22
New Member
Members
Forum Posts: 3
Member Since:
January 18, 2018
sp_UserOfflineSmall Offline

I have Matlab code whose output gets written to my ThingSpeak channel (ID =  379753) in three separate instances. Instance 1) writes to fields 1,3,4,5; Instance 2) writes to field 6; Instance 3) writes to field 7. It's the data logged in instance 1 that I'm having trouble with.

Data gets written successfully but eventually disappears after maybe an hour or two. When I download and view the exported CSV files, it isn't there either. I can't seem to connect the disappearance with any other task being performed by my computer. For instance, it doesn't disappear while writing to the other fields in my channel. Nor does it happen when I call for instance 1 to happen again, because at the moment, I've only got one data point each to write to fields 1,3,4,5; I've called for instance 1 to happen only once. 

While the data remains, I've also noticed that these fields take several moments to show data, something like 10s. Whereas if I refresh my page, all the other fields re-populate almost instantly. Any help with this would be much appreciated. 

January 19, 2018
11:28 pm
Avatar
piajola

Gold
Members
Forum Posts: 87
Member Since:
October 12, 2015
sp_UserOfflineSmall Offline

Hi bbruno22,

As your data seems public I downloaded some with: 

api.thingspeak.com/channels/379753/feeds.csv?start=2018-01-19 00:00:00&end=2018-01-19 23:59:59

obtaining

created_at,entry_id,field1,field3,field4,field5,field6,field7

2018-01-19 00:53:58 UTC,303,,,,,0.01663893511805415,

2018-01-19 01:01:39 UTC,323,,,,,,59

2018-01-19 01:53:58 UTC,304,,,,,0.01668520578309842,

2018-01-19 02:05:22 UTC,324,,,,,,59

2018-01-19 02:53:50 UTC,305,,,,,0.01673640169033585,

2018-01-19 03:08:47 UTC,325,,,,,,59

2018-01-19 03:53:28 UTC,306,,,,,0.008356545955453048,

2018-01-19 04:12:03 UTC,326,,,,,,58.98983045762713

2018-01-19 04:53:30 UTC,307,,,,,0.01668984697099557,

2018-01-19 05:15:33 UTC,327,,,,,,58.99861113333333

2018-01-19 05:53:18 UTC,308,,,,,0.0167061116647121,

2018-01-19 06:20:23 UTC,328,,,,,,58.98722216666667

2018-01-19 06:53:13 UTC,309,,,,,0.08326394677719125,

2018-01-19 07:25:25 UTC,329,,,,,,58.98222226666667

2018-01-19 07:53:24 UTC,310,,,,,0.08375209373545039,

2018-01-19 08:29:58 UTC,330,,,,,,59

2018-01-19 08:52:37 UTC,311,,,,,0.008270158522242643,

2018-01-19 09:39:11 UTC,331,,,,,,59

2018-01-19 09:54:19 UTC,312,,,,,0.01666666666666667,

2018-01-19 10:47:19 UTC,332,,,,,,58.99055565

2018-01-19 10:52:37 UTC,313,,,,,0.01690617074210293,

2018-01-19 11:48:29 UTC,333,,,,,,59

2018-01-19 11:52:37 UTC,314,,,,,0.01666666666666667,

2018-01-19 12:48:30 UTC,334,,,,,,58.98722224999999

2018-01-19 13:48:30 UTC,335,,,,,,59

2018-01-19 14:28:30 UTC,336,,,,,,59

observe entry_id (the order of reception) and created_at (timestamp). These 2 values must be correlated and with the same order ... and they are not. Perhaps that is the problem, I think you are having troubles sending the timestamps

I hope this is some helpSmile

p.

January 21, 2018
11:08 am
Avatar
bbruno22
New Member
Members
Forum Posts: 3
Member Since:
January 18, 2018
sp_UserOfflineSmall Offline

piajola,

Much thanks for your reply. It is strange to me that the entry IDs bounce around like that, however, the data that you pulled up is from the charts that I am having no trouble with. I wonder why this entry_id/timestamp issue would cause problems for one chart and not for others. I also don't know how I would troubleshoot this issue. If it helps, here is an example of the code I use to upload the disappearing data to ThingSpeak. The code for the other fields is the same. 

for j = 1:size(events_data,1)

        thingSpeakURL = 'http://api.thingspeak.com/';

        thingSpeakWriteURL = [thingSpeakURL 'update'];

        writeApiKey = 'XKSEKKX7TEKZ2N8E';

        %fieldName = 'Chart With Multiple Series 1';

        ChannelID = 379753;

        fieldvalues = [masses_M(j)];

        D = datetime(datestr(start_times(j)), 'InputFormat','dd-MMM-yyyy HH:mm:ss');

        thingSpeakWrite(ChannelID, 'Fields',[1],'Values',fieldvalues,'WriteKey',writeApiKey,'TimeStamp',D)

        pause(15);

    end

 

Is there anything in the code above which would cause the assignment of random entry ids?

Thanks. 

January 21, 2018
6:34 pm
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 258
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

I looked at your channel, Anerobic Digester sounds like a fun tracking application for thingSpeak!

I used the following rest call to see the data in fields 1,3,4,5:

http://api.thingspeak.com/channels/379753/charts/1?results=400

http://api.thingspeak.com/channels/379753/charts/3?results=400

etc.

I would suggest using the "Data export" tab next to the API keys tab instead of the data export button if you want all the data in the fields.  Even better to use REST api calls such as

https://api.thingspeak.com/channels/CHANNEL_ID/fields/FIELD_ID.csv?days=3

There are many other parameters you can use to customize the data returned to you with the rest call. ( i used three days as a parameter in the above, just as an example.)

As piojola indicated, different alignment of the entry ID and the time stamps will cause your plots to behave differently, but this is not outside the model of thingSpeak.  EntryID is assigned at the time you enter data, and is presently used to order the database entries.  So if you have one point entered in 2018 but its time stamp is from 2016, it would probably have a higher entry ID than all your data from 2017, for example.

Your data will not disappear once it gets written to a channel, but you may have to adjust some settings to see it.  Have you played with the chart parameters?  Its a pencil icon on the top of the plot.  Try the Results parameter, or Days.  These may help you make the chart show what you want more predictably. The delay you are seeing might be due to the alignment of the chart, settings, and data entry. 

If you have two update frequencies for the two sets of fields: it may be easier to use the default display settings if you update two channels separately.  It is definitely possible to use one channel, but the alternate updates will put empty entries in the fields of the other updates.  This does not change the data, but if you read by the results parameter, you might get a lot of empty entries.  

January 22, 2018
9:22 am
Avatar
bbruno22
New Member
Members
Forum Posts: 3
Member Since:
January 18, 2018
sp_UserOfflineSmall Offline

cstaples, 

Thank you for your reply. It is such a relief to see that my data doesn't actually disappear, though strange that it would seem so when using one export button and not another. Unfortunately, the data export button you recommend is not accessible to the public, making downloads by the public restricted to API calls. I was hoping to make downloading very easy, and my target audience is so much more likely to be familiar with CSV files then with calling an API. Alas. I may try to use separate channels for each data stream in hopes that it may help the charts keep my data visible. Thank you. 

Forum Timezone: America/New_York

Most Users Ever Online: 114

Currently Online:
19 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

rw950431: 261

Vinod: 196

piajola: 85

turgo: 70

vespapierre: 63

Adarsh_Murthy: 62

Member Stats:

Guest Posters: 1

Members: 5703

Moderators: 0

Admins: 2

Forum Stats:

Groups: 4

Forums: 17

Topics: 1313

Posts: 4565

Newest Members:

terranceqh2, DianeEmids, Alanawaype, uqovitay, ufijifode, manish01

Administrators: Hans: 387, lee: 457