Uploading CSV to ThingSpeak using Python code | ThingSpeak API | Forum

Avatar

Please consider registering
Guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —






— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed sp_TopicIcon
Uploading CSV to ThingSpeak using Python code
No permission to create posts
March 29, 2020
11:39 pm
Avatar
144742
New Member
Members
Forum Posts: 2
Member Since:
March 29, 2020
sp_UserOfflineSmall Offline

Hi everyone,

I am working on a project for school and I need to know if this situation is possible:

I have 3 sensors (pH, turbidity, temperature) connected to an Arduino, and the Arduino is connected to a Raspberry Pi. The Raspberry Pi reads serial communication from the Arduino, so I can get the sensor readings in Python on the Pi. I use my Python program to upload these sensor readings to ThingSpeak. All of this is working perfectly.

However, I was asked to implement a workaround if internet connectivity is not available/goes offline on the Raspberry Pi. By workaround, I mean that my Pi should be able to read the sensors and save their timestamp/data to a CSV file, then the CSV file should be automatically uploaded to ThingSpeak the next time the program runs and has internet connectivity.

I understand that if I save my data to a CSV, I can upload the file to ThingSpeak by going into the web interface and uploading the file manually. However, this is not what I was asked to achieve. I want to know if it is possible to read the data in, store it to a CSV, then later upload that CSV to ThingSpeak using a similar method to how it normally operates.

Thanks for your time and please ask any questions if I was not clear on something!

Edit: I forgot to mention that I have already looked this up quite extensively, to my knowledge, and cannot find a clear answer on whether it is possible.

March 30, 2020
8:48 pm
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 861
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

Bulk Write CSV Data
Also there is an example at Hackster showing some code to do it, but you should be able to do it yourself with the link above.

FYI, MATLAB answers is now the preferred location for ThingSpeak community help

March 31, 2020
9:07 pm
Avatar
144742
New Member
Members
Forum Posts: 2
Member Since:
March 29, 2020
sp_UserOfflineSmall Offline

Brilliant, thank you very much for your answer! Somehow I missed the bulk CSV documentation. I have given it a thorough read, along with the examples given and trawled through the support forums of both MATLAB answers and ThingSpeak community, but I can't seem to find much help getting it to work.

Most of the people asking for help are using JSON, or Arduino, or the ones who are using POST and Python (like I will be) have fixed their own issue without really going into detail on how they fixed it. Do you mind linking the Hackster example you found? If it's the "Solar Powered Cellular Weather Station" one, I have looked through it and still can't figure it out.

Let me say that I am still quite new to Python/ThingSpeak and to get the original data upload working (single write), I used bits and pieces of code I had found online. I've done my best to try to understand how the single write process works and apply it to the bulk write process, but I'm just drawing a blank. I guess my main question is that the URL in the API doesn't make sense to me: https://api.thingspeak.com/channels/channel_id/bulk_update.csv

I understand I'm supposed to put my channel ID into the URL, but how does my CSV file data apply to the "bulk_update.csv" part of the URL? Am I thinking about this the wrong way? For reference, here's the Python code I use to upload single entries (I cut out the parts of code that don't apply). Any help is greatly appreciated.

import http.client
import urllib
import time
import serial

#Assign ThingSpeak read API key to a variable
apiKey = 'XXXXXXXXX'

def readSensors(): #Read sensor data
(Code omitted, but it basically just grabs sensor data from the Arduino)

return pH, turbidity, temperature

def uploadReadings(): #ThingSpeak upload
pH, turbidity, temperature = readSensors()
params = urllib.parse.urlencode({'field1': pH, 'field2': turbidity, 'field3': temperature, 'key':apiKey})
headers = {"Content-type": "application/x-www-form-urlencoded","Accept": "text/plain"}
conn = http.client.HTTPConnection("api.thingspeak.com:80")

try:
print("Attempting to contact ThingSpeak")
conn.request("POST", "/update", params, headers)
response = conn.getresponse()
print ("Upload status: ", response.status, response.reason)
conn.close()
except:
print("Connection failed")

while True:
uploadReadings()
break

April 1, 2020
2:50 pm
Avatar
cstapels
Moderator
Members


Moderators
Forum Posts: 861
Member Since:
March 7, 2017
sp_UserOfflineSmall Offline

answering thread on MATLAB answers

Forum Timezone: America/New_York

Most Users Ever Online: 166

Currently Online:
20 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

rw950431: 272

Vinod: 240

piajola: 95

turgo: 70

vespapierre: 63

Adarsh_Murthy: 62

Member Stats:

Guest Posters: 1

Members: 8665

Moderators: 1

Admins: 2

Forum Stats:

Groups: 3

Forums: 14

Topics: 1600

Posts: 5760

Newest Members:

qkpham, SANGAMNATH B, swatiiot, kalyanigaware16@gmail.com, Romi, Drakan01

Moderators: cstapels: 460

Administrators: Hans: 405, lee: 457