May 27, 2018
Hello ThingSpeak Community,
Recently I bought an ESP32-F Module and I started experimenting with the wifi. I want to get data from a website and read them thru the ESP. My code is ok, connecting to my ThingHTTP link and reading the data. Problem is that I can not read data from a website using ThingHTTP. Let's see an example:
I want to get data from openweathermap (yes I know that they have already an API option but it's just an example)
When creating a ThingHTTP, at the URL I paste https://openweathermap.org/city/734077
Then using the inspect option I find the XPath of the humidity which is //*[@id="weather-widget"]/table/tbody/tr/td
Then I paste it at the Parse String, select GET and HTTP Version 1.1 and save it.
Then when I use the link that it says, I keep getting: Error parsing document, try a different parse string.
However in the second example:
Parse String: //*[@id="ct"]
And then I get the data that I want.
Why is that happening? I want to create a project that will get data from a website and display them on an LCD Screen. I also tried with receiving data from youtube without luck.
Any help is welcome!
P.S: I am new to the subject. I am sorry if I did something wrong. Please let me know.
March 7, 2017
What you have found is a common issue with ThingHTTP. You can only parse the content that is originally loaded with the page. If you inspect the source of the first page, you will see a weather-widget java script that is populating the table with the humidity data. That data is provided to the page by the script, but the data isn't there yet when thingHTTP attempts to parse the page for you. If you look at the source of the second page, you will find the element 2:58:45 am, which is populated into the page at runtime (though it may be updated later.)
At the present time, it is not possible for MATLAB or ThingSpeak to parse the final content of pages that have embedded code that populates elements after run time.
Fortunately there is the API option as you pointed out. There are also time APIs to read from the second page you described.
You can use MATLAB analysis and the webread() function to return data from API calls and provide it to other apps or write to your ThingSpeak channel.
Most Users Ever Online: 166
Currently Browsing this Page:
Guest Posters: 1
Newest Members:Team_mafia, icXu, emorphistechno, VTVMART, manimozhi, adamhebrew
Moderators: cstapels: 460
Administrators: Hans: 405, lee: 457