GAS meter automation | ThingSpeak Projects | Forum

Avatar

Please consider registering
Guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —






— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed sp_TopicIcon
GAS meter automation
No permission to create posts
July 22, 2016
2:42 pm
Avatar
kvv213

Gold
Members
Forum Posts: 28
Member Since:
July 22, 2016
sp_UserOfflineSmall Offline

Hello everyone!

I'm working on an amateur project for my summer house. A gas meter with an impulse external module is installed there. I'm going to connect this external module with Arduino and via Internet upload information to ThingsSpeak.

The problem is in the following: the external module provides me only an impulse. And I can upload a ping (1) to ThingsSpeak via Internet. This is useful in terms that I know when the meter counts another 0,01 cub.m. of gas. But I also would like to see not only pings at the time scale but also the current amount of pings. That means that every new ping should increase the total amount of pings at ThingsSpeak.

I briefly checked the samples and functions description but couldn't find a way how to increase the total amount of pings at ThingsSpeak. Please correct me if I'm wrong. 

So the only way how to solve this is to get the last ping number from ThingsSpeak and with the every new ping increase it at Arduino side and send updated value to ThingsSpeak. And here I have also two options. Get the last ping amount from ThingsSpeak at startup of Arduino or get it before every single ping submission.

 

I'm happy to see your opinions about this.

July 23, 2016
6:09 am
Avatar
daturach

Silver
Members
Forum Posts: 7
Member Since:
August 25, 2015
sp_UserOfflineSmall Offline

Hello kvv213

I do upload similar data for my gas consumption, but not on Thingspeak.
However, I am using Thingspeak for other data.

I can imagine the following process:
- Upload your counter (1) on channel 1
- Read channel 1 and write back to channel 2

If, at the same time, you can read channel 2 (last value) and add 1, you should have your accumulated data.
Once again, this is pure theory!

There is an example on this server, under MATLAB Analysis, called "Convert Celsius to Fahrenheit".
It shows how to read from a channel and write to another one.

Let us know if you manage to make it work.

Hope that help.

daturach

July 25, 2016
1:43 am
Avatar
kvv213

Gold
Members
Forum Posts: 28
Member Since:
July 22, 2016
sp_UserOfflineSmall Offline

daturach said

I can imagine the following process:
- Upload your counter (1) on channel 1
- Read channel 1 and write back to channel 2

 

That looks like the 3rd option how to store meter's values at ThingSpeak. I don't see anything difficult in that. And it is is similar to just pre read the latest value from the main channel and upload an incremented value to the same channel.

Option 1: Download the latest value from ThingSpeak at Arduino start-up and store it in the memory. Pros: we need to download it only once. Cons: if something wrong at Arduino side the results may be wrong (but likelikilyhood is very very low).

Option 2: Download the latest value just before update the channel. Increment and update another channel. Pros: we can adjust the latest value from outside (for example justify meter's count) and we don't need to reboot Arduino. Cons: the server responce can be delayed and that leads to a potentiall miss of a click from the meter (the system is busy with waiting responce from the server).

Option 3: Download the latest value from one channel then update both of them. Pros: mmmmm. Cons: requires 3 times more time than Option 1.

 

BTW were did you upload your values? Was it a public service?

July 25, 2016
3:40 am
Avatar
rw950431

Top
Members
Forum Posts: 262
Member Since:
January 30, 2014
sp_UserOfflineSmall Offline

I suspect you really want to display a rate (how many impulses per hour/day etc) rather than a count.

To the above options I'd like to add option 0: Dont worry about retrieving the past count so long as your arduino doesnt restart too often.  ( Every counter has an overflow point so your code just has to deal with a count that goes backward now and again)

See my post on how to convert a counter into a rate at http://community.thingspeak.com/forum/thingspeak-plugins/converting-counter-values-to-rate-of-change/  Doubtless it can also be done in a Matlab visualisation nowadays.

July 25, 2016
4:57 am
Avatar
daturach

Silver
Members
Forum Posts: 7
Member Since:
August 25, 2015
sp_UserOfflineSmall Offline

Hello kvv213

These values are uploaded on https://emoncms.org/  It's a public service but you can install the server on a Raspberry PI if you like.

I have shared some graphs related to gas at https://emoncms.org/daturach 

If you zoom in, you should see the counter at value one when there is gas consumption. The other graphs are just derived by calculation executed directly on the server.

daturach

July 25, 2016
10:51 pm
Avatar
rw950431

Top
Members
Forum Posts: 262
Member Since:
January 30, 2014
sp_UserOfflineSmall Offline

daturach-   cool graphs!  Was it difficult to derive the accumulation graphs from the counter using the emoncms platform?

July 26, 2016
5:07 am
Avatar
daturach

Silver
Members
Forum Posts: 7
Member Since:
August 25, 2015
sp_UserOfflineSmall Offline

Not really. In my case, one pulse = 10 litres. Then you can multiply (on the server) the litres by a factor to get the kWh. On the server again, you apply a function called "accumulator" which obviously accumulates the kWh.

There are a few basic functions you can use. If you already upload content on Thingspeak and I think you do [great rate display graph :-)], you can try emoncms. If you know PHP/Javascript, then you can develop your own applications and have them running on your Raspi.

daturach

July 26, 2016
11:14 am
Avatar
kvv213

Gold
Members
Forum Posts: 28
Member Since:
July 22, 2016
sp_UserOfflineSmall Offline

daturach said
Hello kvv213

These values are uploaded on https://emoncms.org/  It's a public service but you can install the server on a Raspberry PI if you like.

I have shared some graphs related to gas at https://emoncms.org/daturach 

If you zoom in, you should see the counter at value one when there is gas consumption. The other graphs are just derived by calculation executed directly on the server.

daturach  

Now the situation is getting worse. I have TWO data logging and visualization systems 🙂 The graphs are pretty cool.

July 26, 2016
11:22 am
Avatar
kvv213

Gold
Members
Forum Posts: 28
Member Since:
July 22, 2016
sp_UserOfflineSmall Offline

rw950431 said
I suspect you really want to display a rate (how many impulses per hour/day etc) rather than a count.

To the above options I'd like to add option 0: Dont worry about retrieving the past count so long as your arduino doesnt restart too often.  ( Every counter has an overflow point so your code just has to deal with a count that goes backward now and again)

See my post on how to convert a counter into a rate at http://community.thingspeak.com/forum/thingspeak-plugins/converting-counter-values-to-rate-of-change/  Doubtless it can also be done in a Matlab visualisation nowadays.  

Actually I have two goals:

1. Report to the gas company my remote house monthly gas consumption. They don't have a remote meter reading option but receive values via Internet (I should enter them manually). But in that case I need to read the values from the meter and then enter them into their system at their web-site. But due to the remote location I'm not always at the site to read the values 🙂 Video surveliance is an option but it doesn't solve goal two.

2. Try to get information about dependence of gas consumption, temperature and people presense on the site. Of cource there is a correlation: when it is -20C outside the consumption of gas is around 1000 sq.m. per month or even more. But I need more details.

 

BTW I already have infromation from my electricity company. They have remote meter data collection and it is possible to download the data from their servers via a simple script.

 

PS. I'm able with PHP, JS. I have a server under Ubuntu but I don't wan't to use it for temperature and gas data statistics. That is work for Arduino and I have a number of boards prepared for that (and for some other features of my house).

July 26, 2016
1:14 pm
Avatar
daturach

Silver
Members
Forum Posts: 7
Member Since:
August 25, 2015
sp_UserOfflineSmall Offline

Video surveliance is an option but it doesn't solve goal two.

or gprs/umts network with a cheap prepaid data SIM ??

Anyway, it looks like an interesting project.Laugh

daturach

July 26, 2016
3:49 pm
Avatar
piajola

Gold
Members
Forum Posts: 87
Member Since:
October 12, 2015
sp_UserOfflineSmall Offline

Hi kvv213,,

With W10 via Chrome I send:

http://api.thingspeak.com/channels/????/status.json?results=1

and obtain this webpage:

{"channel":{"name":"??????????","latitude":"0.000000","longitude":"0.000000","elevation":"00"},"feeds":[{"created_at":"2016-07-26T18:09:27Z","entry_id":205497,"status":null}]}

where entry_id+amount-in-gas-meter-when-I-started-my-arduino to send "the-pings" must be the actual reading in the gas meter ...
Assuming your Internet Service Provider is infallible (arduino+UPS[nicad/LiFePO4] IS) you send every ping for sure
So I can fill the text box to send my reading to the Gas Co. using "manual internet" as I understood ... (so, from everywhere)

3*60*24*30=129600 is about the performance you need in your "winter observation" i.e. 3 pings/minute 20 seconds ... still in Thingspeak range

People presence (via PIR?) so 0/1 and temp can be sent with each Gas Ping

My opinion as I am one in Hello everyone! in post 1Smile

Good luck !!

H.S.

July 26, 2016
4:39 pm
Avatar
kvv213

Gold
Members
Forum Posts: 28
Member Since:
July 22, 2016
sp_UserOfflineSmall Offline

daturach said

Video surveliance is an option but it doesn't solve goal two.

or gprs/umts network with a cheap prepaid data SIM ??

Anyway, it looks like an interesting project.Laugh

daturach  

The remote house has fiber-optic internet (that is in the country in Russia). So SIM is not required. Normal internet connection exists.

I'm going to include also Metane gas sensor, CO sensor as the must options for this project. That is because Arduino is going to be installed at the gas heater room. It is also intresting to analyze these two data feeds.

And if I have enough pins at Arduino board then I'll add two digital temperature and humidity meters.

BTW I already implemented Option 1 with ThingSpeak library. During the implementation I found out that if I don't have internet connection when I receive impulse from Gas meter then the library caches it and upload to the server when the connection restores. That is a good feature because I was thinking about a workaround for this.

July 26, 2016
4:47 pm
Avatar
kvv213

Gold
Members
Forum Posts: 28
Member Since:
July 22, 2016
sp_UserOfflineSmall Offline

piajola said
Hi kvv213,,

With W10 via Chrome I send:

http://api.thingspeak.com/channels/????/status.json?results=1

and obtain this webpage:

{"channel":{"name":"??????????","latitude":"0.000000","longitude":"0.000000","elevation":"00"},"feeds":[{"created_at":"2016-07-26T18:09:27Z","entry_id":205497,"status":null}]}

where entry_id+amount-in-gas-meter-when-I-started-my-arduino to send "the-pings" must be the actual reading in the gas meter ...
Assuming your Internet Service Provider is infallible (arduino+UPS[nicad/LiFePO4] IS) you send every ping for sure
So I can fill the text box to send my reading to the Gas Co. using "manual internet" as I understood ... (so, from everywhere)

3*60*24*30=129600 is about the performance you need in your "winter observation" i.e. 3 pings/minute 20 seconds ... still in Thingspeak range

People presence (via PIR?) so 0/1 and temp can be sent with each Gas Ping

My opinion as I am one in Hello everyone! in post 1Smile

Good luck !!

H.S.  

There is a library for ThingSpeak and it has some methods that can help in receiving values from ThingSpeak. And it works well in version 1.1.1 of the library.

People presence can be implemented much easier than PIR. Everyone has a mobile phone. And if you have Wi-Fi at the site then you can track presence of people at the site by their mobile phones connected to your wi-fi. Of cource the mobile phone shouldn't switch off wi-fi module when the screen is off. I already implemented this at bash.

PS. The link you mentioned works only for public channels. So if it is a private one you need to use Read API-Key (Write API-Key probably will work also). It is possible to send API-keys via from fields.

July 27, 2016
2:41 pm
Avatar
kvv213

Gold
Members
Forum Posts: 28
Member Since:
July 22, 2016
sp_UserOfflineSmall Offline

During long period testing - found out something not good. It seems that Arduino went to sleep somehow. 

I left the device with connected button (instead of gas meter for test purposes) overnight. I pressed the button several times in the morning and left it for almost 12 hours alone connected to Internet. The first press of the button woke the device up (it started Setup routine! I also calculate Arduino wakes up via code in Setup function) and the impulse was missed.... Next button press was collected well.

I'm going to add blinkin LED that will show that the device works at least. Plus warning and error leds. So this should prevent the device from sleep. I hope.

July 27, 2016
8:06 pm
Avatar
rw950431

Top
Members
Forum Posts: 262
Member Since:
January 30, 2014
sp_UserOfflineSmall Offline

Is it possible that your network card somehow lost connection due to inactivity so missed the first pulse while trying to reconnect?  

July 28, 2016
8:55 am
Avatar
kvv213

Gold
Members
Forum Posts: 28
Member Since:
July 22, 2016
sp_UserOfflineSmall Offline

rw950431 said
Is it possible that your network card somehow lost connection due to inactivity so missed the first pulse while trying to reconnect?    

By network errors it is not possible. By 'green' reasons it could be. But in that case it shouldn't call for set-up routine. Drop an exception or error is OK but not reset.

 

I've found out the problem with Setup invoke after a long time period without any impulses. Arduino connected to my PC (test and developing mode). When I wake up my computer at some period of its loading USB ports are dropped. So Arduino goes to reboot also.

But I have to confirm the problem with missing impulses after the long period of 'no impulses'. I see that Ethernet shield has connection to Internet and sometimes it receives packets. But it seems that something inside drops the connection.

https://i2.wp.com/s019.radikal.ru/i600/1607/23/e1fb71321bec.png?w=100Image Enlarger

As you can see at the graph. After reboot ThingSpeak libraries couldn't get values even from the server. I need to think about a workaround for such a behaviour.

PS. LED blinker is useful but didn't help with missing impulses 🙂

July 28, 2016
9:52 pm
Avatar
rw950431

Top
Members
Forum Posts: 262
Member Since:
January 30, 2014
sp_UserOfflineSmall Offline

I guess its possible that your program or libraries leak memory or otherwise overflow after prolonged running time.

You could add a section that writes an increasing counter to a a second channel regularly (every minute or hour) just to see if there is a problem with inactiivty or a long run time. (or maybe put in your temperature sensor for something useful)

July 31, 2016
5:13 pm
Avatar
kvv213

Gold
Members
Forum Posts: 28
Member Since:
July 22, 2016
sp_UserOfflineSmall Offline

rw950431 said
I guess its possible that your program or libraries leak memory or otherwise overflow after prolonged running time.

You could add a section that writes an increasing counter to a a second channel regularly (every minute or hour) just to see if there is a problem with inactiivty or a long run time. (or maybe put in your temperature sensor for something useful)  

Yes, that could be the solution. I have plans to post temperature and humidity from two of my sensors DHT-22 (one inside of the gas heater room and and outside the building) every 30 minutes. Also I'm thinking about a simple web-server (ethernet arduino library allows to set-up a light web-server). That will be useful to enter remotely to the device (from local network of cource) and see its condition on-line. And I hope that it also help to prevent the device from loosing the connection. But I have less than 25% free memory for the program already. Probably I'll cut down some text strings and other debug output but there is a risk that something I should implement using one more Arduino board 🙂

During the weekedn I found out that if the ethernet board is not connected to ethernet the procedure of init of ethernet can lasts for ever.  I did a number tests and only once I got a reasonable error that the shield couldn't connect to DHCP server. The rest of times just infinite initialization procedure. 

Also there can be a problem with the 15 seconds values upload interval. I didn't get in touch with it during my tests (when I pressed the button repeatedly). But looking after the speed of the Gas meter I think that a special procedure should be realized that accumulates impulses and post them with pace less than 1 upload per 15 second.

August 2, 2016
5:07 pm
Avatar
kvv213

Gold
Members
Forum Posts: 28
Member Since:
July 22, 2016
sp_UserOfflineSmall Offline

Current situation:

1. I've realized Web-server at Arduino that shows simple values from the sensors I have. Will test if it help with Ethernet deep sleep.

2. I've connected two MQ sensors to the board. *

3. I've connected two DHT11 sensors to the board. *

4. * I've found out that Ehternet shield doesn't work if I connect at least one MQ sensor. The board is feeded via USB and it is not enough. Ethernet shield doesn't connect to my network and stops the entire programm run (I wrote about this above). If I feed the board via external power source using this special round plug Ethernet shield doesn't work with all the symptomes above (I used 9V 1A power source). So it seems that Ethernet shields takes too much energy (and DC-DC step-down onboard converter is too weak) and I have to use external 5V power source that will be connected into 5Vin pin.

So, the project is a little bit delayed due to point 5. Also I'm waiting for arrival DHT22 sensors.

Current breadboardImage Enlarger

August 3, 2016
2:22 pm
Avatar
kvv213

Gold
Members
Forum Posts: 28
Member Since:
July 22, 2016
sp_UserOfflineSmall Offline

The idea with Web-server works. After 12 hours of no activity (except blinking LED) the device received button event and post it to ThingSpeak succesfully.

PS. Still struggle with not enough power for Ethernet Board :-/

No permission to create posts
Forum Timezone: America/New_York

Most Users Ever Online: 114

Currently Online:
19 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

rw950431: 261

Vinod: 196

piajola: 85

turgo: 70

vespapierre: 63

Adarsh_Murthy: 62

Member Stats:

Guest Posters: 1

Members: 5703

Moderators: 0

Admins: 2

Forum Stats:

Groups: 4

Forums: 17

Topics: 1313

Posts: 4565

Newest Members:

MiguelWap, JamesdinnA, ScottTug, ThomasApesy, RicardoVeque, AlbertRop

Administrators: Hans: 387, lee: 457