Non-invasive Power Monitor (W.I.P)

Had a bit of time this week to start building my non-invasive power monitoring system.
The increasing winter power bills may have been an incentive.
Thought I would share this work in progress,

Here is the current hardware I am using
Total cost: $10.50 & some cents

While testing, I just have the CT sensor clipped over an exposed neutral wire of a socket board with a 2000W heater plugged in.
(The meter is just for checking the readings are correct)

I may eventually change to a different bare-bones ESP8266 / ESP8285 board that uses less power when sleeping, but the D1 is really easy to develop with for now.

The code running on the D1 currently reads the power every 30 seconds.
It stores 10x readings and then connects and send them via WiFi.
This should help the board last much longer on batteries than if it sent the reading every time.

Here is a snippet of the D1 code:
My next step was to find a FREE service online where the data could be sent.

There are plenty of these but most don't store historical data for long.
I also need the ability to run a script to process the incoming data to work out the actual timestamp for each reading.

The solution was Google Sheets & Google Scripts!
Here is a snippet of the Google Scripts code (Javascript) that processes the incoming readings.
It gets the current timestamp. It then basically works backwards to find the time that the first of the readings would have been done at. Then increments going forward as it adds the data to the spreadsheet.

And here is the populated spreadsheet
My next step is to write a Google Script that runs at 11:45pm each night and calculates my highest usage hour.
It will then update my Electric Kiwi Free Hour of Power to it.

I decided to do the most complicated stand-alone setup first so it should be an off-the-shelf solution for other Electric Kiwi customers. It will be easy to work backwards from this to integrate it with other things (smart home setups etc)

Update 18/06/2018

Today I am working on switching all the communication between D1 and server to use JSON data.
Also, all config will be stored on the server and the D1 will fetch / update it's config when it pushes up it's readings.
This allows for one lot of D1 code and the user being able to adjust it's config on-the-fly via changing the online config.

You can see the current base config here:
The new code returns it's config values to the D1 every-time it receives readings.
The D1 will then update it's settings accordingly.

I also had a good idea on how to reduce the power used by having the D1 sleep for the time periods where you can't have a free power hour (7am - 9am, & 4pm - 9pm).
You could also add more time periods you know won't be the highest eg. 12am - 5am.
That's at least 7 hours a day where the D1 could be sleeping and saving battery!

Here is the latest process code:
You can see that if the last reading's time is within a sleep time period, it works out how long it can sleep in seconds and sends that back to the D1. The D1 will then sleep for that amount of time.

Also, it's now storing all the times in local date time (instead of unix timestamp).

Update 19/06/2018

Added ADS1015 to part lists.
The ESP8622 built-in ADC is not very accurate and crashes often when trying to read over X readings for second. The ADS1015 should easily handle the task and give more accurate readings.

Update 22/06/2018

Removed ADS1015 from part list as now have the ESP8266 reading stable without it. YAH!

This is good as the ADS1015 would have added a bit more power usage.
If you wanted to measure smaller loads and needed more accurate readings - it would be the way to go.

Update 25/06/2018

Getting close to first GIT push of initial code!
I moved most the config to volatile memory.
This means if the device loses power, it will be wiped.

Which is perfect for this use case as I want the config to reset if power-loss as we don't then know how long it's been between a reading. We just want to start fresh. It also means the flash storage is barely used.

As the config is pulled from online - it simply power's up - fetches the config and starts doing it's read / sleep routine.

I've also added new config for "battery mode". If this is set to 0, then the device will stay powered on and connected to WiFi. This is what you will use if using permanent power.

I've also added WiFiManager so the device will start an access point if it can't connect to WiFi.
The user can then connect to this AP and set the devices WiFi network to use.

It also allows the user to set the Google Script ID for their online script.

As the scripts need to be "Public" to allow the device to write to them, I have also added a security key that the device sends so the script knows it's them. This key is simply the devices MAC address.

So, this is the end-user process from when they get the device.
  1. Apply power to device
  2. It will create a WiFi network called POWER MONITOR with a common password
  3. They connect to this network
  4. The device will direct to the "Sign In" page
  5. They copy the devices MAC address under "Info"
  6. They paste this into the Google Script
  7. Now, they can setup the WiFi and enter their Google Script ID
  8. The device will then connect to their WiFi, post it's initial reading to the Google Script and get back it's initial config.
  9. Result!
The combination of the long Google Script ID (about 60 characters) and the devices MAC address (12 characters), a 2 second delay for any wrong MAC requests and using HTTPS - means it would be near impossible to brute force attack. Even if "they" did - it's not like we are dealing with any sensitive data.

Update 05/07/18

All working pretty well now.

Still not happy with the battery usage, so going to try using RF link between 2x D1s with the 2nd D1 on mains power forwarding the data via WiFi.
This also would be a good solution for people with the Power Box out of reach of WiFi.
I will be using 2x Lora RA-01 Modules $5.10 each for this.
Just waiting for them to land...

Update 16/08/18

Made a lot of progress on the Google Script code.

It now can pull the available hours from Electric Kiwi.
Can calculate the sleep times for any periods that no hour can be set.
Can calculate the highest hour for the day.
Can update Electric Kiwi power hour to the highest hour.

Initial code is now up on my Github

https://github.com/matthuisman/power_monitor

I will continue to update this post with my progress.