Real-Time Energy Monitoring With Apache NiFi

Monitoring Energy




A simple example of using Apache NiFi to receive and process electric data stored as JSON.   We add SQL based alerting and storage to Parquet files with no code.



Cloudera Data Science Workbench

Using PySQL SQL to analyze Parquet files built by electric data stored to HDFS via Apache NiFi.

 We can display alerts easily with Slack.

If you want to know how something works in the Apache NiFi UI you can analyze the REST calls made by JSON.

Python Libraries

pip3 install pyhs100

pip3 install psutil 

Github Repos

• https://github.com/tspannhw/nifi-energy-monitoring
• https://github.com/tspannhw/nifi-smartplug

Hardware Meter

• https://www.amazon.com/TP-Link-HS110-Monitoring-Required-Assistant/dp/B0178IC5ZY

Resources

• https://community.hortonworks.com/articles/155326/monitoring-energy-usage-utilizing-apache-nifi-pyth.html 
• https://github.com/GadgetReactor/pyHS100
• https://pypi.python.org/pypi/pyHS100