Skip to main content

Posts

Showing posts from July, 2019

Edge Processing with Jetson Nano Part 2 - Apache NiFi Flow

Edge Data Processing with Jetson Nano Part 2 - Apache NiFi - Process, Route, Transform, Store Source:   https://github.com/tspannhw/minifi-jetson-nano Part 1:  https://www.datainmotion.dev/2019/07/edge-data-processing-with-jetson-nano.html



Apache NiFi Flow to Process Data

We route images from the webcameras, logs from the runs and JSON sensor readings to appropriate processors.  We also convert JSON to AVRO for storage in Hadoop or S3 while running queries on the data to check temperatures of the device.   TensorFlow and Apache MXNet are run on the images in-stream as they pass through Apache NiFi. Example Device and Deep Learning Data

Logs Returned From the Device

Push Some Results to Slack



Edge Data Processing with Jetson Nano Part 1 - Deploy, Setup and Ingest

Edge Data Processing with Jetson Nano Part 1 - Deploy, Setup and Ingest


















Configuring Executing Image Capture and Jetson Nano Classify Python Script


Configuring Tailing JSON Log


Configuring Acquiring Images from File Directory



Configuring the Remote Connection to NiFi





Example CEM Events




Simple NiFi Flow to Receive Remote Events

Apache NiFi Server receives from annotated images as well as JSON packets.

JSON Data Packet Example {"uuid": "nano_uuid_kwo_20190719182103", "ipaddress": "192.168.1.254", "top1pct": 32.6171875, "top1": "desktop computer", "cputemp": "32.5", "gputemp": "31.5", "gputempf": "89", "cputempf": "90", "runtime": "5", "host": "jetsonnano", "filename": "/opt/demo/images/image_bei_20190719182103.jpg", "imageinput": "/opt/demo/images/2019-07-19_1421.jpg"…

Philadelphia Open Crime Data on Phoenix / HBase

This is an update to a previous article on accessing Philadelphia Open Crime Data and storing it in Apache Phoenix on HBase.

It seems an update to Spring Boot, Phoenix and Zeppelin make for a cleaner experience.

I also added a way to grab years of historical Policing data.

All NiFi, Zeppelin and Source is here:   https://github.com/tspannhw/phillycrime-springboot-phoenix




Part 1: https://community.hortonworks.com/articles/54947/reading-opendata-json-and-storing-into-phoenix-tab.html












We convert JSON to Phoenix Upserts. We push JSON Records to HBase with PutHBaseReord.

Query Phoenix at the Command Line, Super Fast SQL


Resources

https://community.hortonworks.com/articles/56642/creating-a-spring-boot-java-8-microservice-to-read.html https://community.hortonworks.com/content/kbentry/54947/reading-opendata-json-and-storing-into-phoenix-tab.html

Powering Edge AI with the Powerful Jetson Nano

NVidia Jetson Nano Deep Learning Edge Device

Nano The Cat




Hardware:
Jetson Nano developer kit. Built around a 128-core Maxwell GPU and quad-core ARM A57 CPU running at 1.43 GHz and coupled with 4GB of LPDDR4 memory! This is power at the edge. I now have a favorite new device.

You need to add some kind of USB WiFi adaptor if you are not hardwired to ethernet. This is cheap and easy, I added a tiny $15 WiFi adapter and was off to the races.


Operating System:
Ubuntu 18.04

Library Setup:


sudo apt-get update -y sudo apt-get install git cmake -y sudo apt-get install libatlas-base-dev gfortran -y sudo apt-get install libhdf5-serial-dev hdf5-tools -y
sudo apt-get install python3-dev -y sudo apt-get install libcv-dev libopencv-dev -y sudo apt-get install fswebcam -y sudo apt-get install libv4l-dev -y sudo apt-get install python-opencv -y pip3 install psutil pip2 install psutil pip3.6 install easydict -U pip3.6 install scikit-learn -U pip3.6 install opencv-python -U --user pip3.6 install numpy -U pip3.6 inst…