Skip to main content

Posts

Showing posts from July, 2019

Edge Processing with Jetson Nano Part 2 - Apache NiFi Flow

Edge Data Processing with Jetson Nano Part 2 - Apache NiFi - Process, Route, Transform, Store Source:    https://github.com/tspannhw/minifi-jetson-nano Part 1:   https://www.datainmotion.dev/2019/07/edge-data-processing-with-jetson-nano.html Apache NiFi Flow to Process Data We route images from the webcameras, logs from the runs and JSON sensor readings to appropriate processors.  We also convert JSON to AVRO for storage in Hadoop or S3 while running queries on the data to check temperatures of the device.   TensorFlow and Apache MXNet are run on the images in-stream as they pass through Apache NiFi. Example Device and Deep Learning Data Logs Returned From the Device Push Some Results to Slack

Edge Data Processing with Jetson Nano Part 1 - Deploy, Setup and Ingest

Edge Data Processing with Jetson Nano Part 1 - Deploy, Setup and Ingest Configuring Executing Image Capture and Jetson Nano Classify Python Script Configuring Tailing JSON Log Configuring Acquiring Images from File Directory Configuring the Remote Connection to NiFi Example CEM Events Simple NiFi Flow to Receive Remote Events Apache NiFi Server receives from annotated images as well as JSON packets. JSON Data Packet Example {" uuid ": "nano_uuid_kwo_20190719182103", " ipaddress ": "192.168.1.254", "top1pct": 32.6171875, " top1 ": "desktop computer", " cputemp ": "32.5", " gputemp ": "31.5", " gputempf ": "89", " cputempf ": "90", " runtime ": "5", " host ": "jetsonnano", " filename ": "/opt/dem

Philadelphia Open Crime Data on Phoenix / HBase

This is an update to a previous article on accessing Philadelphia Open Crime Data and storing it in Apache Phoenix on HBase. It seems an update to Spring Boot, Phoenix and Zeppelin make for a cleaner experience. I also added a way to grab years of historical Policing data. All NiFi, Zeppelin and Source is here:    https://github.com/tspannhw/phillycrime-springboot-phoenix Part 1:  https://community.hortonworks.com/articles/54947/reading-opendata-json-and-storing-into-phoenix-tab.html We convert JSON to Phoenix Upserts. We push JSON Records to HBase with PutHBaseReord. Query Phoenix at the Command Line, Super Fast SQL Resources https://community.hortonworks.com/articles/56642/creating-a-spring-boot-java-8-microservice-to-read.html https://community.hortonworks.com/content/kbentry/54947/reading-opendata-json-and-storing-into-phoenix-tab.html https://data.phila.gov/resource/sspu-uyfa.json?$$app_token=76MVJD

Powering Edge AI with the Powerful Jetson Nano

NVidia Jetson Nano Deep Learning Edge Device Nano The Cat Hardware: Jetson Nano developer kit. Built around a 128-core Maxwell GPU and quad-core ARM A57 CPU running at 1.43 GHz and coupled with 4GB of LPDDR4 memory! This is power at the edge. I now have a favorite new device. You need to add some kind of USB WiFi adaptor if you are not hardwired to ethernet. This is cheap and easy, I added a tiny $15 WiFi adapter and was off to the races. Operating System: Ubuntu 18.04 Library Setup: sudo apt-get update -y sudo apt-get install git cmake -y sudo apt-get install libatlas-base-dev gfortran -y sudo apt-get install libhdf5-serial-dev hdf5-tools -y sudo apt-get install python3-dev -y sudo apt-get install libcv-dev libopencv-dev -y sudo apt-get install fswebcam -y sudo apt-get install libv4l-dev -y sudo apt-get install python-opencv -y pip3 install psutil pip2 install psutil pip3.6 install easydict -