The Rise of the Mega Edge (FLaNK)

At one point edge devices were cheap, low energy and low powered.   They may have some old WiFi and a single core CPU running pretty slow.    Now power, memory, GPUs, custom processors and substantial power has come to the edge.

Sitting on my desk is the NVidia Xaver NX which is the massively powerful machine that can easily be used for edge computing while sporting 8GB of fast RAM, a 384 NVIDIA CUDA® cores and 48 Tensor cores GPU, a 6 core 64-bit ARM CPU and is fast.   This edge device would make a great workstation and is now something that can be affordably deployed in trucks, plants, sensors and other Edge and IoT applications.  

https://www.datainmotion.dev/2020/06/unboxing-most-amazing-edge-ai-device.html

Next that titan device is the inexpensive hobby device, the Raspberry Pi 4 that now sports 8 GB of LPDDR4 RAM, 4 core 64-bit ARM CPU and is speedy!   It can also be augmented with a Google Coral TPU or Intel Movidius 2 Neural Compute Stick.   

https://dzone.com/articles/efm-series-using-minifi-agents-on-raspberry-pi-4-w

https://www.datainmotion.dev/2020/02/edgeai-google-coral-with-coral.html

These boxes come with fast networking, bluetooth and the modern hardware running in small edge devices that can now deployed en masse.    Enabling edge computing, fast data capture, smart processing and integration with servers and cloud services.    By adding Apache NiFi's subproject MiNiFi C++ and Java agents we can easily integrate these powerful devices into a Streaming Data Pipeline.   We can now build very powerful flows from edge to cloud with Apache NiFi, Apache Flink, Apache Kafka  (FLaNK) and Apache NiFi - MiNiFi.    I can run AI, Deep Learning, Machine Learning including Apache MXNet, DJL, H2O, TensorFlow, Apache OpenNLP and more at any and all parts of my data pipeline.   I can push models to my edge device that now has a powerful GPU/TPU and adequate CPU, networking and RAM to do more than simple classification.    The NVIDIA Jetson Xavier NX will run multiple real-time inference streams at 60 fps on multiple cameras.  

I can run live SQL against these events at every segment of the data pipeline and combine with machine learning, alert checks and flow programming.   It's now easy to build and deploy applications from edge to cloud.

I'll be posting some examples in my next article showing some simple examples.

By next year, 12 or 16 GB of RAM may be a common edge device RAM, perhaps 2 CPUs with 8 cores, multiple GPUs and large fast SSD storage.   My edge swarm may be running much of my computing power as my flows running elastically on public and private cloud scale up and down based on demand in real-time.

Unboxing the Most Amazing Edge AI Device Part 1 of 3 - NVIDIA Jetson Xavier NX

Unboxing the Most Amazing Edge AI Device 

Fast, Intuitive, Powerful and Easy.

Part 1 of 3

NVIDIA Jetson Xavier NX

This is the first of a series on articles on using the Jetson Xavier NX Developer kit for EdgeAI applications.   This will include running various TensorFlow, Pytorch, MXNet and other frameworks.  I will also show how to use this amazing device with Apache projects including the FLaNK Stack of Apache Flink, Apache Kafka, Apache NiFi, Apache MXNet and Apache NiFi - MiNiFi.

These are not words that one would usually use to define AI, Deep Learning, IoT or Edge Devices.    They are now.    There is a new tool for making what was incredibly slow and difficult to something that you can easily get your hands on and develop with.  Supporting running multiple models simultaneously in containers with fast frame rates is not something I thought you could affordably run in robots and IoT devices.    Now it is and this will drive some amazingly smart robots, drones, self-driving machines and applications that are not yet in prototypes.

Out of the box, this machine is sleek, light weight and ready to go.   And now with built-in fast WiFi, yet another great upgrade!   I added a 256GB SSD Hard drive and it took seconds and a few quick Linux commands.   It's running Ubuntu 18.04 LTS which supports all the deep learning and python libraries you need and runs well.     It has a powerful fan already attached and judging by the fast spinning when I was running benchmarks it probably needs it.   

It was super easy to get working, just plugged in a USB mouse and keyboard and HDMI monitor. 

I ran the benchmarks and was massively impressed with the FPS that can be processed.   This machine has some serious power.  Basically, this device you are going to locate at the edge in a robot, drone, car or other edge point could be your desktop machine.

I ran a few graphics demos and tests to validate everything once my keyboard, mouse and HDMI monitor were connected.   The abilities are awesome.   I can see why NVIDIA GPUs are amazing for gaming.   

The specifications for the edge device are very impressive.   The 8GB of RAM makes this feel like a powerful desktop and not a low powered edge device.  

I ran the benchmarks and they were smoking fast.   I can see using this as a workstation as the FPS were nice as you can see below.

In part 2, I am going to show how to run some edge AI workloads at tremendous speed and stream the results and images to your cloud or big data environments using Apache open source frameworks including Apache Flink, Apache NiFi - MiNiFi and Apache Kafka.

In part 3, We will push the processing capabilities and amp up the workloads and test all the impressive features of this new killer edge device.

There is so many great tutorials and learning materials available for the NVIDIA Xavier NX.     I have found that all my work for Jetson Nano has been working here, only faster.  So this is great, I'll have a few interesting demos and run throughs and a video in the follow up articles.   

I added a standard USB hub and a Logitech C270 USB Web Camera which worked perfectly.   I will use that in the follow up articles and some edge applications.

Tutorials and Guides

• https://developer.nvidia.com/embedded/twodaystoademo
• https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-console-2.md
• https://github.com/dusty-nv/jetson-inference
• https://docs.nvidia.com/jetson/l4t/index.html#

References:

• https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-xavier-nx/
• https://elinux.org/Jetson_Zoo
• https://ngc.nvidia.com/catalog/containers/nvidia:l4t-ml
• https://www.nvidia.com/en-us/deep-learning-ai/education/?ncid=so-dis-dldlwsd1-72342
• https://developer.nvidia.com/embedded/jetpack
• https://elinux.org/Jetson_Nano#Cameras
• https://developer.nvidia.com/embedded/community/jetson-projects
• https://github.com/neuralet/neuralet/tree/master/applications/smart-distancing
• https://developer.nvidia.com/embedded/downloads
• https://www.jetsonhacks.com/
• https://docs.nvidia.com/jetson/jetpack/introduction/
• https://devblogs.nvidia.com/bringing-cloud-native-agility-to-edge-ai-with-jetson-xavier-nx/
• https://github.com/tspannhw/minifi-jetson-nano
• https://community.cloudera.com/t5/Community-Articles/Edge-Data-Processing-with-Jetson-Nano-Part-3-AI-Integration/ta-p/93642
• https://www.slideshare.net/bunkertor/iot-edge-data-processing-with-nvidia-jetson-nano-oct-3-2019
• https://dzone.com/articles/edge-data-processing-with-jetson-nano
• https://github.com/dusty-nv/jetson-inference/blob/master/python/examples/segnet-camera.py
• https://github.com/tspannhw/nvidiajetsontx1-mxnet/blob/master/classify.py
• https://github.com/tspannhw/ApacheDeepLearning202
• https://github.com/tspannhw/OpenSourceComputerVision
sudo /usr/sbin/nvpmodel -q

I highly recommend all AI, Deep Learning, IoT, IIoT, Edge and streaming developers obtain one or more of these developer kits.

This is a powerful machine in a small box.   From edge applications to robotics to smart devices to anything that needs powerful processing at the edge, this is your device.  A fast CPU, fast GPU and all the interfaces you need.  This should be part of any project.   Joining my NVIDIA Jetson Nano you now have some great affordable options for Edge AI applications.   It is amazing to test drive the performance of this device.   I will also be showing this at my online meetups, so join me or watch the video on Youtube later.

===

Jetson Xavier NX Developer Kit features:

 

Power:  10W (Max efficiency) | 15W (Max performance)

NVIDIA Volta architecture with 384 NVIDIA CUDA® cores and 48 Tensor cores

6-core NVIDIA Carmel ARM®v8.2 64-bit CPU 6 MB L2 + 4 MB L3

2x NVDLA Engines

8 GB 128-bit LPDDR4x @ 51.2GB/s

2x 4K @ 30 | 6x 1080p @ 60 | 14x 1080p @ 30 (H.265/H.264)

Gigabit Ethernet, M.2 Key E (WiFi/BT included), M.2 Key M (NVMe)

HDMI 

4x USB 3.1, USB 2.0 Micro-B

2x 4K @ 60 !  If you lower the resolution, it scales up the numbers.

The Jetson Xavier NX Developer Kit is now available for $399 US at NVIDIA.com and from channel partners worldwide.    I would recommend acquiring some ASAP before current supplies wane and you may have to wait.

Cloudera Edge Management 1.1.0 Release

Cloudera Edge Management 1.1.1 Release

See:   https://docs.cloudera.com/cem/1.1.0/index.html

https://docs.cloudera.com/cem/1.1.0/using-cem/topics/cem-deployment-monitor.html

We can now integrate with Grafana and use Prometheus Metrics:

https://docs.cloudera.com/cem/1.1.0/using-cem/topics/cem-monitoring-grafana-dashboard.html

EdgeAI: Google Coral with Coral Environmental Sensors and TPU With NiFi and MiNiFi (Updated EFM)

EdgeAI:   Google Coral with Coral Environmental Sensors and TPU With NiFi and MiNiFi

Building MiNiFi IoT Apps with the new Cloudera EFM 

It is very easy to build a drag and drop EdgeAI application with EFM and then push to all your MiNiFi agents.

Cloudera Edge Management CEM-1.1.1
Download the newest CEM today!

https://www.cloudera.com/downloads/cdf/cem.html

https://docs.cloudera.com/cem/1.1.1/release-notes/topics/cem-whats-new.html

NiFi Flow Receiving From MiNiFi Java Agent

In a cluster in my CDP-DC Cluster I consume Kafka messages sent from my remote NiFi gateway to publish alerts to Kafka and push records to Apache HBase and Apache Kudu.  We filter our data with Streaming SQL.

We can use SQL to route, create aggregates like averages, chose a subset of fields and limit data returned.   Using the power of Apache Calcite, Streaming SQL in NiFi is a game changer against Record Data Types including CSV, XML, Avro, Parquet, JSON and Grokable text.   Read and write different formats and convert when your SQL is done.   Or just to SELECT * FROM FLOWFILE to get everything.  

We can see this flow from Atlas as we trace the data lineage and provenance from Kafka topic.

We can search Atlas for Kafka Topics.

From coral Kafka topic to NiFi to Kudu.

Details on Coral Kafka Topic

Examining the Hive Metastore Data on the Coral Kudu Table

NiFi Flow Details in Atlas

Details on Alerts Topic

'

Statistics from Atlas

See:   https://www.datainmotion.dev/2020/02/connecting-apache-nifi-to-apache-atlas.html

Example Web Camera Image

 Example JSON Record

[{"cputemp":59,"id":"20200221190718_2632409e-f635-48e7-9f32-aa1333f3b8f9","temperature":"39.44","memory":91.1,"score_1":"0.29","starttime":"02/21/2020 14:07:13","label_1":"hair spray","tempf":"102.34","diskusage":"50373.5 MB","message":"Success","ambient_light":"329.92","host":"coralenv","cpu":34.1,"macaddress":"b8:27:eb:99:64:6b","pressure":"102.76","score_2":"0.14","ip":"127.0.1.1","te":"5.10","systemtime":"02/21/2020 14:07:18","label_2":"syringe","humidity":"10.21"}]

Querying Kudu results in Hue

Pushing Alerts to Slack from NiFi

I am running on Apache NiFi 1.11.1 and wanted to point out a new feature.   Download flow:   Will download the highlighted flow/pgroup as JSON.

Looking at NiFi counters to monitor progress:

We can see how easy it is to ingest IoT sensor data and run AI algorithms on Coral TPUs.

Shell (coralrun.sh)

#!/bin/bash

DATE=$(date +"%Y-%m-%d_%H%M%S")

fswebcam -q -r 1280x720 /opt/demo/images/$DATE.jpg

python3 -W ignore /opt/demo/test.py --image /opt/demo/images/$DATE.jpg 2>/dev/null

Kudu Table DDL

https://github.com/tspannhw/table-ddl


Python 3 (test.py)

import time

import sys

import subprocess

import os

import base64

import uuid

import datetime

import traceback

import base64

import json

from time import gmtime, strftime

import math

import random, string

import time

import psutil

import uuid 

from getmac import get_mac_address

from coral.enviro.board import EnviroBoard

from luma.core.render import canvas

from PIL import Image, ImageDraw, ImageFont

import os

import argparse

from edgetpu.classification.engine import ClassificationEngine

# Importing socket library 

import socket 

start = time.time()

starttf = datetime.datetime.now().strftime('%m/%d/%Y %H:%M:%S')

def ReadLabelFile(file_path):

    with open(file_path, 'r') as f:

        lines = f.readlines()

    ret = {}

    for line in lines:

        pair = line.strip().split(maxsplit=1)

        ret[int(pair[0])] = pair[1].strip()

    return ret

# Google Example Code

def update_display(display, msg):

    with canvas(display) as draw:

        draw.text((0, 0), msg, fill='white')

def getCPUtemperature():

    res = os.popen('vcgencmd measure_temp').readline()

    return(res.replace("temp=","").replace("'C\n",""))

# Get MAC address of a local interfaces

def psutil_iface(iface):

    # type: (str) -> Optional[str]

    import psutil

    nics = psutil.net_if_addrs()

    if iface in nics:

        nic = nics[iface]

        for i in nic:

            if i.family == psutil.AF_LINK:

                return i.address

# /opt/demo/examples-camera/all_models  

row = { }

try:

#i = 1

#while i == 1:

    parser = argparse.ArgumentParser()

    parser.add_argument('--image', help='File path of the image to be recognized.', required=True)

    args = parser.parse_args()

    # Prepare labels.

    labels = ReadLabelFile('/opt/demo/examples-camera/all_models/imagenet_labels.txt')

    # Initialize engine.

    engine = ClassificationEngine('/opt/demo/examples-camera/all_models/inception_v4_299_quant_edgetpu.tflite')

    # Run inference.

    img = Image.open(args.image)

    scores = {}

    kCount = 1

    # Iterate Inference Results

    for result in engine.ClassifyWithImage(img, top_k=5):

        scores['label_' + str(kCount)] = labels[result[0]]

        scores['score_' + str(kCount)] = "{:.2f}".format(result[1])

        kCount = kCount + 1    

    enviro = EnviroBoard()

    host_name = socket.gethostname()

    host_ip = socket.gethostbyname(host_name)

    cpuTemp=int(float(getCPUtemperature()))

    uuid2 = '{0}_{1}'.format(strftime("%Y%m%d%H%M%S",gmtime()),uuid.uuid4())

    usage = psutil.disk_usage("/")

    end = time.time()

    row.update(scores)

    row['host'] = os.uname()[1]

    row['ip'] = host_ip

    row['macaddress'] = psutil_iface('wlan0')

    row['cputemp'] = round(cpuTemp,2)

    row['te'] = "{0:.2f}".format((end-start))

    row['starttime'] = starttf

    row['systemtime'] = datetime.datetime.now().strftime('%m/%d/%Y %H:%M:%S')

    row['cpu'] = psutil.cpu_percent(interval=1)

    row['diskusage'] = "{:.1f} MB".format(float(usage.free) / 1024 / 1024)

    row['memory'] = psutil.virtual_memory().percent

    row['id'] = str(uuid2)

    row['message'] = "Success"

    row['temperature'] = '{0:.2f}'.format(enviro.temperature)

    row['humidity'] = '{0:.2f}'.format(enviro.humidity)

    row['tempf'] = '{0:.2f}'.format((enviro.temperature * 1.8) + 32)    

    row['ambient_light'] = '{0}'.format(enviro.ambient_light)

    row['pressure'] = '{0:.2f}'.format(enviro.pressure)

    msg = 'Temp: {0}'.format(row['temperature'])

    msg += 'IP: {0}'.format(row['ip'])

    update_display(enviro.display, msg)

#    i = 2

except:

    row['message'] = "Error"

print(json.dumps(row)) 

Source Code:

https://github.com/tspannhw/nifi-minifi-coral-env

Sensors / Devices / Hardware:

• Humdity-HDC2010 humidity sensor
• Light-OPT3002 ambient light sensor
• Barometric-BMP280 barometric pressure sensor
• PS3 Eye Camera and Microphone USB
• Raspberry Pi 3B+
• Google Coral Environmental Sensor Board
• Google Coral USB Accelerator TPU

References:

• https://coral.ai/docs/enviro-board/get-started/
• https://coral.ai/products/accelerator/
• https://coral.ai/docs/enviro-board/datasheet/
• https://github.com/tspannhw/nifi-minifi-coral-env
• https://github.com/tspannhw/nifi-minifi-coral
• https://www.datainmotion.dev/2019/08/google-coral-tpu-with-edge-devices-and.html
• https://github.com/tspannhw/minifi-grove-sensors
• https://coral.ai/docs/enviro-board/get-started/
• https://coral.ai/products/accelerator/
• https://github.com/google/mediapipe/tree/master/mediapipe/examples/coral
• https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/raspberry_pi
• https://github.com/google-coral/examples-camera
• https://github.com/google-coral/project-keyword-spotter
• https://github.com/google/mediapipe/tree/master/mediapipe/examples/coral

EdgeAI: Jetson Nano with MiNiFi C++ Agent

Build and Utilizing The Apache NiFi - MiNiFi C++ Agent For Jetson Nano

(EdgeAI:   Jetson Nano with MiNiFi C++ Agent)

source.hostname
jetsonnano

source.ipv4
192.168.1.217

GetUSBCamera

FPS: .5

Bootstrap and Build

/opt/demo/nifi-minifi-cpp-source/build

bootstrap.sh

Options:  Kafka, OpenCV, TensorFlow, USB Camera

****************************************

 Select MiNiFi C++ Features to toggle.

****************************************

A. Persistent Repositories .....Enabled

B. Lib Curl Features ...........Enabled

C. Lib Archive Features ........Enabled

D. Execute Script support ......Enabled

E. Expression Language support .Enabled

F. Kafka support ...............Enabled

G. PCAP support ................Disabled

H. USB Camera support ..........Enabled

I. GPS support .................Disabled

J. TensorFlow Support ..........Disabled

K. Bustache Support ............Disabled

L. MQTT Support ................Enabled

M. SQLite Support ..............Disabled

N. Python Support ..............Enabled

O. COAP Support ................Enabled

S. SFTP Support ................Enabled

V. AWS Support .................Disabled

T. OpenCV Support ..............Enabled

U. OPC-UA Support...............Enabled

****************************************

sudo apt-get install libcurl-dev libcurl4-openssl-dev -y

make

We can see when data arrives in NiFi from a MiNiFi Agent.

 We can publish to Kafka directly from our MiNiFi C++ agent.

If CEM/Edge Flow Manager is a mystery to you, check out the live Swagger REST Documentation.

With MiNiFi C++ I can add a USB Camera.

 In NiFi we can see the Host Information that MiNiFi attached.

Example Data

{"uuid": "nano_uuid_crr_20200218002610", "ipaddress": "192.168.1.217", "top1pct": 54.833984375, "top1": "cab, hack, taxi, taxicab", "cputemp": "45.5", "gputemp": "43.5", "gputempf": "110", "cputempf": "114", "runtime": "4", "host": "jetsonnano", "filename": "/opt/demo/images/image_esq_20200218002610.jpg", "imageinput": "/opt/demo/images/2020-02-17_1926.jpg", "host_name": "jetsonnano", "macaddress": "ec:08:6b:18:0d:7f", "end": "1581985574.6246474", "te": "4.158604383468628", "systemtime": "02/17/2020 19:26:14", "cpu": 51.8, "diskusage": "5479.7 MB", "memory": 71.4, "id": "20200218002610_8a12dd65-1038-41ac-b923-98fc907f5be0"}

Example Config.yml Section

  name: AppendHostInfo

  class: org.apache.nifi.minifi.processors.AppendHostInfo

  max concurrent tasks: 1

  scheduling strategy: TIMER_DRIVEN

  scheduling period: 1000 ms

  penalization period: 30000 ms

  yield period: 1000 ms

  run duration nanos: 0

  auto-terminated relationships list: []

  Properties:

    Hostname Attribute: source.hostname

    IP Attribute: source.ipv4

    Network Interface Name: wlan0

Example Output

[2020-02-11 19:35:09.116] [org::apache::nifi::minifi::processors::ExecuteProcess] [info] Execute Command /opt/demo/rundemo.sh 

[2020-02-11 19:35:11.275] [org::apache::nifi::minifi::c2::C2Agent] [info] Checking 0 triggers

[2020-02-11 19:35:13.742] [org::apache::nifi::minifi::c2::C2Agent] [info] Checking 0 triggers

[2020-02-11 19:35:15.568] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 899b5964-4d2f-11ea-8b9a-6e260e221e3d from ExecuteProcess - Python to relationship success

[2020-02-11 19:35:15.568] [org::apache::nifi::minifi::processors::ExecuteProcess] [info] Execute Command Complete /opt/demo/rundemo.sh status 0 pid 31004

[2020-02-11 19:35:15.569] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 899b5964-4d2f-11ea-8b9a-6e260e221e3d from AppendHostInfo to relationship success

[2020-02-11 19:35:15.649] [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [info] Site to Site transaction 4d0b460e-e4f6-4ca1-8c56-30d310a0712b sent flow 1flow records, with total size 3581

[2020-02-11 19:35:15.785] [org::apache::nifi::minifi::sitetosite::HttpSiteToSiteClient] [info] Site to Site closed transaction 4d0b460e-e4f6-4ca1-8c56-30d310a0712b

[2020-02-11 19:35:15.841] [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [info] Site2Site transaction 4d0b460e-e4f6-4ca1-8c56-30d310a0712b peer finished transaction

[2020-02-11 19:35:15.841] [org::apache::nifi::minifi::io::HttpStream] [warning] Future status already cleared for http://ec2-35-171-154-174.compute-1.amazonaws.com:8080/nifi-api/data-transfer/input-ports/17979d5f-0170-1000-0000-000011f1cc00/transactions/4d0b460e-e4f6-4ca1-8c56-30d310a0712b/flow-files, continuing

[2020-02-11 19:35:16.236] [org::apache::nifi::minifi::c2::C2Agent] [info] Checking 0 triggers

[2020-02-11 19:35:16.263] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 8a05413a-4d2f-11ea-8b9a-6e260e221e3d from TailFile to relationship success

[2020-02-11 19:35:16.264] [org::apache::nifi::minifi::processors::TailFile] [info] TailFile nano.log for 616 bytes

[2020-02-11 19:35:16.273] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 8a05413a-4d2f-11ea-8b9a-6e260e221e3d from AppendHostInfo to relationship success

[2020-02-11 19:35:16.274] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 8a05413a-4d2f-11ea-8b9a-6e260e221e3d from PublishKafka to relationship success

[2020-02-11 19:35:18.748] [org::apache::nifi::minifi::c2::C2Agent] [info] Checking 0 triggers

[2020-02-11 19:35:21.260] [org::apache::nifi::minifi::c2::C2Agent] [info] Checking 0 triggers

Using Apache NiFi - MiNiFi C++ Agent Elsewhere

I am working on a Jetbot robot powered by NVidia Jetson Nano that will use the MiNiFi C++ agent.

References

• https://minifi.xnson.org/trying-minifi-on-the-jetson-nano

• https://github.com/apache/nifi-minifi-cpp

• https://gist.github.com/achristianson/1dea217e5fcbc88b87e526d919dad2c0