The Rise of the Mega Edge (FLaNK)

At one point edge devices were cheap, low energy and low powered.   They may have some old WiFi and a single core CPU running pretty slow.    Now power, memory, GPUs, custom processors and substantial power has come to the edge.

Sitting on my desk is the NVidia Xaver NX which is the massively powerful machine that can easily be used for edge computing while sporting 8GB of fast RAM, a 384 NVIDIA CUDA® cores and 48 Tensor cores GPU, a 6 core 64-bit ARM CPU and is fast.   This edge device would make a great workstation and is now something that can be affordably deployed in trucks, plants, sensors and other Edge and IoT applications.  

https://www.datainmotion.dev/2020/06/unboxing-most-amazing-edge-ai-device.html

Next that titan device is the inexpensive hobby device, the Raspberry Pi 4 that now sports 8 GB of LPDDR4 RAM, 4 core 64-bit ARM CPU and is speedy!   It can also be augmented with a Google Coral TPU or Intel Movidius 2 Neural Compute Stick.   

https://dzone.com/articles/efm-series-using-minifi-agents-on-raspberry-pi-4-w

https://www.datainmotion.dev/2020/02/edgeai-google-coral-with-coral.html

These boxes come with fast networking, bluetooth and the modern hardware running in small edge devices that can now deployed en masse.    Enabling edge computing, fast data capture, smart processing and integration with servers and cloud services.    By adding Apache NiFi's subproject MiNiFi C++ and Java agents we can easily integrate these powerful devices into a Streaming Data Pipeline.   We can now build very powerful flows from edge to cloud with Apache NiFi, Apache Flink, Apache Kafka  (FLaNK) and Apache NiFi - MiNiFi.    I can run AI, Deep Learning, Machine Learning including Apache MXNet, DJL, H2O, TensorFlow, Apache OpenNLP and more at any and all parts of my data pipeline.   I can push models to my edge device that now has a powerful GPU/TPU and adequate CPU, networking and RAM to do more than simple classification.    The NVIDIA Jetson Xavier NX will run multiple real-time inference streams at 60 fps on multiple cameras.  

I can run live SQL against these events at every segment of the data pipeline and combine with machine learning, alert checks and flow programming.   It's now easy to build and deploy applications from edge to cloud.

I'll be posting some examples in my next article showing some simple examples.

By next year, 12 or 16 GB of RAM may be a common edge device RAM, perhaps 2 CPUs with 8 cores, multiple GPUs and large fast SSD storage.   My edge swarm may be running much of my computing power as my flows running elastically on public and private cloud scale up and down based on demand in real-time.

Explore Enterprise Apache Flink with Cloudera Streaming Analytics - CSA 1.2

Explore Enterprise Apache Flink with Cloudera Streaming Analytics - CSA 1.2

What's New in Cloudera Streaming Analytics

https://docs.cloudera.com/csa/1.2.0/release-notes/topics/csa-what-new.html

https://docs.cloudera.com/csa/1.2.0/index.html

Try out the tutorials now:   https://github.com/cloudera/flink-tutorials

So let's get our Apache Flink on, as part of my FLaNK Stack series I'll show you some fun things we can do with Apache Flink + Apache Kafka + Apache NiFi.

We will look at some of updates in Apache Flink 1.10 including the SQL Client and API.

We are working with Apache Flink 1.10, Apache NiFi 1.11.4 and Apache Kafka 2.4.1.

The SQL features are strong and we will take a look at what we can do.

https://docs.cloudera.com/csa/1.2.0/release-notes/topics/csa-supported-sql.html

Table connectors

• Kafka
• Kudu
• Hive (through catalog)

Data formats (Kafka)

• JSON
• Avro
• CSV

Using Hive Catalog with Flink SQL:   https://docs.cloudera.com/csa/1.2.0/flink-sql-table-api/topics/csa-hive-catalog.html

Use Kudu Catalog with Flink SQL:   https://docs.cloudera.com/csa/1.2.0/flink-sql-table-api/topics/csa-kudu-catalog.html

Building a DataStream Application in Flink

https://docs.cloudera.com/csa/1.2.0/flink-sql-table-api/topics/csa-sql-table-api.html

Build A Flink Project

mvn archetype:generate                               \
      -DarchetypeGroupId=org.apache.flink              \
      -DarchetypeArtifactId=flink-quickstart-java      \
      -DarchetypeVersion=1.10.0

References:

• https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sqlClient.html
• https://github.com/apache/bahir-flink/tree/master/flink-connector-kudu#datastream-api

Using Apache Kafka Using Cloudera Data Platform Data Center 7.1.1

Using Apache Kafka Using Cloudera Data Platform Data Center 7.1.1

Primary Documentation Sources

https://docs.cloudera.com/csp/2.0.1/index.html

Kafka Public API List

https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/kafka_api.html

SMM REST API

https://docs.cloudera.com/smm/2.0.0/rest-api-reference/index.html

SRM REST API

https://docs.cloudera.com/srm/1.0.0/rest-api-reference/index.html

Schema Registry

https://docs.cloudera.com/csp/2.0.1/schema-registry-overview/topics/csp-schema_registry_overview.html

Kafka Using SR Libraries

https://docs.cloudera.com/csp/2.0.1/integrating-with-schema-registry/topics/csp-integrating_kafka_and_schema_registry.html

Development Libraries and SDKs

Adding additional metrics to SMM

https://docs.cloudera.com/csp/2.0.0/monitoring-end-to-end-latency/topics/smm-enabling-interceptors.html

Maven Repositories

https://mvnrepository.com/artifact/com.hortonworks.smm

Clients

https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/kafka_clients.html

Schema Registry Usage

https://registry-project.readthedocs.io/en/latest/

Tutorials

https://www.cloudera.com/tutorials/schema-registry-in-trucking-iot/2.html

https://www.cloudera.com/tutorials/kafka-in-trucking-iot/3.html

Kafka Crash Course

https://www.slideshare.net/Hadoop_Summit/kafkasmm-crash-course

Details

https://www.slideshare.net/ConfluentInc/disaster-recovery-with-mirrormaker-20-ryanne-dolan-cloudera-kafka-summit-london-2019

https://www.slideshare.net/AbdelkrimHadjidj/disaster-recovery-and-high-availability-with-kafka-srm-and-mm2

FAQ

https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/kafka_faq.html

Unboxing the Most Amazing Edge AI Device Part 1 of 3 - NVIDIA Jetson Xavier NX

Unboxing the Most Amazing Edge AI Device 

Fast, Intuitive, Powerful and Easy.

Part 1 of 3

NVIDIA Jetson Xavier NX

This is the first of a series on articles on using the Jetson Xavier NX Developer kit for EdgeAI applications.   This will include running various TensorFlow, Pytorch, MXNet and other frameworks.  I will also show how to use this amazing device with Apache projects including the FLaNK Stack of Apache Flink, Apache Kafka, Apache NiFi, Apache MXNet and Apache NiFi - MiNiFi.

These are not words that one would usually use to define AI, Deep Learning, IoT or Edge Devices.    They are now.    There is a new tool for making what was incredibly slow and difficult to something that you can easily get your hands on and develop with.  Supporting running multiple models simultaneously in containers with fast frame rates is not something I thought you could affordably run in robots and IoT devices.    Now it is and this will drive some amazingly smart robots, drones, self-driving machines and applications that are not yet in prototypes.

Out of the box, this machine is sleek, light weight and ready to go.   And now with built-in fast WiFi, yet another great upgrade!   I added a 256GB SSD Hard drive and it took seconds and a few quick Linux commands.   It's running Ubuntu 18.04 LTS which supports all the deep learning and python libraries you need and runs well.     It has a powerful fan already attached and judging by the fast spinning when I was running benchmarks it probably needs it.   

It was super easy to get working, just plugged in a USB mouse and keyboard and HDMI monitor. 

I ran the benchmarks and was massively impressed with the FPS that can be processed.   This machine has some serious power.  Basically, this device you are going to locate at the edge in a robot, drone, car or other edge point could be your desktop machine.

I ran a few graphics demos and tests to validate everything once my keyboard, mouse and HDMI monitor were connected.   The abilities are awesome.   I can see why NVIDIA GPUs are amazing for gaming.   

The specifications for the edge device are very impressive.   The 8GB of RAM makes this feel like a powerful desktop and not a low powered edge device.  

I ran the benchmarks and they were smoking fast.   I can see using this as a workstation as the FPS were nice as you can see below.

In part 2, I am going to show how to run some edge AI workloads at tremendous speed and stream the results and images to your cloud or big data environments using Apache open source frameworks including Apache Flink, Apache NiFi - MiNiFi and Apache Kafka.

In part 3, We will push the processing capabilities and amp up the workloads and test all the impressive features of this new killer edge device.

There is so many great tutorials and learning materials available for the NVIDIA Xavier NX.     I have found that all my work for Jetson Nano has been working here, only faster.  So this is great, I'll have a few interesting demos and run throughs and a video in the follow up articles.   

I added a standard USB hub and a Logitech C270 USB Web Camera which worked perfectly.   I will use that in the follow up articles and some edge applications.

Tutorials and Guides

• https://developer.nvidia.com/embedded/twodaystoademo
• https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-console-2.md
• https://github.com/dusty-nv/jetson-inference
• https://docs.nvidia.com/jetson/l4t/index.html#

References:

• https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-xavier-nx/
• https://elinux.org/Jetson_Zoo
• https://ngc.nvidia.com/catalog/containers/nvidia:l4t-ml
• https://www.nvidia.com/en-us/deep-learning-ai/education/?ncid=so-dis-dldlwsd1-72342
• https://developer.nvidia.com/embedded/jetpack
• https://elinux.org/Jetson_Nano#Cameras
• https://developer.nvidia.com/embedded/community/jetson-projects
• https://github.com/neuralet/neuralet/tree/master/applications/smart-distancing
• https://developer.nvidia.com/embedded/downloads
• https://www.jetsonhacks.com/
• https://docs.nvidia.com/jetson/jetpack/introduction/
• https://devblogs.nvidia.com/bringing-cloud-native-agility-to-edge-ai-with-jetson-xavier-nx/
• https://github.com/tspannhw/minifi-jetson-nano
• https://community.cloudera.com/t5/Community-Articles/Edge-Data-Processing-with-Jetson-Nano-Part-3-AI-Integration/ta-p/93642
• https://www.slideshare.net/bunkertor/iot-edge-data-processing-with-nvidia-jetson-nano-oct-3-2019
• https://dzone.com/articles/edge-data-processing-with-jetson-nano
• https://github.com/dusty-nv/jetson-inference/blob/master/python/examples/segnet-camera.py
• https://github.com/tspannhw/nvidiajetsontx1-mxnet/blob/master/classify.py
• https://github.com/tspannhw/ApacheDeepLearning202
• https://github.com/tspannhw/OpenSourceComputerVision
sudo /usr/sbin/nvpmodel -q

I highly recommend all AI, Deep Learning, IoT, IIoT, Edge and streaming developers obtain one or more of these developer kits.

This is a powerful machine in a small box.   From edge applications to robotics to smart devices to anything that needs powerful processing at the edge, this is your device.  A fast CPU, fast GPU and all the interfaces you need.  This should be part of any project.   Joining my NVIDIA Jetson Nano you now have some great affordable options for Edge AI applications.   It is amazing to test drive the performance of this device.   I will also be showing this at my online meetups, so join me or watch the video on Youtube later.

===

Jetson Xavier NX Developer Kit features:

 

Power:  10W (Max efficiency) | 15W (Max performance)

NVIDIA Volta architecture with 384 NVIDIA CUDA® cores and 48 Tensor cores

6-core NVIDIA Carmel ARM®v8.2 64-bit CPU 6 MB L2 + 4 MB L3

2x NVDLA Engines

8 GB 128-bit LPDDR4x @ 51.2GB/s

2x 4K @ 30 | 6x 1080p @ 60 | 14x 1080p @ 30 (H.265/H.264)

Gigabit Ethernet, M.2 Key E (WiFi/BT included), M.2 Key M (NVMe)

HDMI 

4x USB 3.1, USB 2.0 Micro-B

2x 4K @ 60 !  If you lower the resolution, it scales up the numbers.

The Jetson Xavier NX Developer Kit is now available for $399 US at NVIDIA.com and from channel partners worldwide.    I would recommend acquiring some ASAP before current supplies wane and you may have to wait.

Discord Integration with Apache NiFi

Discord Integration with Apache NiFi

For Slack, it's easy and built-in but sometimes as I was reminded you may want to send messages elsewhere.   Thanks to Brian Stitt for the starting information.

I created a free Discord Server in a few seconds and was ready to send messages.

https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks

You then copy the webhook URL and you are ready to call.   It's a long SSL link, just past that into the InvokeHTTP processor.

In our flow we need to create an SSL Context for NiFi.   If you are using NiFi in a CDP DataHub there's one for you already configured!

The output will not have a body but there will be cool headers if you want to save them or use them for processing.   Otherwise wrap this in a process group and you have an alert system of your own.

Before we call the InvokeHTTP we need to set the username of the bot.

Your message must be encoded as JSON like so:

{

"content":  "This is my message"

}

Let's call the hook with our data every field is important.

HTTP Method:   POST

Remote URL:  https://discord.com ... our copied webhooks link

SSL Context:  Create or use a StandardSSLContextService

Date Header:  False

Follow Redirecrs:  True

Use Digest: False

Always Output Response:  True

Add Response Headers to Request:  False

Content-Type:   application/json

Send Message Body: true

Chunk Encoding:  False

Ignore responses content: false

Here are the results:

We can see the output of the call to Discord Webhook HTTPS REST Endpoint:

We can see our bot, Spidey Bot, has posted some exciting test messages from NiFi.

For Source Code:

https://github.com/tspannhw/NiFiToDiscord

References:

• https://www.datainmotion.dev/2019/08/find-cacerts-from-java-jre-lib-security.html
• https://dzone.com/articles/lets-build-a-simple-ingest-to-cloud-data-warehouse

No More Spaghetti Flows

Spaghetti Flows

You may have heard of:   https://en.wikipedia.org/wiki/Spaghetti_code.   For Apache NiFi, I have seen some (and have done some of them in the past), I call them Spaghetti Flows.

Let's avoid them.   When you are first building a flow it often meanders and has lots of extra steps and extra UpdateAttributes and random routes. This applies if you are running on-premise, in CDP or in other stateful NiFi clusters (or single nodes). The following video from Mark Payne is a must watch before you write any NiFi flows.

Apache NiFi Anti-Patterns with Mark Payne

https://www.youtube.com/watch?v=RjWstt7nRVY

https://www.youtube.com/watch?v=v1CoQk730qs

https://www.youtube.com/watch?v=JbUjYr6Kd3I

https://github.com/tspannhw/EverythingApacheNiFi 

Do Not:

• Do not Put 1,000 Flows on one workspace.

• If your flow has hundreds of steps, this is a Flow Smell.   Investigate why.

• Do not Use ExecuteProcess, ExecuteScripts or a lot of Groovy scripts as a default, look for existing processors

• Do not Use Random Custom Processors you find that have no documentation or are unknown.

• Do not forget to upgrade, if you are running anything before Apache NiFi 1.10, upgrade now!

• Do not run on default 512M RAM.

• Do not run one node and think you have a highly available cluster.

• Do not split a file with millions of records to individual records in one shot without checking available space/memory and back pressure.

• Use Split processors only as an absolute last resort. Many processors are designed to work on FlowFiles that contain many records or many lines of text. Keeping the FlowFiles together instead of splitting them apart can often yield performance that is improved by 1-2 orders of magnitude.

Do:

• Reduce, Reuse, Recycle.    Use Parameters to reuse common modules.

• Put flows, reusable chunks (write to Slack, Database, Kafka) into separate Process Groups.

• Write custom processors if you need new or specialized features

• Use Cloudera supported NiFi Processors

• Use RecordProcessors everywhere

• Read the Docs!

• Use the NiFi Registry for version control.

• Use NiFi CLI and DevOps for Migrations.

• Run a CDP NiFi Datahub or CFM managed 3 or more node cluster.

• Walk through your flow and make sure you understand every step and it’s easy to read and follow.   Is every processor used?   Are there dead ends?

• Do run Zookeeper on different nodes from Apache NiFi.

• For Cloud Hosted Apache NiFi - go with the "high cpu" instances, such as 8 cores, 7 GB ram.

• same flow 'templatized' and deployed many many times with different params in the same instance

• Use routing based on content and attributes to allow one flow to handle multiple nearly identical flows is better than deploying the same flow many times with tweaks to parameters in same cluster.

• Use the correct driver for your database.   There's usually a couple different JDBC drivers.

• Make sure you match your Hive version to the NiFi processor for it.   There are ones out there for Hive 1 and Hive 3!   HiveStreaming needs Hive3 with ACID, ORC.  https://community.cloudera.com/t5/Support-Questions/how-to-use-puthivestreaming/td-p/108430

Let's revisit some Best Practices:

https://medium.com/@abdelkrim.hadjidj/best-practices-for-using-apache-nifi-in-real-world-projects-3-takeaways-1fe6912101db

Get your Apache NiFi for Dummies.   My own NiFi 101.

Here are a few things you should have read and tried before building your first Apache NiFi flow:

• https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html

• https://pierrevillard.com/best-of-nifi/

• https://blogs.apache.org/nifi/

• https://www.freecodecamp.org/news/nifi-surf-on-your-dataflow-4f3343c50aa2/

• https://www.nifi.rocks/documents/nifi-expression-language-cheat-sheet.pdf

Also when in doubt, use Records!  Use Record Processors and use pre-defined schemas, this will be easier to develop, cleaner and more performant. Easier, Faster, Better!!!

There are record processors for Logs (Grok), JSON, AVRO, XML, CSV, Parquet and more.

Look for a processor that has “Record” in the name like PutDatabaseRecord or QueryRecord.

Use the best DevOps processes, testing and tools.

• https://www.datainmotion.dev/2019/11/nifi-toolkit-cli-for-nifi-110.html

• https://pierrevillard.com/2018/04/09/automate-workflow-deployment-in-apache-nifi-with-the-nifi-registry/

• https://dzone.com/articles/devops-for-apache-nifi-17-and-more

• https://community.cloudera.com/t5/Community-Articles/Big-Data-DevOps-Apache-NiFi-Flow-Versioning-and-Automation/ta-p/247976

Some newer features in 1.8, 1.9, 1.10, 1.11 that you need to use.

• https://blogs.apache.org/nifi/entry/load-balancing-across-the-cluster

• https://community.cloudera.com/t5/Community-Articles/Offload-NiFi-Cluster-Nodes-using-the-UI-NiFi-1-8-0/ta-p/249070

• https://www.datainmotion.dev/2019/10/apache-nifi-load-balancing-via-load.html

• https://medium.com/@abdelkrim.hadjidj/best-practices-for-using-apache-nifi-in-real-world-projects-3-takeaways-1fe6912101db

• https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.5.1/nifi-configuration-best-practices/hdf-nifi-configuration-best-practices.pdf

• https://www.datainmotion.dev/p/links.html

Advanced Articles:

• https://community.cloudera.com/t5/Community-Articles/Dissecting-the-NiFi-quot-connection-quot-Heap-usage-and/ta-p/248166

• https://digitalplumber.me/2018/01/15/the-power-of-the-advanced-button-on-nifis-updateattribute-processor/

• https://community.cloudera.com/t5/Community-Articles/ETL-With-Lookups-with-Apache-HBase-and-Apache-NiFi/ta-p/248243

• https://medium.com/@abdelkrim.hadjidj/data-flow-enrichment-with-apache-nifi-d221f1dde419

• https://community.cloudera.com/t5/Community-Articles/Update-the-Contents-of-FlowFile-by-using-UpdateRecord/ta-p/248267

• https://community.cloudera.com/t5/Community-Articles/Optimizing-Performance-of-Apache-NiFi-s-Network-Listening/ta-p/246072

Spaghetti is for eating, not for real-time data streams.   Let's keep it that way.

If you are not sure what to do check out the Cloudera Community, NiFi Slack or the NiFi docs.   Also I may have a helpful article here. Join me and my NiFi friends at virtual meetups for more in-depth NiFi, Flink, Kafka and more. We keep it interactive so you can feel free to ask questions.

Note:   In this picture I am in Italy doing spaghetti research.