Cloudera Edge Management 1.1.0 Release

Cloudera Edge Management 1.1.1 Release

See:   https://docs.cloudera.com/cem/1.1.0/index.html

https://docs.cloudera.com/cem/1.1.0/using-cem/topics/cem-deployment-monitor.html

We can now integrate with Grafana and use Prometheus Metrics:

https://docs.cloudera.com/cem/1.1.0/using-cem/topics/cem-monitoring-grafana-dashboard.html

Apache Atlas for Monitoring Edge2AI IoT Flows

Apache Atlas for Monitoring Edge2AI IoT Flows

Google Coral TPU with Edge Devices and MiNiFi

Google Coral TPU with Edge Devices and MiNiFi 

Designing Our Edge AI Flow with Cloudera Edge Flow Manager.

Configure Your Remote Process Group to Send Data to Your NiFi Cluster

Monitor Your Agents From the Events Screen

Let's grab all the new images and then delete on completion

We have Input and Output Ports to have Bidirectional communication with 0-n MiNiFi agents

Our NiFi flow to process calls from MiNiFi Agents running Coral TPUs

We run a query to check the TensorFlow Lite classification results and send out a slack message.

Let's push JSON data to a Kafka Cluster in AWS

EFM Series: Using MiNiFi Agents on Raspberry Pi 4 with Intel Movidius Neural Compute Stick 2, Apache NiFi and AI

EFM Series:   Using MiNiFi Agents on Raspberry Pi 4 with Intel Movidius Neural Compute Stick 2, Apache NiFi and AI

The good news is Raspberry Pi 4 can run MiNiFi Java Agents, Intel Movidius Neural Compute Stick 2 and AI libraries.   You can now use this 4GB of RAM device to run IoT with AI on the edge.

Flow From MiNiFi Agent Running OpenVino, SysLog Tail and Grabbing WebCam Images

Configure The Execution of OpenVino Python Applications on RPI 4

Events Returning from Raspberry Pi 4

Models Used

Download model using downloader.

Github

https://github.com/tspannhw/minifi-rpi4-ncc2

DATE=$(date +"%Y-%m-%d_%H%M")
fswebcam -q -r 1280x720 --no-banner /opt/demo/images/$DATE.jpg

python3 /opt/intel/openvino/build/test.py /opt/demo/images/$DATE.jpg

Software

• Apache NiFi
• Apache NiFi - MiNiFi Agents
• TensorFlow
• OpenVino
• Python 3
• FSWEBCAM
• OpenCV DNN
• PSUTIL

Python Libraries

pip3 install getmac

pip3 install psutil

pip3 install --upgrade pip
pip3 install --upgrade setuptools

• Intel Movidius Neural Compute Stick 2
• Raspberry Pi 4 with 4GB RAM

Hosting

We can run On-Premise on VMs, Containers, K8 or Bare metal.   Or on own of the clouds such as Google.

Resources

• https://github.com/movidius/ncappzoo
• https://docs.openvinotoolkit.org/latest/_docs_install_guides_installing_openvino_raspbian.html
• https://github.com/movidius/ncappzoo/blob/master/tensorflow/README.md
• https://software.intel.com/en-us/articles/python3-sbc-and-ncs2
• https://github.com/opencv/open_model_zoo
• https://download.01.org/opencv/2019/open_model_zoo/R2/20190628_180000_models_bin/
• https://docs.openvinotoolkit.org/latest/_intel_models_index.html
• https://docs.openvinotoolkit.org/latest/_intel_models_person_detection_retail_0013_description_person_detection_retail_0013.html
• https://docs.openvinotoolkit.org/latest/_intel_models_person_detection_action_recognition_0006_description_person_detection_action_recognition_0006.html
• https://docs.openvinotoolkit.org/latest/_intel_models_pedestrian_and_vehicle_detector_adas_0001_description_pedestrian_and_vehicle_detector_adas_0001.html
• https://github.com/opencv/open_model_zoo/blob/master/tools/downloader/README.md
• https://github.com/opencv/open_model_zoo/blob/master/demos/README.md
• https://github.com/opencv/open_model_zoo/blob/master/demos/gaze_estimation_demo/README.md
• https://github.com/opencv/open_model_zoo/blob/master/demos/security_barrier_camera_demo/README.md
• https://github.com/opencv/open_model_zoo/blob/master/demos/pedestrian_tracker_demo/README.md
• https://github.com/opencv/open_model_zoo/blob/master/demos/text_detection_demo/README.md
• https://github.com/movidius/ncappzoo/tree/master/apps/simple_classifier_py
• https://download.01.org/openvinotoolkit/2018_R5/open_model_zoo/text-detection-0001/FP32/
• http://docs.openvinotoolkit.org/latest/_inference_engine_samples_object_detection_sample_ssd_README.html
• https://software.intel.com/en-us/articles/get-started-with-neural-compute-stick
• https://apps.apple.com/us/app/pihelper/id1369930932

Directories

•  /root/inference_engine_samples_build/armv7l/Release
• /opt/intel/openvino

Barcelona DataWorks Summit March 2019

I just returned from this awesome event.   Not even a rough plane trip can damper my spirits after seeing all the amazing things and all that we got to do this year.   It was nice to see familiar faces from attendees from 2017 and 2018 including my friends from Prague and Germany!

Thanks to Andy LoPresto, George Vetticaden, Dinesh Chandrasekhar, Purnima, Nathan, Dan Chaffelson for great pictures, talks, support and being an amazing team for Data in Motionists.

Meetup

The meetup was great and in the same hall as some other amazing meetups at the same time. A great experience for those at Summit early (and open to all people for free).

https://www.slideshare.net/bunkertor/the-edge-to-ai-deep-dive-barcelona-meetup-march-2019

https://www.meetup.com/futureofdata-barcelona/events/259345951/

Highlight:  Dan spinning up NiFi at scale in the audience on Google Cloud on K8 with ease!

Highlight:  Andy’s crushing it MiNiFi and NiFi presentation! I think he has too many RPIs!

Demopalooza

Edge to AI

https://dataworkssummit.com/barcelona-2019/session/edge-to-ai-analytics-from-edge-to-cloud-with-efficient-movement-of-machine-data/

https://www.slideshare.net/bunkertor/edge-to-ai-analytics-from-edge-to-cloud-with-efficient-movement-of-machine-data

Apache Deep Learning 201

https://dataworkssummit.com/barcelona-2019/session/apache-deep-learning-201/

https://www.slideshare.net/bunkertor/apache-deep-learning-201-barcelona-dws-march-2019

All the Githubs

https://github.com/tspannhw/DWS2019Barcelona?_lrsc=c48b47b6-6b54-466a-87f6-457457dd21ec

All the great Apache NiFi content from Andy LoPresto is here including custom processors:
https://github.com/alopresto/slides/blob/master/dws_barcelona_2019/
https://github.com/alopresto/slides