Data In Motion

Cloudera Edge Management 1.1.1 Release See :    https://docs.cloudera.com/cem/1.1.0/index.html https://docs.cloudera.com/cem/1.1.0/using-cem/topics/cem-deployment-monitor.html We can now integrate with Grafana and use Prometheus Metrics: https://docs.cloudera.com/cem/1.1.0/using-cem/topics/cem-monitoring-grafana-dashboard.html

Google Coral TPU with Edge Devices and MiNiFi  Designing Our Edge AI Flow with Cloudera Edge Flow Manager. Configure Your Remote Process Group to Send Data to Your NiFi Cluster Monitor Your Agents From the Events Screen Let's grab all the new images and then delete on completion We have Input and Output Ports to have Bidirectional communication with 0-n MiNiFi agents Our NiFi flow to process calls from MiNiFi Agents running Coral TPUs We run a query to check the TensorFlow Lite classification results and send out a slack message. Let's push JSON data to a Kafka Cluster in AWS

EFM Series:   Using MiNiFi Agents on Raspberry Pi 4 with Intel Movidius Neural Compute Stick 2, Apache NiFi and AI The good news is Raspberry Pi 4 can run MiNiFi Java Agents, Intel Movidius Neural Compute Stick 2 and AI libraries.   You can now use this 4GB of RAM device to run IoT with AI on the edge. Flow From MiNiFi Agent Running OpenVino, SysLog Tail and Grabbing WebCam Images Configure The Execution of OpenVino Python Applications on RPI 4 Events Returning from Raspberry Pi 4 Models Used Download model using downloader. Github https://github.com/tspannhw/minifi-rpi4-ncc2 DATE=$(date +"%Y-%m-%d_%H%M") fswebcam -q -r 1280x720 --no-banner /opt/demo/images/$DATE.jpg python3 /opt/intel/openvino/build/test.py /opt/demo/images/$DATE.jpg Software Apache NiFi Apache NiFi - MiNiFi Agents TensorFlow OpenVino Python 3 FSWEBCAM OpenCV DNN PSUTIL Python Libraries pip3 install getmac pip3 install psutil pip3 install --u

Energy Monitoring via Apache NiFi MiNiFi 0.6.0 C++ Agent with Cloudera Edge Manager With the advent of version 0.6.0 of the C++ Agent, we can have native fast Python Processors that can easily be added to your palate in Cloudera Edge Manager. Source: https://github.com/tspannhw/minificpp-python-EnergyMonitoring

NiFi Operations https://community.hortonworks.com/articles/207858/more-devops-for-hdf-apache-nifi-and-friends.html https://community.hortonworks.com/articles/92495/monitor-apache-nifi-with-apache-nifi.html https://dzone.com/articles/building-a-custom-apache-nifi-operations-dashboard https://dzone.com/articles/simple-apache-nifi-operations-dashboard-part-2-spr https://www.slideshare.net/Hadoop_Summit/best-practices-and-lessons-learnt-from-running-apache-nifi-at-renault https://community.hortonworks.com/articles/183217/devops-backing-up-apache-nifi-registry-flows.html https://community.hortonworks.com/articles/177349/big-data-devops-apache-nifi-hwx-schema-registry-sc.html https://community.hortonworks.com/articles/177301/big-data-devops-apache-nifi-flow-versioning-and-au.html https://community.hortonworks.com/articles/161761/new-features-in-apache-nifi-15-apache-nifi-registr.html https://community.hortonworks.com/articles/191658/devops-tips-using-the-apache-nifi-

I just returned from this awesome event.   Not even a rough plane trip can damper my spirits after seeing all the amazing things and all that we got to do this year.   It was nice to see familiar faces from attendees from 2017 and 2018 including my friends from Prague and Germany! Thanks to Andy LoPresto, George Vetticaden, Dinesh Chandrasekhar, Purnima, Nathan, Dan Chaffelson for great pictures, talks, support and being an amazing team for Data in Motionists. Meetup The meetup was great and in the same hall as some other amazing meetups at the same time. A great experience for those at Summit early (and open to all people for free). https://www.slideshare.net/bunkertor/the-edge-to-ai-deep-dive-barcelona-meetup-march-2019 https://www.meetup.com/futureofdata-barcelona/events/259345951/ Highlight :  Dan spinning up NiFi at scale in the audience on Google Cloud on K8 with ease! Highlight :  Andy’s crushing it MiNiFi and NiFi presentation! I think he h

Using Raspberry Pi 3B+ with Apache NiFi MiNiFi and Google Coral Accelerator and Pimoroni Inky Phat Architecture Introduction First we need to unbox our new goodies.   The Inky Phat is an awesome E-Ink display with low power usage that stays displayed after shutdown!  Next I added a new Google Coral Edge TPU ML Accelerator USB Coprocessor to a new Raspberry Pi 3B+.    This was so easy to integrate and get up and running. Let's unbox this beautiful device (but be careful when it runs it can get really hot and there is a warning in the instructions).   So I run this on top of an aluminum case and with a big fan on it. Pimoroni Inky Phat It is pretty easy to set this up and it provides a robust Python library to write to our E-Ink display.   You can see an example screen here. https://github.com/pimoroni/inky Pimoroni Inky pHAT ePaper eInk Display in Red Pimoroni Inky Phat (Red) https://shop.pimoroni.com/products/inky-phat https://github.com