Skip to main content

Year and Decade End Report : 201*

A Year in Big Data 2019

This has been an amazing year for Big Data, Streaming, Tech, The Cloud, AI, IoT and everything in between.   I got to witness the merger of the two Big Data giants into one unstoppable cloud data machine from the inside.   The sum of the new Cloudera is far greater than just Hortonworks + Cloudera.   It has been a great year working with amazing engineers, leaders, clients, partners, prospects, community members, data scientists, analysts, marketing mavens and everyone I have gotten to see this year.   It's been busy traveling the globe spreading the good word and solving some tough problems.   

In 2019, Edge2AI became something we could teach and implement in a single day to newbies.   The combination of MiNiFi + NiFi + Kafka + KuDu + Cloud is unstoppable.  Once we added Flink later this year, the FLaNK stack became amazing.   I see amazing stuff for this in the 20's.     I got to use new projects like Kudu (awesome), Impala, Cloudera Manager and new tools from the Data in Motion team.   Streams Messaging Manager became the best way to manage, monitor, create, alert on and use Kafka across clusters anywhere.   This is now my favorite way to demo anything.   So much transparency, awesome.   Having the power of Apache Flink is just making any problem solve-able, even those that scale to thousands of nodes.   Running just one node of Flink has been awesome.   I am a Squirrel Dev now!

Strata, DataWorksSummit and NoSQL Day were awesome, but working with charities and non-profits solving real world problems was amazing.     Helping at Nethope is the highlight of my professional year.   I am so thankful to the Cloudera Foundation for having me help.   I am really impressed with the Cloudera Foundation, Nethope and everyone involved.  I am hoping to speak to a few different conferences in 2020, but we'll see where Edge2AI takes me.

There's a lot to wrap up for 2019, so I attempted to put most of it following this break.

For DZone Articles See Here:

It was a great year talking with the tech people of the Princeton area.


Deep Learning

Enterprise IoT

At other events, I got to join some amazing colleagues spreading the word about cool open source technology.

The Cloudera Forum for CDP in  MINNEAPOLIS.   I got to help out on the amazing launch of Data in Motion on Cloudera.  

I got to lead all-day workshops in NYC, Boston and Washington DC.   I also got to talk about and do demos on NiFi and Phoenix for Cloudera Now online and some followup webinars.   It was great working with Milind, Paul, John and my local Cloudera crew.   Also the DIM Field Team has been amazing, big shout outs to Dan, Abdelkrim, Andre and Vasilis.   Andre made workshops so easy it's insane:   Dan's whoville and are next gen tools.   

I got to speak at a few awesome conferences this year.

Dataworks Summit Barcelona (

NoSQL Day DC ( See my joint talk on Phoenix and NiFi.   I was lucky to have Henry Sowell lead the HBase/Phoenix presentation with me.

Dataworks Summit DC ( See my talks on NiFi, Blockchain and Deep Learning.   I found a great way to have more amazing people speak at conferences, I had them added as co-speakers to all my talks.  Mehul and John did awesome.    Speaking is much better with a smarter buddy.

Strata NYC ( . I helped with some hands-on training sessions on Data in Motion.   We had most of the DIM Field team helping out on our multiple hands-on courses.   We were lucky to have unofficial DIM Field superstar Purnima do a lot of Kafka work for us.

Nethope Conference Puerto Rico

I wrote a few articles this year:

Here are my slides from 2019:

A Decade End Wrap Up 2010-2019

Blockchain and Cryptocurrency blew up and then blew up.   There's a few solid use cases though and the similarities between early Hadoop and Blockchain are interesting.   You can't count this stuff out.   From HPE to Pivotal to AirisData to Hortonworks to Cloudera, it's been an awesome ride doing Big Data, Spring, Java, Data in Motion, Streaming, Cloud, PaaS, Microservices, Containers, Kafka, NiFi, Spring XD, NodeJS.

I started posting to DZone in 2012 about MongoDB, Spring, NodeJS and such.

I did a ton of Spring stuff on and

What's Coming in 2020

  • Cloud Enterprise Data Platforms
  • Hybrid Cloud
  • Streaming with Flink, Kafka, NiFi
  • AI at the Edge with Microcontrollers and Small Devices
  • Voice Data In Queries
  • Event Handler as a Service (Automatic Kafka Message Reading)
  • More Powerful Parameter Based Modular Streaming 
  • Cloud First For Big Data
  • Log Handling Moves to MiNiFi
  • Full AI At The Edge with Deployable Models
  • More Powerful Edge TPU/GPU/VPU
  • Kafka is everywhere
  • Open Source UI Driven Event Engines
  • FLaNK Stack gains popularity
  • FLINK Everywhere


Popular posts from this blog

Ingesting Drone Data From DJII Ryze Tello Drones Part 1 - Setup and Practice

Ingesting Drone Data From DJII Ryze Tello Drones Part 1 - Setup and Practice In Part 1, we will setup our drone, our communication environment, capture the data and do initial analysis. We will eventually grab live video stream for object detection, real-time flight control and real-time data ingest of photos, videos and sensor readings. We will have Apache NiFi react to live situations facing the drone and have it issue flight commands via UDP. In this initial section, we will control the drone with Python which can be triggered by NiFi. Apache NiFi will ingest log data that is stored as CSV files on a NiFi node connected to the drone's WiFi. This will eventually move to a dedicated embedded device running MiniFi. This is a small personal drone with less than 13 minutes of flight time per battery. This is not a commercial drone, but gives you an idea of the what you can do with drones. Drone Live Communications for Sensor Readings and Drone Control You must connect t

NiFi on Cloudera Data Platform Upgrade - April 2021

CFM 2.1.1 on CDP 7.1.6 There is a new Cloudera release of Apache NiFi now with SAML support. Apache NiFi Apache NiFi Registry See:   For changes: Get your download on: To start researching for the future, take a look at some of the technical preview features around Easy Rules engine and handlers. Make sure you use the latest possible JDK 8 as there are some bugs out there.   Use a recent v

Using Apache NiFi in OpenShift and Anywhere Else to Act as Your Global Integration Gateway

Using Apache NiFi in OpenShift and Anywhere Else to Act as Your Global Integration Gateway What does it look like? Where Can I Run This Magic Engine: Private Cloud, Public Cloud, Hybrid Cloud, VM, Bare Metal, Single Node, Laptop, Raspberry Pi or anywhere you have a 1GB of RAM and some CPU is a good place to run a powerful graphical integration and dataflow engine.   You can also run MiNiFi C++ or Java agents if you want it even smaller. Sounds Too Powerful and Expensive: Apache NiFi is Open Source and can be run freely anywhere. For What Use Cases: Microservices, Images, Deep Learning and Machine Learning Models, Structured Data, Unstructured Data, NLP, Sentiment Analysis, Semistructured Data, Hive, Hadoop, MongoDB, ElasticSearch, SOLR, ETL/ELT, MySQL CDC, MySQL Insert/Update/Delete/Query, Hosting Unlimited REST Services, Interactive with Websockets, Ingesting Any REST API, Natively Converting JSON/XML/CSV/TSV/Logs/Avro/Parquet, Excel, PDF, Word Documents, Syslog, Kafka, JMS, MQTT, TCP