Skip to main content

Posts

Showing posts with the label apache-kudu

Using Cloudera Data Platform with Flow Management and Streams on Azure

Using Cloudera Data Platform with Flow Management and Streams on Azure Today I am going to be walking you through using Cloudera Data Platform (CDP) with Flow Management and Streams on Azure Cloud.  To see a streaming demo video, please join my webinar (or see it on demand) at  Streaming Data Pipelines with CDF in Azure .  I'll share some additional how-to videos on using Apache NiFi and Apache Kafka in Azure very soon.    Apache NiFi on Azure CDP Data Hub Sensors to ADLS/HDFS and Kafka In the above process group we are using QueryRecord to segment JSON records and only pick ones where the Temperature in Fahrenheit is over 80 degrees then we pick out a few attributes to display from the record and send them to a slack channel. To become a Kafka Producer you set a Record Reader for the type coming in, this is JSON in my case and then set a Record Writer for the type to send to the  sensors  topic.    In this case we kept it as JSON, but we could convert to AVRO.   I usually do that

Read Apache Impala - Apache KUDU Tables and Send To Apache Kafka In Bulk Easily with Apache NiFi

See:   https://www.flankstack.dev/