Data In Motion

CALL SMM REST API from nifi https://docs.hortonworks.com/HDPDocuments/SMM/SMM-1.1.0/rest-api-reference/index.html#!/Topic_metrics_operations/getTopicMetrics https://hortonworks.com/blog/introducing-hortonworks-streams-messaging-manager-smm/ --- article , demo curl -X GET "http://magellan-5.field.hortonworks.com:8585/api/v1/admin/metrics/aggregated/topics/syndicate-speed-event-avro?duration=LAST_ONE_HOUR&state=all" -H "accept: application/json" curl -X GET "http://magellan-5.field.hortonworks.com:8585/api/v1/admin/metrics/brokers/1003?duration=LAST_ONE_HOUR&from=-1&to=-1" -H "accept: application/json" curl -X GET "http://magellan-5.field.hortonworks.com:8585/api/v1/admin/brokers?brokerIds=1004%2C1002%2C1003" -H "accept: application/json" Kafka Cluster Details curl -X GET "http://magellan-5.field.hortonworks.com:8585/api/v1/admin/cluster" -H "accept: application/json&q

Energy Monitoring via Apache NiFi MiNiFi 0.6.0 C++ Agent with Cloudera Edge Manager With the advent of version 0.6.0 of the C++ Agent, we can have native fast Python Processors that can easily be added to your palate in Cloudera Edge Manager. Source: https://github.com/tspannhw/minificpp-python-EnergyMonitoring

TIBCO Enterprise Message Service https://www.tibco.com/products/tibco-enterprise-message-service I tested this against the most recent release of TIBCO Enterprise Message Service and their JMS driver available via trial download. I followed the very easy install directions. I downloaded it to a Centos 7 server. Expanded my download to TIB_ems-dev_8.4.0_linux_x86_64 Then made it executable and ran TIBCOUniversalInstaller-lnx-x86-64.bin --console. I used all the defaults (I picked server and client) and then quickly ran the finished install server. Running Tibco on Centos 7 cd / opt / tibco / ems / 8.4 / bin / ./ tibemsd64 - config ~ /TIBCO_HOME/ tibco / cfgmgmt / ems / data / tibemsd . conf Example JMS Queue Settings URL : tcp : //servername:7222   class : com . tibco . tibjms . TibjmsQueueConnectionFactory   Directory : /opt/ tibco / ems / 8.4 / lib / I believe it just uses these files from that directory: tibjms.jar jms-2.0.j

Article MiniFi Java Agent 0.5 Copy over necessary NARs from Apache NiFi 1.7 lib: nifi-ssl-context-service-nar-1.7.0.nar nifi-standard-services-api-nar-1.7.0.nar nifi-kafka-1-0-nar-1.7.0.nar This will support PublishKafka_1_0 and ConsumeKafka_1_0. Then create a consume and/or publish flow. You can combine the two based on your needs. In my simple example I consume the Kafka messages in MiniFi and write to a file. I also write the metadata to a JSON file. Consume Kafka Publish Electric Monitoring Data To Kafka Let's monitor the messages going through our topic,  smartPlug . Publish Messages to Kafka Consume Any Messages From the  smartPlug  topic Logs Provenance Event file containing 377 records . In the past 5 minutes , 1512 events have been written to the Provenance Repository , totaling 839.32 KB   2018 - 11 - 26 19 : 42 : 32 , 473 INFO [ main ] o . a . n . c . s . StandardProcessScheduler Starting PutFile [ id = 25a8650