Data In Motion

CALL SMM REST API from nifi

https://docs.hortonworks.com/HDPDocuments/SMM/SMM-1.1.0/rest-api-reference/index.html#!/Topic_metrics_operations/getTopicMetrics

https://hortonworks.com/blog/introducing-hortonworks-streams-messaging-manager-smm/

--- article , demo

curl -X GET "http://magellan-5.field.hortonworks.com:8585/api/v1/admin/metrics/aggregated/topics/syndicate-speed-event-avro?duration=LAST_ONE_HOUR&state=all" -H "accept: application/json"

curl -X GET "http://magellan-5.field.hortonworks.com:8585/api/v1/admin/metrics/brokers/1003?duration=LAST_ONE_HOUR&from=-1&to=-1" -H "accept: application/json"

curl -X GET "http://magellan-5.field.hortonworks.com:8585/api/v1/admin/brokers?brokerIds=1004%2C1002%2C1003" -H "accept: application/json"

Kafka Cluster Details

curl -X GET "http://magellan-5.field.hortonworks.com:8585/api/v1/admin/cluster" -H "accept: application/json"

curl -X GET "http://magellan-5.f…

Energy Monitoring via Apache NiFi MiNiFi 0.6.0 C++ Agent with Cloudera Edge Manager
With the advent of version 0.6.0 of the C++ Agent, we can have native fast Python Processors that can easily be added to your palate in Cloudera Edge Manager.


















Source:

https://github.com/tspannhw/minificpp-python-EnergyMonitoring

TIBCO Enterprise Message Service https://www.tibco.com/products/tibco-enterprise-message-service I tested this against the most recent release of TIBCO Enterprise Message Service and their JMS driver available via trial download. I followed the very easy install directions. I downloaded it to a Centos 7 server. Expanded my download to TIB_ems-dev_8.4.0_linux_x86_64 Then made it executable and ran TIBCOUniversalInstaller-lnx-x86-64.bin --console. I used all the defaults (I picked server and client) and then quickly ran the finished install server. Running Tibco on Centos 7 cd /opt/tibco/ems/8.4/bin/./tibemsd64 -config ~/TIBCO_HOME/tibco/cfgmgmt/ems/data/tibemsd.confExample JMS Queue Settings URL: tcp://servername:7222class: com.tibco.tibjms.TibjmsQueueConnectionFactoryDirectory:/opt/tibco/ems/8.4/lib/ I believe it just uses these files from that directory: tibjms.jarjms-2.0.jar Once I have my server and port shown, it's easy to add those settings to Apache NiFi. The settings I…

Article MiniFi Java Agent 0.5 Copy over necessary NARs from Apache NiFi 1.7 lib: nifi-ssl-context-service-nar-1.7.0.narnifi-standard-services-api-nar-1.7.0.narnifi-kafka-1-0-nar-1.7.0.nar This will support PublishKafka_1_0 and ConsumeKafka_1_0. Then create a consume and/or publish flow. You can combine the two based on your needs. In my simple example I consume the Kafka messages in MiniFi and write to a file. I also write the metadata to a JSON file. Consume Kafka Publish Electric Monitoring Data To Kafka Let's monitor the messages going through our topic, smartPlug. Publish Messages to Kafka Consume Any Messages From the smartPlug topic Logs ProvenanceEvent file containing 377 records.In the past 5 minutes,1512 events have been written to the ProvenanceRepository, totaling 839.32 KB2018-11-2619:42:32,473 INFO [main] o.a.n.c.s.StandardProcessSchedulerStartingPutFile[id=25a86505-031a-37d9-0000-000000000000]2018-11-2619:42:32,474 INFO [main