Best in Flow Competition: Streaming Data Sets Available for Apps

Streaming Data Sets Available for Apps



The following Kafka topics are being populated with streaming data for you.



These come from the read-only Kafka cluster.


Navigate to the Data Hub Clusters.




Click on oss-kafka-datagen.


Click Schema Registry.



Click Streams Messaging Manager.




Use these brokers to connect to them:


Brokers


oss-kafka-datagen-corebroker1.oss-demo.qsm5-opic.cloudera.site:9093,oss-kafka-datagen-corebroker0.oss-demo.qsm5-opic.cloudera.site:9093,oss-kafka-datagen-corebroker2.oss-demo.qsm5-opic.cloudera.site:9093


Use this link for Schema Registry


https://#{Schema2}:7790/api/v1


Schema Registry Parameter Hostname:   Schema2


oss-kafka-datagen-master0.oss-demo.qsm5-opic.cloudera.site


To View Schemas in the Schema Registry click the icon from the datahub


https://oss-kafka-datagen-gateway.oss-demo.qsm5-opic.cloudera.site/oss-kafka-datagen/cdp-proxy/schema-registry/ui/#/


Schemas
https://github.com/tspannhw/FLaNK-DataFlows/tree/main/schemas


Group ID:   yourid_cdf 



Customers  (customer)


Example Row

{"first_name":"Charley","last_name":"Farrell","age":19,"city":"Sawaynside","country":"Guinea","email":"keven.herzog@hotmail.com","phone_number":"312-269-6619"}



IP Tables (ip_address)


Example Row

{"source_ip":"216.25.204.241","dest_port":219,"tcp_flags_ack":0,"tcp_flags_reset":0,"ts":"2023-04-20 15:26:45.517"}



Orders       (orders)


Example Row

{"order_id":84170282,"city":"Wintheiserton","street_address":"80206 Caroyln Lakes","amount":29,"order_time":"2023-04-20 13:25:06.097","order_status":"DELIVERED"}





Plants (plant)


Example Row

{"plant_id":829,"city":"Lake Gerald","lat":"39.568679","lon":"-151.64497","country":"Eritrea"}


Sensors (sensor)


Example Row

{"sensor_id":264,"timestamp_of_production":"2023-04-20 18:28:42.751"}




Sensor Data (sensor_data)


Example Row
{"sensor_id":250,"timestamp_of_production":"2023-04-20 18:42:04.847","sensor_value":-72}



Weather (weather)


Example Row

{"city":"New Ernesto","temp_c":21,"description":"Sleet"}




Transactions  (transactions)


Example Row

{"sender_id":40816,"receiver_id":96057,"amount":557,"execution_date":"2023-04-20 16:15:30.744","currency":"UYU"}




These are realistic generated data sources that you can use, they are available from read-only Kafka topics.   These can be consumed by any developers in the sandbox.


Make sure you name your Kafka Consumer your Workload Username _ Some Name.


Ex:    tim_customerdata_reader

on