Skip to main content

Integration Testing for Apache NiFi Development

There are many ways to generate decent data to work with.

One good testing option is using REST APIs.

Another option is using a generator to generate CSV or JSON files.

There are also external data generators.

There are also NiFi processors for generating data.

The most common way to test your flows is with the GenerateFlowFile processor which lets you send valid flow files into your flow at a schedule or in rapid fire secession.

For my example, I am using NiFi Expression Language to generate some data.

Example Expression Language in GenerateFlowFile

{"id": "${now():format("yyyyMMddHHmmss")}_${UUID()}_${thread()}",
"te": "0.${random():mod(100000):plus(1)}",
"diskusage": "${math("random")}.3 MB",
"memory": ${random():mod(95):plus(10)},
"cpu": ${nextInt()}.${random():mod(99):plus(1)},
"host": "${ip()}/${hostname(true)}",
"temperature": "${random():mod(60):plus(60)}",
"macaddress": "${UUID()}",
"end": "${random():mod(100000000000000):plus(1)}",
"systemtime": "${now():format("MM/dd/yyyy HH:mm:ss", "EST")}"}

Example JSON Produced

{"id": "20190425131936_f061de76-edaf-4d9e-a144-2aeff2b1576a_Timer-Driven Process Thread-3",
"te": "0.28235",
"diskusage": "0.05997607531046045.3 MB",
"memory": 58,
"cpu": 0.52,
"host": "",
"temperature": "136",
"macaddress": "db00aef2-b242-4483-a552-223d74133aa5",
"end": "18296140941736",
"systemtime": "04/25/2019 12:19:36"}