How about Some Free Cloud Training?

 

CDP Private Cloud Fundamentals

https://www.cloudera.com/about/training/courses/cdp-private-cloud-fundamentals.html

CDP Private Cloud Fundamentals

Cloudera, IBM, and Red Hat

Our CDP Private Cloud Fundamentals OnDemand course provides a solid introduction to CDP Private Cloud. In addition to learning what CDP Private Cloud is and how it fits into the Enterprise Data Cloud vision, you'll find out about its architecture and how it uses cloud-native design elements such as containerization in order to overcome limitations of the traditional bare metal cluster architecture. Following a summary of the system requirements, the course concludes with a demonstration of a CDP Private Cloud installation.

https://www.cloudera.com/about/training/courses/cloudera-ibm-redhat-cdp-pvc-fundamentals.html

Cloudera Essentials for CDP

https://www.cloudera.com/about/training/courses/cloudera-essentials-for-cdp.html

Introduction to Cloudera Manager

https://www.cloudera.com/about/training/courses/introduction-to-cloudera-manager.html

Introduction to Cloudera Data Warehouse: Self-Service Analytics in the Cloud with CDP

https://www.cloudera.com/about/training/courses/cdp-intro-to-cloudera-data-warehouse.html

Introduction to Cloudera Machine Learning

https://www.cloudera.com/about/training/courses/cdp-intro-to-cloudera-machine-learning.html

Introduction to Apache Impala

'

Enriching Data with Pyspark

Introduction to Hive

Demo Jam Build a Flow with Apache NiFi

Semantic Analysis

Apache Yunikorn

Introduction to Apache Ozone

Importing Data into the Cloud with Apache NiFi

Tutorial: Ingesting into Hive with NiFi

Processing Fixed Width and Complex Files

Pointers

The first decision you will have to make is if it's structured at all.   If it is a known type like CSV, JSON, AVRO, XML or Parquet then just use a record.

If it's semi-structured like a log file, GrokReader may work or ExtractGrok.

If it's like CSV, you may be able to tweak the CSV reader to work (say header or no header) or try one of the two CSV parsers NiFi has (Jackson or Apache Commons).    

If it's a format like PDF, Word, Excel, RTF or something like that, I have a custom processor that uses Apache Tika and that should be able to parse it into text.   Once it is text you can probably work with it.

Examples

• https://community.cloudera.com/t5/Support-Questions/How-to-parse-w-fixed-width-instead-of-char-delimited/td-p/102597
• https://community.cloudera.com/t5/Support-Questions/Best-way-to-parse-Fixed-width-file-using-Nifi-Kindly-help/m-p/177637
• https://community.cloudera.com/t5/Support-Questions/Split-one-Nifi-flow-file-into-Multiple-flow-file-based-on/td-p/203387
• https://community.cloudera.com/t5/Support-Questions/Splitting-a-Nifi-flowfile-into-multiple-flowfiles/td-p/139930
• https://community.cloudera.com/t5/Support-Questions/How-to-ExtractText-from-flow-file-using-Nifi-Processor/td-p/190826
• https://community.cloudera.com/t5/Community-Articles/Running-SQL-on-FlowFiles-using-QueryRecord-Processor-Apache/ta-p/246671
• https://www.datainmotion.dev/2020/12/smart-stocks-with-flank-nifi-kafka.html
• https://www.datainmotion.dev/2021/01/flank-real-time-transit-information-for.html
• https://medium.com/@nlabadie/apache-nifi-netflow-to-syslog-117d46867ae1
• https://medium.com/@nlabadie/apache-nifi-sftp-csv-to-syslog-json-d9da6938defa
• https://medium.com/@nlabadie/apache-nifi-pulling-from-mysql-and-sending-to-syslog-181dd4ae969c
• https://stackoverflow.com/questions/59291548/how-to-use-nifi-extractgrok-properly

Documentation

• https://nifi.apache.org/docs/nifi-docs/

Processors To Use For File Manipulation

• AttributesToCSV
• AttributesToJSON
• ConvertExcelToCSVProcessor 
• ConvertRecord
• ConvertText
• CSVReader
• EvaluateJSONPath
• EvaluateXPath
• EvaluateXQuery
• ExecuteScript
• ExecuteStreamCommand
• ExtractGrok
• ExtractText
• FlattenJson
• ForkRecord
• GrokReader
• JsonPathReader
• JsonTreeReader
• JoltTransformJSON
• JoltTransformRecord
• LookupAttribute
• LookupRecord
• MergeContent
• MergeRecord
• ModifyBytes
• ParseSyslog*
• PartitionRecord
• QueryRecord
• ReaderLookup
• ReplaceText
• ReplaceTextWithMapping
• ScriptedReader
• ScriptedRecordSink
• ScriptedTransformRecord
• SegmentContent
• SplitContent
• SplitJson
• SplitRecord
• SplitText
• SplitXml
• SyslogReader
• TransformXml
• UnpackContent
• UpdateAttribute
• UpdateRecord
• ValidCsv
• ValidateRecord
• ValidateXml

Custom Processors

• https://community.cloudera.com/t5/Community-Articles/Parsing-Any-Document-with-Apache-NiFi-1-5-with-Apache-Tika/ta-p/247672
• https://community.cloudera.com/t5/Community-Articles/Creating-HTML-from-PDF-Excel-and-Word-Documents-using-Apache/ta-p/247968
• https://github.com/tspannhw/nifi-extracttext-processor

Helper Projects, SDK, Libraries and Services

• https://tika.apache.org/ - Apache Tika can be integrated as a custom processor or called via REST and run as a seperate server/service.
• Cloudera Machine Learning - call this service from REST and have AI do it.   https://blog.cloudera.com/integrating-machine-learning-models-into-your-big-data-pipelines-in-real-time-with-no-coding/
• REST Service - there may be a service you can run locally or use in the cloud that may be able to parse it.   NiFi can call this
• Python - execute a stream command and have Python or a shell script or OS executeable do it!
• Spark - try custom Spark with Java, Python or Scala.
• Flink - try custom Flink with Java.
• XSLT
• XPath
• XQuery
• JsonPath
• Json
• https://github.com/AbsaOSS/cobrix
• https://github.com/tspannhw/EverythingApacheNiFi
• You may need to use Cache:   https://www.datainmotion.dev/2021/01/flank-using-apache-kudu-as-cache-for.html

 Using Cloudera Flow Management (Powered by Apache NiFi) to Track Cloud Services

See:

https://community.cloudera.com/t5/Community-Articles/Using-Cloudera-Flow-Management-To-Ingest-and-Process-RSS/ta-p/313074

Price Comparisons Using Retail REST APIs with Apache NiFi, Kafka and Flink SQL

 Price Comparisons Using Retail REST APIs with Apache NiFi, Kafka and Flink SQL

Part 1:   NiFi Rest

Part 2:   Kafka - Flink SQL

Part 3:  Cloudera Visual Apps

Part 4:   Smart Shelf Updates - MiNiFi Agents

MiNiFi Agent Update March 2021

 Cloudera Agent Availability

• https://docs.cloudera.com/cem/1.2.2/release-notes/topics/cem-minifi-cpp-agent-updates.html
• https://docs.cloudera.com/cem/1.2.2/release-notes/topics/cem-minifi-cpp-download-locations.html

Getting Started

https://docs.cloudera.com/cem/1.2.2/minifi-agent-quick-start/topics/cem-install-and-start-minifi-cpp.html

MiNiFi (C++)

Version cpp-0.9.0

Release Date: 1 March 2021

Highlights of 0.9.0 release include:

• Added support for RocksDB-based content repository for better performance
• Added SQL extension
• Improved task scheduling
• Various C2 improvements
• Bug fixes and improvements to TailFile, ConsumeWindowsEventLog, MergeContent, CompressContent, PublishKafka, InvokeHTTP
• Implemented RetryFlowFile and smart handling of loopback connections
• Added a way to encrypt sensitive config properties and the flow configuration
• Implemented full S3 support
• Reduced memory footprint when working with many flow files

Build Notes:

It is advised that you use the bootstrap.sh when not building on windows.

https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Versioncpp-0.9.0

Download Now As Source or Pre-Build for Your Platform

https://nifi.apache.org/minifi/download.html