Skip to main content

Posts

Showing posts with the label nijjwoliamsal

Ingest Salesforce Data into Hive Using Apache Nifi

Ingest Salesforce Data into Hive Using Apache NiFi TOOLS AND ACCOUNT USED:  Salesforce JDBC driver : https://www.progress.com/jdbc/salesforce Nifi : 1.9.2 - https://nifi.apache.org/docs/nifi-docs/html/getting-started.html Hive : 3.1.2 - https://luckymrwang.github.io/2018/03/14/Install-hive-on-Mac-with-Homebrew/ Java : 1.8.0_221  OSS : MacOS Mojave 10.14.5 SDFC Developer account : https://developer.salesforce.com/signup - 15 day trial Reference Article : https://www.progress.com/tutorials/jdbc/ingest-salesforce-data-incrementally-into-hive-using-apache-nifi Overall Flow : Install Hive - Thrift, Metastore, and Hive Table schema that matches SDFC opportunity table.  Install NiFi - Add PutHive3streaming nar  Install and Configure Drivers : SalesforceConnect and Avroreader Add Processors - QueryDatabaseTable and Puthive3Streaming Install Salesforce JDBC and Hive drivers Download DataDirect Salesforce JDBC driver from here . Downlo