FLaNK Stack Weekly for 20 June 2023

 

20-June-2023

FLiPN-FLaNK Stack Weekly

Tim Spann @PaaSDev

We are publishing late due to Father's Day and https://en.wikipedia.org/wiki/Juneteenth.

happyfathersday

NiFI updates

Apache NiFi 1.22 is now available for download! Release Date: June 11, 2023 MiNiFi agents can now talk to C2 servers using reverse proxy/load balancers. New processor to support modifying the compression algorithm of content which still incurs the CPU hit substantially improves IO by avoiding writing and reading the intermediary decompressed form. New processor for Azure Queue Storage using Azure SDK 12. Put Database Record now allows for Upserts. Deprecated additional components and features in preparation toward an Apache NiFi 2.0. Upgraded numerous dependencies due to potential vulnerabilities and latest stable lines. Numerous bug fixes to processors but also core NiFi framework behavior A full list of issues that were resolved can be found at: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353069

https://cwiki.apache.org/confluence/display/NIFI/Deprecated+Components+and+Features

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

**This is Issue #90 **

You may notice a version jump, Linked in says we had 89 already, so I am assuming two other articles got assimilated. I will go with this, since 90 is a better number.

https://github.com/tspannhw/FLiPStackWeekly

https://www.linkedin.com/pulse/schedule-2023-tim-spann-/

Videos

https://www.youtube.com/watch?v=_uYp8s6_6GA&t=1s&pp=ygUQIlRpbSBzcGFubiIgbmlmaQ%3D%3D

https://www.cloudera.com/about/events/cloudera-now-cdp.html

Articles

https://dzone.com/articles/harnessing-the-power-of-nifi-building-a-seamless-f

https://medium.com/@tspann/apache-nifi-1-22-updates-e658eaff3308

https://medium.com/@tspann/harnessing-the-power-of-nifi-building-a-seamless-flow-to-ingest-pm2-5-90246393fcab

https://blog.cloudera.com/one-big-cluster-stuck-platform-health/

https://medium.com/@ayushtkn/apache-hive-esri-geospatial-support-5ca815daaa7e

https://medium.com/@kiranprabhu/kafka-topic-naming-conventions-best-practices-6b6b332769a3

https://www.datanami.com/2023/01/30/five-drivers-behind-the-rapid-rise-of-apache-flink/

https://webtechie.be/post/2023-06-15-crac-on-raspberry-pi/

https://www.cdc.gov/nbs/modernization/stories/202304.html

https://www.smarthomebeginner.com/podman-vs-docker/

https://speakerdeck.com/stevenz3wu/streaming-from-apache-iceberg-qcon-ny-2023?slide=8

https://thenewstack.io/kelsey-hightower-predicts-how-the-kubernetes-community-will-evolve/

Recent Talks

https://www.youtube.com/watch?v=Ws7YmAHE1O8

https://www.cloudera.com/about/events/evolve.html

https://speakerdeck.com/stevenz3wu/streaming-from-apache-iceberg-qcon-ny-2023

Events

June 26-28, 2023: NLIT Summit. Milwaukee.
https://www.fbcinc.com/e/nlit/default.aspx

June 28, 2023: NiFi Meetup. Milwaukee and Hybrid. https://www.meetup.com/futureofdata-princeton/events/292976004/

meetup

July 19, 2023: 2-Hours to Data Innovation: Data Flow https://www.cloudera.com/about/events/hands-on-lab-series-2-hours-to-data-innovation.html

October 18, 2023: 2-Hours to Data Innovation: Data Flow https://www.cloudera.com/about/events/hands-on-lab-series-2-hours-to-data-innovation.html

Cloudera Events https://www.cloudera.com/about/events.html

More Events: https://www.linkedin.com/pulse/schedule-2023-tim-spann-/

LLM Generated Code

hyperflank

Code

NiFi Code

Tools

© 2020-2023 Tim Spann

FLiPN-FLaNK Stack Weekly - 12 June 2023

 

12-June-2023

FLiPN-FLaNK Stack Weekly

Tim Spann @PaaSDev

Cloudera Now is coming and we have a cool Edge Demo.

https://www.cloudera.com/about/events/cloudera-now-cdp.html

Hope everyone is safe, air quality readings from the many forest firest have been very high. Stay indoors East Coast people.

diagram

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

This is Issue #87

https://github.com/tspannhw/FLiPStackWeekly

https://www.linkedin.com/pulse/schedule-2023-tim-spann-/

Videos

https://www.youtube.com/watch?v=_uYp8s6_6GA&t=1s&pp=ygUQIlRpbSBzcGFubiIgbmlmaQ%3D%3D

https://www.youtube.com/watch?v=X7m4nZH8bUw&t=1s&pp=ygUQIlRpbSBzcGFubiIgbmlmaQ%3D%3D

https://www.youtube.com/watch?v=WvPqE8J3ZOE&t=1700s&pp=ygUQIlRpbSBzcGFubiIgbmlmaQ%3D%3D

https://www.youtube.com/watch?v=NMgkPFEQ0jA&pp=ygUOVGltIFNwYW5uIG5pZmk%3D

Articles

https://medium.com/@tspann/harnessing-the-power-of-nifi-building-a-seamless-flow-to-ingest-pm2-5-90246393fcab

https://medium.com/@tspann/wildfires-air-quality-time-to-fire-up-the-sensors-and-start-flanking-12ea0ba33f63

https://blog.cloudera.com/aaand-the-new-nifi-champion-is/

https://platform.openai.com/docs/guides/gpt-best-practices/six-strategies-for-getting-better-results

https://blog.devgenius.io/presto-introduction-10b3ba5020e8

https://medium.com/swlh/become-a-smart-content-creator-in-these-4-steps-6d4d813863c2

https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-programming-environment-on-an-ubuntu-20-04-server

https://www.cloudera.com/about/customers/metro-de-madrid.html

https://prestodb.io/docs/current/connector/kafka-tutorial.html

https://education.cloudera.com/store/2515343-apache-nifi-anti-patterns#description

https://medium.com/cloudera-inc/seamless-integration-unleashing-the-power-of-real-time-groceries-with-nifi-kafka-flink-and-32d659fe0903

https://docs.cloudera.com/cem/1.5.1/using-asset-push-command/topics/cem-using-asset-push-command.html

https://eng.lyft.com/gotchas-of-streaming-pipelines-profiling-performance-improvements-301439f46412

https://bryanbende.com/development/2021/07/19/apache-nifi-1-14-0-secure-by-default

https://docs.cloudera.com/cem/1.5.1/rest-api-reference/index.html#api-AgentClassManifestConfig

https://docs.cloudera.com/csp/2.0.1/monitoring-end-to-end-latency/topics/smm-enabling-interceptors.html

https://blog.cloudera.com/implementing-and-using-udfs-in-cloudera-sql-stream-builder/

Recent Talks

https://www.slideshare.net/bunkertor/ossna-building-modern-data-streaming-apps

https://www.slideshare.net/bunkertor/using-apache-nifi-with-apache-pulsar-for-fast-data-onramp

https://www.slideshare.net/bunkertor/budapest-dataml-building-modern-data-streaming-apps-with-nifi-flink-and-kafka

https://grafana.com/docs/grafana-cloud/data-configuration/metrics/prometheus-config-examples/the-apache-software-foundation-minifi/

Events

https://www.linkedin.com/posts/cloudera-partners_llm-opensource-llms-activity-7064751460844015616-gF45?utm_source=share&utm_medium=member_desktop

https://www.youtube.com/watch?v=Ws7YmAHE1O8

https://www.cloudera.com/about/events/evolve.html

https://web.cvent.com/event/7598f981-2f7e-4915-b662-bd7be9b5f48d/summary?RefId=homepage_impact24

https://www.cloudera.com/about/events/cloudera-now-cdp.html?i

June 14: 12PM EDT Cloudera Now - Virtual https://www.cloudera.com/about/events/cloudera-now-cdp.html?internal_keyplay=ALL&internal_campaign=FY24-Q2_AMER_CNOW_Q2_WEB_EP_P07_2023-06-14&cid=7012H000001ZLmyQAG&internal_link=p07

June 26-28, 2023: NLIT Summit. Milwaukee.
https://www.fbcinc.com/e/nlit/default.aspx

June 28, 2023: NiFi Meetup. Milwaukee and Hybrid. https://www.meetup.com/futureofdata-princeton/events/292976004/

meetup

July 19, 2023: 2-Hours to Data Innovation: Data Flow https://www.cloudera.com/about/events/hands-on-lab-series-2-hours-to-data-innovation.html

October 18, 2023: 2-Hours to Data Innovation: Data Flow https://www.cloudera.com/about/events/hands-on-lab-series-2-hours-to-data-innovation.html

Cloudera Events https://www.cloudera.com/about/events.html

More Events: https://www.linkedin.com/pulse/schedule-2023-tim-spann-/

Open Source Fonts

https://github.com/intel/intel-one-mono/releases

Tools

MiNiFi Tip

Start agent to beartbeat (register to efm) before the kafka nar was added

If not, change the agent class name the agent belongs to, restart minifi

© 2020-2023 Tim Spann

Timothy Spann: Building Real-time Travel Alerts

FLaNK Stack Weekly for 5 June 2023

 

05-June-2023

FLiPN-FLaNK Stack Weekly

Tim Spann @PaaSDev

Cloudera Now is coming and we have a cool Edge Demo.

https://www.cloudera.com/about/events/cloudera-now-cdp.html

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

This is Issue #86

https://github.com/tspannhw/FLiPStackWeekly

https://www.linkedin.com/pulse/schedule-2023-tim-spann-/

Videos

https://www.youtube.com/watch?v=UZRvUyjNWmg&t=1s&ab_channel=VMwareTanzu

Articles

https://medium.com/@tspann/seamless-integration-unleashing-the-power-of-real-time-groceries-with-nifi-kafka-flink-and-32d659fe0903

https://www.infoq.com/news/2023/05/apache-pulsar-messaging/

https://cldr-steven-matison.github.io//blog/NIFI-11608/

https://www.linkedin.com/posts/peterableda_generative-ai-for-the-enterprise-cloudera-activity-7069934437991391232-T776/

https://hubertdulay.substack.com/p/real-time-streaming-ecosystem-part

https://medium.com/@adipolak/the-developer-voice-guide-how-to-write-articles-engineers-want-to-read-446cdd1f0866

https://www.dremio.com/blog/the-life-of-a-write-query-for-apache-iceberg-tables/

Recent Talks

https://www.slideshare.net/ssuser73434e/bigdatafest-building-modern-data-streaming-apps-258024601

https://www.youtube.com/watch?v=X7m4nZH8bUw&ab_channel=StreamNative

https://www.youtube.com/watch?v=_uYp8s6_6GA&ab_channel=DatainMotion

https://www.youtube.com/watch?v=WvPqE8J3ZOE&pp=ygUIdGltIG5pZmk%3D

Events

https://www.linkedin.com/posts/cloudera-partners_llm-opensource-llms-activity-7064751460844015616-gF45?utm_source=share&utm_medium=member_desktop

https://www.youtube.com/watch?v=Ws7YmAHE1O8

https://www.cloudera.com/about/events/evolve.html

https://web.cvent.com/event/7598f981-2f7e-4915-b662-bd7be9b5f48d/summary?RefId=homepage_impact24

https://www.cloudera.com/about/events/cloudera-now-cdp.html?i

June 8: Budapest https://budapestdata.hu/2023/en/speakers/timothy-spann/ budapest

June 14: 12PM EDT Cloudera Now - Virtual https://www.cloudera.com/about/events/cloudera-now-cdp.html?internal_keyplay=ALL&internal_campaign=FY24-Q2_AMER_CNOW_Q2_WEB_EP_P07_2023-06-14&cid=7012H000001ZLmyQAG&internal_link=p07

June 26-28, 2023: NLIT Summit. Milwaukee.
https://www.fbcinc.com/e/nlit/default.aspx

June 28, 2023: NiFi Meetup. Milwaukee and Hybrid. https://www.meetup.com/futureofdata-princeton/events/292976004/

meetup

July 19, 2023: 2-Hours to Data Innovation: Data Flow https://www.cloudera.com/about/events/hands-on-lab-series-2-hours-to-data-innovation.html

October 18, 2023: 2-Hours to Data Innovation: Data Flow https://www.cloudera.com/about/events/hands-on-lab-series-2-hours-to-data-innovation.html

Cloudera Events https://www.cloudera.com/about/events.html

More Events: https://www.linkedin.com/pulse/schedule-2023-tim-spann-/

Tools

© 2020-2023 Tim Spann