FLaNK Stack Weekly 29 Jan 2024

 

29-January-2024

FLaNK Stack Weekly

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

Get your new Apache NiFi for Dummies!

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

Trial: https://console.us-west-1.cdp.cloudera.com/trial/register.html#/

Building Realtime AI Applications with Apache Flink

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

image

**This is Issue #122 **

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

Articles

Apache NiFi and Amazon Textract for Machine Learning https://medium.com/@tspann/apache-nifi-and-amazon-textract-for-machine-learning-e45f4af12e68

Apache NiFi and Amazon Transcribe for Machine Learning https://medium.com/@tspann/apache-nifi-and-amazon-transcribe-for-machine-learning-00db5ed0996a

Building a Library of Python Processors https://medium.com/@tspann/building-a-library-of-python-processors-6b5517404a58

Harnessing the Power of Apache NiFi and Amazon Polly for Machine Learning https://medium.com/@tspann/harnessing-the-power-of-apache-nifi-and-amazon-polly-for-machine-learning-4ea3139fbe77

Building LLM Pipelines with Pinecone, HuggingFace, Python and Apache NiFi https://medium.com/@tspann/llm-pipelines-with-pinecone-and-huggingface-with-python-and-apache-nifi-a96c20be93b7

Writing A Gen AI Processor with Python https://medium.com/@tspann/writing-a-generative-ai-python-processor-ed0655cf4e3f

Raspberry Pi 5 Setup https://medium.com/@tspann/i-setup-too-many-sbcs-d6417081a200

Codeless Generative AI Pipelines with Chroma Vector DB & Apache NiFi https://medium.com/@tspann/codeless-generative-ai-pipelines-with-chroma-vector-db-apache-nifi-43e77d75952f

Using NiFi to Augment and Enrich LLM Results with Real-Time Contextual Data https://medium.com/@tspann/augmenting-and-enriching-llm-with-real-time-context-b6da7ba4960a

ReadyFlow with WatsonX https://community.cloudera.com/t5/Community-Articles/Processing-real-time-unstructured-data-with-GenAI-using/ta-p/378191

AWS Open Source https://community.aws/content/2bJFKCPKPttVH0yPHPPt3XHoZJR/aws-open-source-newsletter-184?lang=en

Checkpoint Chronicle December 2023 https://decodable.co/blog/checkpoint-chronicle-december-2023

ADSB with NiFi https://www.researchgate.net/publication/352469660_Near-Real-Time_IDS_for_the_US_FAA's_NextGen_ADS-B

NiFi Security https://exceptionfactory.com/posts/2021/07/21/single-user-access-and-https-in-apache-nifi/

4 Wars orf AI https://www.latent.space/p/dec-2023

DocLLM for PDF https://medium.com/@basics.machinelearning/discover-docllm-the-new-llm-from-jpmorgan-for-working-with-complex-documents-5f54ea287d52

GRPC and Protobuf are growing https://www.infoq.com/news/2023/12/linkedin-grpc-protobuf-rest-json/

Multi-Layered Cache https://www.infoq.com/news/2023/10/doordash-multilayered-cache/

CDF Updates https://community.cloudera.com/t5/What-s-New-Cloudera/Cloudera-DataFlow-adds-Change-Data-Capture-processors-flow/ba-p/381727

LLM https://www.infoq.com/articles/large-language-models-llms-prompting/

Top 10 Challenges to GenAI https://www.datanami.com/2024/01/22/top-10-challenges-to-genai-success/

Data Engineering in 2024 https://www.datanami.com/2024/01/23/data-engineering-in-2024-predictions-for-data-lakes-and-the-serving-layer/

EdgeAI https://docs.omniverse.nvidia.com/dev-guide/latest/index.html https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pyg#new_tab https://developer.nvidia.com/blog/generate-synthetic-data-for-deep-object-pose-estimation-training-with-nvidia-isaac-ros/https://developer.nvidia.com/blog/how-to-build-vision-ai-applications-at-the-edge-with-nvidia-metropolis-microservices-and-apis/ https://developer.nvidia.com/blog/bringing-generative-ai-to-the-edge-with-nvidia-metropolis-microservices-for-jetson/ https://developer.nvidia.com/blog/getting-started-on-jetson-top-resources-from-gtc-21/ https://developer.nvidia.com/blog/bringing-generative-ai-to-life-with-jetson/ https://www.jetson-ai-lab.com/

Fine Tuning LLM https://www.philschmid.de/fine-tune-llms-in-2024-with-trl

Use Markdown in Google https://support.google.com/docs/answer/12014036

Videos

Seven Videos on Real-Time Streaming https://medium.com/@tspann/seven-videos-on-real-time-streaming-02711320afa8

Unlocking Financial Data with Real-Time Pipelines (OSACon 2023) https://www.youtube.com/watch?v=Q7gF7m4yFi4&ab_channel=OSACon

Looking at the New Features of Apache NiFi (Halifax Community over Code) https://www.youtube.com/watch?v=_orD9aAXk48&ab_channel=TheASF

Utilizing Real-Time Transit Data for Travel Optimization (Halifax Community over Code) Sunday Oct 8 2023, Canada https://www.youtube.com/watch?v=OWQmeF-UeEc&ab_channel=TheASF

Continuous SQL with Kafka and Flink | Timothy Spann (EN) https://www.youtube.com/watch?v=IGs0k240zhU&ab_channel=JAVAPRO

Events

Feb 8, 2024: NYC. https://www.meetup.com/new-york-open-source-data-infrastructure-meetup/events/297484047/

18:00 - 18:30 Welcome: Networking & snacks 18:30 - 18:35 Kickoff: Welcome Aiven 18:35 - 19:00 A Guide to Product Experimentation (Erin Mikail Staples, LaunchDarkly) 19:00 - 19:30 Building Real-time Pipelines: A Case Study with Transit Data (Tim Spann, Cloudera) 19:30 ~ 21:00 Food & networking

Feb 2024: Webinar https://www.cloudera.com/about/events/webinars/stay-ahead-of-cyber-threats-by-utilizing-data-in-motion.html?utm_medium=virtual-event&utm_source=resources-module&keyplay=ALL&utm_campaign=FY25-Q1-CorporateWebinar-AMER-cyber-threats&cid=701Hr000001pXCQIA2

Feb 20, 2024: 12-1PM EST. Virtual. Azure Data Tech Groups: DBA Fundamentals Group https://www.meetup.com/dba-fundamentals-group/events/296855261/

Feb 28, 2024: NYC. Cloudera Meetup. Flink https://www.meetup.com/futureofdata-princeton/events/298661947/

March 5, 2024: Princeton. Meetup. GenAI. https://www.meetup.com/applied-generative-artificial-intelligence-applications/

March 15, 2024: Princeton. IT Professional Conference at Trenton Computer Festival IEEE Information Technology Professional Conference on Friday, March 15th, 2024 https://princetonacm.acm.org/tcfpro/

April 2024: XtremeJ 2024. Virtual. https://xtremej.dev/2023/schedule/

Cloudera Events https://www.cloudera.com/about/events.html

More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

Models

Tools

© 2020-2024 Tim Spann