FLaNK AI Weekly 18 March 2024

 

18-March-2024

FLaNK Stack Weekly

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

Congrats to my wife for being the youngest Leader of our local Elks!

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

**This is Issue #129 **

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

New Releases

https://cldr-steven-matison.github.io//blog/CEM-2.1.2-Release/

Articles

Image Processing with Custom Python and Apache NiFi 2.0 https://medium.com/@tspann/image-processing-with-custom-python-and-nifi-2-0-06eadc62c03c

Mixtral Deep Dive https://dzone.com/articles/mixtral-generative-sparse-mixture-of-experts-in-da

AI Augmented DevRel part 1 https://medium.com/@tspann/ai-augmented-devrel-part-1-4058af905a89

Next Level Flink with Nussknacker https://medium.com/@tspann/next-level-flink-with-nussknacker-fe7294e2ef21

Mixtral Generative Sparse Mixture of Experts in DataFlows https://medium.com/@tspann/mixtral-generative-sparse-mixture-of-experts-in-dataflows-59744f7d28a9

https://news.mit.edu/2024/researchers-enhance-peripheral-vision-ai-models-0308

https://medium.com/@1709deepesh/connecting-apache-nifi-to-microsoft-graph-reading-emails-with-invokehttp-processors-6d84db9fa157

https://community.cloudera.com/t5/Community-Articles/How-to-call-a-CML-Deployed-Model-From-Apache-NiFi-in-10/ta-p/374853

https://www.infoq.com/news/2024/03/java-22-so-far/

https://www.infoq.com/news/2024/03/lapce-rust-editor

https://www.infoq.com/news/2024/03/mistral-ai-aws/

https://www.infoq.com/news/2024/03/anthropic-claude-ai/

https://www.quantamagazine.org/new-breakthrough-brings-matrix-multiplication-closer-to-ideal-20240307/

https://venturebeat.com/ai/hugging-face-is-launching-an-open-source-robotics-project-led-by-former-tesla-scientist/?

https://dbos-project.github.io/

https://www.datanami.com/2024/03/07/cloudera-unveils-next-phase-of-open-data-lakehouse-to-unlock-enterprise-ai/?

https://www.decodable.co/blog/taxonomy-of-data-change-events

https://www.europarl.europa.eu/news/en/press-room/20240308IPR19015/artificial-intelligence-act-meps-adopt-landmark-law

https://medium.com/plain-simple-software/the-llm-app-stack-2024-eac28b9dc1e7

https://www.slideshare.net/JulienSIMON5/an-introduction-to-computer-vision-with-hugging-face

https://huggingface.co/learn/nlp-course/chapter1/2?fw=pt

https://huggingface.co/timm

https://github.com/huggingface/pytorch-image-models

https://www.slideshare.net/JulienSIMON5/an-introduction-to-computer-vision-with-hugging-face

https://www.infoq.com/news/2024/03/azure-openai-your-data-ga/

https://developers.redhat.com/articles/2024/03/13/kafka-tiered-storage-deep-dive?

https://community.cloudera.com/t5/What-s-New-Cloudera/Cloudera-DataFlow-adds-Change-Data-Capture-processors-flow/ba-p/381727

Videos

Streaming Traffic Cameras https://www.youtube.com/watch?v=85ECRGJBEQU&ab_channel=DatainMotion-HowToBeaStreamingEngineer

Python Processor https://www.youtube.com/watch?v=jF5FSY0xFiQ&t=9s&ab_channel=DatainMotion-HowToBeaStreamingEngineer

Preview of TCF Pro Talk https://youtu.be/ce9lhtbp48M?si=Svjb2-bIIPXLwXD1

Feb 22, 2024 NYC Meetup

https://www.slideshare.net/slideshows/2024-feb-ai-meetup-nyc-genaillmsmldata-codeless-generative-ai-pipelines/266444687

Feb 28, 2024 NYC Flink Meetup

https://www.slideshare.net/slideshows/2024-february-28-nyc-meetup-unlocking-financial-data-with-realtime-pipelines/266539528

Feb 29, 2024 Conf42 Python 2024

https://www.slideshare.net/slideshows/conf42python-using-apache-nifi-apache-kafka-risingwave-and-apache-iceberg-with-stock-data-and-llm/266521940

https://www.slideshare.net/slideshows/conf42pythonbuilding-apache-nifi-20-python-processors/266522007

https://www.youtube.com/watch?v=awxzG7laWx4&ab_channel=Conf42

https://www.youtube.com/watch?v=FD16_oZ65Ug&ab_channel=Conf42

March 11, 2024 Princeton 23 Orchard Event

https://www.slideshare.net/slideshows/2024-build-generative-ai-for-nonprofits/266748822

march 15, 2024 Trenton TCF

https://www.slideshare.net/slideshows/tcfpro24-building-realtime-generative-ai-pipelines/266807785

Events

March 27, 2024: Startup Grind. Jersey City https://www.startupgrind.com/events/details/startup-grind-princeton-presents-startup-grind-princeton-amp-nj-big-data-alliance-generative-ai-reverse-pitch/

March 28, 2024: Pinot + NiFi + Flink + Kafka Meetup NYC https://www.meetup.com/real-time-analytics-meetup-ny/events/299290822/

April 2, 2024: XtremeJ 2024. Virtual. https://xtremej.dev/2023/schedule/

April 8-11, 2024: NLIT Summit. Seattle. https://www.fbcinc.com/e/nlit/default.aspx 

April 11, 2024: Conf42 LLM. Virtual. https://www.conf42.com/llms2024

April 12, 2024: AI Max Conference. 23 Orchard Princeton https://www.startupgrind.com/events/details/startup-grind-princeton-presents-startup-grind-hosts-ai-max-summit/

April 2024: AI Meetup NJ https://www.meetup.com/nj-gai/

May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx

Cloudera Events https://www.cloudera.com/about/events.html

More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

• https://github.com/tspannhw/FLaNK-python-processors

Models

• https://github.com/urchade/GLiNER
• https://github.com/deepseek-ai/DeepSeek-VL
• https://github.com/sieve-community/fast-asd
• https://arxiv.org/abs/2403.09611
• https://github.com/xai-org/grok-1

Datasets

• https://hf.co/datasets

Tools

• https://github.com/echasnovski/mini.nvim/blob/main/readmes/mini-indentscope.md
• https://datavolo.io/2024/03/data-engineering-for-advanced-rag-small-to-big-with-pinecone-langchain-and-datavolo/
• https://docs.pinecone.io/docs/metadata-filtering
• https://python.langchain.com/docs/modules/data_connection/retrievers/parent_document_retriever
• https://docs.pinecone.io/reference/list
• https://colab.research.google.com/drive/1AvPpRvzLvGPMG3vVgcRDOvSdar_lsTks
• https://github.com/pi-ra/beesy-issue-tracker
• https://palindromicity.blogspot.com/2020/06/dotifi-generating-dot-files-from-apache.html
• https://github.com/flxzt/rnote
• https://github.com/teableio/teable
• https://localsend.org/#/
• https://github.com/leobeeson/llm_benchmarks
• https://www.cognition-labs.com/blog
• https://github.com/openvinotoolkit/openvino_notebooks/tree/recipes/recipes/defect_detection_anomalib
• https://brave.com/
• https://github.com/PeerDB-io/peerdb
• https://osm2pgsql.org/
• https://arc.net/
• https://github.com/truera/trulens
• https://github.com/has2k1/plotnine
• https://github.com/altair-viz/altair
• https://github.com/quarto-dev/quarto-cli
• https://github.com/tobymao/sqlglot
• https://github.com/quarkiverse/quarkus-langchain4j
• https://github.com/ELLA-Diffusion/ELLA
• https://github.com/skills-cogrammar/C7-Lecture-Backpack
• https://github.com/LucasPickering/slumber
• https://webhook.site/
• https://github.com/paveldedik/ludic
• https://github.com/bananaml/fructose
• https://github.com/betwixt-labs/bebop
• https://github.com/jafioti/luminal
• https://github.com/soorajshankar/logScreen
• https://github.com/flydelabs/flyde
• https://lite.ip2location.com/ip2location-lite
• https://github.com/getindata/flink-http-connector
• https://github.com/phospho-app/phospho
• https://github.com/phospho-app/fastassert
• https://github.com/developersdigest/llm-answer-engine
• https://docs.litellm.ai/docs/proxy/quick_start
• https://letsbuild.ai/
• https://github.com/albertan017/LLM4Decompile
• https://vector.dev/
• https://github.com/ArroyoSystems/arroyo
• https://github.com/stanfordnlp/pyvene
• https://www.mewho.com/titan/
• https://radicle.xyz/
• https://github.com/ianand/spreadsheets-are-all-you-need

Tips

https://www.datainmotion.dev/2020/05/one-minute-nifi-tip-calcite-sql-notes.html

Cool Tool

These are amazing diagrams and graphics.
https://drawify.com/templates/341/personal-user-manual

© 2020-2024 Tim Spann

FLaNK Stack 04 March 2024

04-March-2024

FLaNK Stack Weekly

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

**This is Issue #127 **

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

Project Updates

Apache Kafka 3.7.0

https://kafka.apache.org/blog#apache_kafka_370_release_announcement

Courses

https://www.youtube.com/watch?v=mEsleV16qdo&ab_channel=freeCodeCamp.org

Articles

Yet another Python Processor https://medium.com/@tspann/yet-another-python-processor-45aaae6fe406

Streaming Street Cams to YoLo v8 with Python and NiFi to MinIO (S3) https://medium.com/@tspann/streaming-street-cams-to-yolo-v8-with-python-and-nifi-to-minio-s3-3277e73723ce

Meetup Report 28 Feb 2024 https://medium.com/@tspann/report-28-feb-2024-building-realtime-ai-applications-with-apache-flink-76edb957b996

Using OLLAMA with Mistral and Apache NiFi https://medium.com/@tspann/using-ollama-with-mistral-and-apache-nifi-720c17f5ff12

Python to Apache Iceberg https://medium.com/@tspann/python-to-apache-iceberg-s-5d642e1170ae https://www.youtube.com/watch?v=pRTNQ2Ddu88

Using Google Gemma https://medium.com/@tspann/google-gemma-for-real-time-lightweight-open-llm-inference-88efe98e580f

NYC Traffic?? (NiFi, Kafka, Flink) https://medium.com/@tspann/nyc-traffic-are-you-kidding-me-6d3fa853903b

Subways and Transit Updates in Real-Time https://medium.com/@tspann/subways-and-transit-updates-in-real-time-30c104c359ef

Open Source Data Infrastructure Meetup - Feb 2024 https://medium.com/@tspann/open-source-data-infrastructure-meetup-feb-2024-9e8048666828

https://towardsdatascience.com/all-public-transport-leads-to-utrecht-not-rome-bb9674600e81

https://datavolo.io/2024/02/collecting-logs-with-apache-nifi-and-opentelemetry/

https://zilliz.com/learn/milvus-vector-database-quickstart

https://exceptionfactory.com/posts/2024/02/26/building-opentelemetry-collection-in-apache-nifi-with-netty/

https://echarts.apache.org/handbook/en/get-started/

https://www.decodable.co/blog/flink-sql-and-the-joy-of-jars?

https://techcrunch.com/2024/02/28/diffusion-transformers-are-the-key-behind-openais-sora-and-theyre-set-to-upend-genai/

https://www-bleepingcomputer-com.cdn.ampproject.org/c/s/www.bleepingcomputer.com/news/security/malicious-ai-models-on-hugging-face-backdoor-users-machines/amp/

https://www.philschmid.de/dpo-align-llms-in-2024-with-trl?

https://www.infoq.com/articles/architecting-java-persistence-patterns-and-strategies/

https://docs.cloudera.com/cdp-public-cloud-preview-features/cloud/dw-hue-sql-ai-assistant/dw-hue-sql-ai-assistant.pdf

https://medium.com/@yogi_r/relationship-graphs-using-llm-with-retrieval-augmented-generation-rag-and-vector-database-d3f12c914ade

https://news.samsung.com/global/samsungs-new-microsd-cards-bring-high-performance-and-capacity-for-the-new-era-in-mobile-computing-and-on-device-ai?cid=sem-mktg-pfs-mob-us-other-na-01312024-141981-

https://gonzoml.substack.com/p/big-post-about-big-context

https://ben11kehoe.medium.com/the-end-of-programming-will-look-a-lot-like-programming-8b877c8efef8

https://apiiro.com/blog/malicious-code-campaign-github-repo-confusion-attack/

https://vickiboykis.com/2024/02/28/gguf-the-long-way-around/

https://newsroom.ibm.com/2024-02-29-IBM-Announces-Availability-of-Open-Source-Mistral-AI-Model-on-watsonx,-Expands-Model-Choice-to-Help-Enterprises-Scale-AI-with-Trust-and-Flexibility

https://thenewstack.io/the-new-monitoring-for-services-that-feed-from-llms/?

https://nagarajtantri.medium.com/chaining-multiple-http-apis-via-apache-nifi-72c4d14c072d

https://webchick.hashnode.dev/no-one-gives-a-bleep-about-your-devrel-community-programs-and-what-to-do-about-it-2-collaboration

https://webchick.tech/no-one-gives-a-bleep-about-your-devrel-community-programs-and-what-to-do-about-it-1-organizational-alignment

Videos

Streaming Traffic Cameras https://www.youtube.com/watch?v=85ECRGJBEQU&ab_channel=DatainMotion-HowToBeaStreamingEngineer

Joining Three Kafka Topics in Flink SQL https://youtu.be/NI2n7uQJiP0?si=0aAFrkhOdqzZKisw

Continuous SQL with Kafka and Flink https://www.youtube.com/watch?v=0Fb8ggZlPrQ&ab_channel=stevecantrell

Building Real-time Pipelines: A Case Study by Transit Data https://www.youtube.com/watch?v=VjmC4J7KZgw&t=2s&ab_channel=Aiven

https://www.youtube.com/watch?v=29JnbO6LL6g

https://www.youtube.com/watch?v=0cdGwP3Shxs

https://www.youtube.com/watch?v=H7uUDLo_XI0

Feb 22, 2024 NYC Meetup

https://www.slideshare.net/slideshows/2024-feb-ai-meetup-nyc-genaillmsmldata-codeless-generative-ai-pipelines/266444687

Feb 28, 2024 NYC Flink Meetup

https://www.slideshare.net/slideshows/2024-february-28-nyc-meetup-unlocking-financial-data-with-realtime-pipelines/266539528

Feb 29, 2024 Conf42 Python 2024

https://www.slideshare.net/slideshows/conf42python-using-apache-nifi-apache-kafka-risingwave-and-apache-iceberg-with-stock-data-and-llm/266521940

https://www.slideshare.net/slideshows/conf42pythonbuilding-apache-nifi-20-python-processors/266522007

https://www.youtube.com/watch?v=awxzG7laWx4&ab_channel=Conf42

https://www.youtube.com/watch?v=FD16_oZ65Ug&ab_channel=Conf42

Events

March 11, 2024: Princeton. Meetup. GenAI. https://www.meetup.com/applied-generative-artificial-intelligence-applications/ https://23orchard.com/

March 15, 2024: TCF Pro. Princeton, NJ. IT Professional Conference at Trenton Computer Festival IEEE Information Technology Professional Conference on Friday, March 15th, 2024 https://princetonacm.acm.org/tcfpro/

March 27, 2024: Startup Grind. Jersey City https://www.startupgrind.com/events/details/startup-grind-princeton-presents-startup-grind-princeton-amp-nj-big-data-alliance-generative-ai-reverse-pitch/

March 28, 2024: Pinot + NiFi + Flink + Kafka Meetup NYC https://www.meetup.com/real-time-analytics-meetup-ny/events/299290822/

April 2024: XtremeJ 2024. Virtual. https://xtremej.dev/2023/schedule/

April 8-11, 2024: NLIT Summit. Seattle. https://www.fbcinc.com/e/nlit/default.aspx 

April 11, 2024: Conf42 LLM. Virtual. https://www.conf42.com/llms2024

April 2024: AI Meetup NJ https://www.meetup.com/nj-gai/

May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx

Cloudera Events https://www.cloudera.com/about/events.html

More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

• https://github.com/tspannhw/FLaNK-python-processors
• https://github.com/tspannhw/FLaNK-IceIceData
• https://github.com/tspannhw/PaK-Stocks
• https://github.com/tspannhw/meetups/tree/main/28feb2024
• https://github.com/SuperEllipse/LLM-demo-on-CML

Models

• https://github.com/salesforce/LAVIS
• https://github.com/lm-sys/FastChat
• https://github.com/salesforce/LAVIS/blob/main/examples/blip_image_captioning.ipynb

Tools

• https://llm.mlc.ai/docs/
• https://docs.stackable.tech/home/stable/demos/data-lakehouse-iceberg-trino-spark.html
• https://traefik.me/
• https://hub.docker.com/r/gooddata/gooddata-cn-ce
• https://medium.com/plain-simple-software/software-engineer-to-devrel-a-guide-13d4bee97631
• https://huggingface.co/spaces/etri-vilab/KOALA
• https://github.com/Azure/PyRIT
• https://github.com/youngwanLEE/sdxl-koala
• https://shop.sb-components.co.uk/collections/raspberry-pi-pico/products/ardipi-uno-r3-alternative-board-based-on-pico-w
• https://aihub.qualcomm.com/models
• https://developer.nvidia.com/blog/build-an-llm-powered-api-agent-for-task-execution/
• https://github.com/NVIDIA/GenerativeAIExamples?nvid=nv-int-tblg-585510
• https://github.com/explodinggradients/ragas
• https://github.com/eladlev/autoprompt
• https://arxiv.org/abs/2402.17764
• https://github.com/ryogesh/llm-rag-graph
• https://github.com/pgvector/pgvector
• https://chartfox.org/
• https://github.com/AlmasB/FXGL
• https://github.com/bruin-data/ingestr
• https://github.com/baverman/sqlbind
• https://pgplayground.com/
• https://github.com/ytang07/ai_agents_cookbooks
• https://github.com/yerfor/GeneFacePlusPlus
• https://github.com/yerfor/Real3DPortrait
• https://pygments.org/demo/
• https://github.com/vectara/react-search
• https://github.com/deptofdefense/AndroidTacticalAssaultKit-CIV
• https://gist.github.com/loreanvictor/bddd8824c744024d338e935bd7e96707
• https://github.com/dotenv-org/dotenv-vault
• https://pql.dev/
• https://sql-workbench.com/
• https://slidesynth.com/
• https://github.com/slingdata-io/sling-cli

Notable Tools

Commands Du Jour

© 2020-2024 Tim Spann