FLaNK Stack 04 March 2024
Tim Spann @PaaSDev
https://www.youtube.com/@FLaNK-Stack
https://www.threads.net/@tspannhw
https://medium.com/@tspann/subscribe
https://www.cloudera.com/campaign/apache-nifi-for-dummies.html
https://ossinsight.io/analyze/tspannhw
Please join my meetup group NJ/NYC/Philly/Virtual.
http://www.meetup.com/futureofdata-princeton/
https://www.meetup.com/futureofdata-newyork/
https://www.meetup.com/futureofdata-philadelphia/
**This is Issue #127 **
https://github.com/tspannhw/FLiPStackWeekly
https://www.cloudera.com/solutions/dim-developer.html
Apache Kafka 3.7.0
https://kafka.apache.org/blog#apache_kafka_370_release_announcement
https://www.youtube.com/watch?v=mEsleV16qdo&ab_channel=freeCodeCamp.org
Yet another Python Processor https://medium.com/@tspann/yet-another-python-processor-45aaae6fe406
Streaming Street Cams to YoLo v8 with Python and NiFi to MinIO (S3) https://medium.com/@tspann/streaming-street-cams-to-yolo-v8-with-python-and-nifi-to-minio-s3-3277e73723ce
Meetup Report 28 Feb 2024 https://medium.com/@tspann/report-28-feb-2024-building-realtime-ai-applications-with-apache-flink-76edb957b996
Using OLLAMA with Mistral and Apache NiFi https://medium.com/@tspann/using-ollama-with-mistral-and-apache-nifi-720c17f5ff12
Python to Apache Iceberg https://medium.com/@tspann/python-to-apache-iceberg-s-5d642e1170ae https://www.youtube.com/watch?v=pRTNQ2Ddu88
Using Google Gemma https://medium.com/@tspann/google-gemma-for-real-time-lightweight-open-llm-inference-88efe98e580f
NYC Traffic?? (NiFi, Kafka, Flink) https://medium.com/@tspann/nyc-traffic-are-you-kidding-me-6d3fa853903b
Subways and Transit Updates in Real-Time https://medium.com/@tspann/subways-and-transit-updates-in-real-time-30c104c359ef
Open Source Data Infrastructure Meetup - Feb 2024 https://medium.com/@tspann/open-source-data-infrastructure-meetup-feb-2024-9e8048666828
https://towardsdatascience.com/all-public-transport-leads-to-utrecht-not-rome-bb9674600e81
https://datavolo.io/2024/02/collecting-logs-with-apache-nifi-and-opentelemetry/
https://zilliz.com/learn/milvus-vector-database-quickstart
https://echarts.apache.org/handbook/en/get-started/
https://www.decodable.co/blog/flink-sql-and-the-joy-of-jars?
https://www.philschmid.de/dpo-align-llms-in-2024-with-trl?
https://www.infoq.com/articles/architecting-java-persistence-patterns-and-strategies/
https://gonzoml.substack.com/p/big-post-about-big-context
https://ben11kehoe.medium.com/the-end-of-programming-will-look-a-lot-like-programming-8b877c8efef8
https://apiiro.com/blog/malicious-code-campaign-github-repo-confusion-attack/
https://vickiboykis.com/2024/02/28/gguf-the-long-way-around/
https://thenewstack.io/the-new-monitoring-for-services-that-feed-from-llms/?
https://nagarajtantri.medium.com/chaining-multiple-http-apis-via-apache-nifi-72c4d14c072d
Streaming Traffic Cameras https://www.youtube.com/watch?v=85ECRGJBEQU&ab_channel=DatainMotion-HowToBeaStreamingEngineer
Joining Three Kafka Topics in Flink SQL https://youtu.be/NI2n7uQJiP0?si=0aAFrkhOdqzZKisw
Continuous SQL with Kafka and Flink https://www.youtube.com/watch?v=0Fb8ggZlPrQ&ab_channel=stevecantrell
Building Real-time Pipelines: A Case Study by Transit Data https://www.youtube.com/watch?v=VjmC4J7KZgw&t=2s&ab_channel=Aiven
https://www.youtube.com/watch?v=29JnbO6LL6g
https://www.youtube.com/watch?v=0cdGwP3Shxs
https://www.youtube.com/watch?v=H7uUDLo_XI0
https://www.youtube.com/watch?v=awxzG7laWx4&ab_channel=Conf42
https://www.youtube.com/watch?v=FD16_oZ65Ug&ab_channel=Conf42
March 11, 2024: Princeton. Meetup. GenAI. https://www.meetup.com/applied-generative-artificial-intelligence-applications/ https://23orchard.com/
March 15, 2024: TCF Pro. Princeton, NJ. IT Professional Conference at Trenton Computer Festival IEEE Information Technology Professional Conference on Friday, March 15th, 2024 https://princetonacm.acm.org/tcfpro/
March 27, 2024: Startup Grind. Jersey City https://www.startupgrind.com/events/details/startup-grind-princeton-presents-startup-grind-princeton-amp-nj-big-data-alliance-generative-ai-reverse-pitch/
March 28, 2024: Pinot + NiFi + Flink + Kafka Meetup NYC https://www.meetup.com/real-time-analytics-meetup-ny/events/299290822/
April 2024: XtremeJ 2024. Virtual. https://xtremej.dev/2023/schedule/
April 8-11, 2024: NLIT Summit. Seattle. https://www.fbcinc.com/e/nlit/default.aspx 
April 11, 2024: Conf42 LLM. Virtual. https://www.conf42.com/llms2024
April 2024: AI Meetup NJ https://www.meetup.com/nj-gai/
May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx
Cloudera Events https://www.cloudera.com/about/events.html
More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe
- https://github.com/tspannhw/FLaNK-python-processors
- https://github.com/tspannhw/FLaNK-IceIceData
- https://github.com/tspannhw/PaK-Stocks
- https://github.com/tspannhw/meetups/tree/main/28feb2024
- https://github.com/SuperEllipse/LLM-demo-on-CML
- https://github.com/salesforce/LAVIS
- https://github.com/lm-sys/FastChat
- https://github.com/salesforce/LAVIS/blob/main/examples/blip_image_captioning.ipynb
- https://llm.mlc.ai/docs/
- https://docs.stackable.tech/home/stable/demos/data-lakehouse-iceberg-trino-spark.html
- https://traefik.me/
- https://hub.docker.com/r/gooddata/gooddata-cn-ce
- https://medium.com/plain-simple-software/software-engineer-to-devrel-a-guide-13d4bee97631
- https://huggingface.co/spaces/etri-vilab/KOALA
- https://github.com/Azure/PyRIT
- https://github.com/youngwanLEE/sdxl-koala
- https://shop.sb-components.co.uk/collections/raspberry-pi-pico/products/ardipi-uno-r3-alternative-board-based-on-pico-w
- https://aihub.qualcomm.com/models
- https://developer.nvidia.com/blog/build-an-llm-powered-api-agent-for-task-execution/
- https://github.com/NVIDIA/GenerativeAIExamples?nvid=nv-int-tblg-585510
- https://github.com/explodinggradients/ragas
- https://github.com/eladlev/autoprompt
- https://arxiv.org/abs/2402.17764
- https://github.com/ryogesh/llm-rag-graph
- https://github.com/pgvector/pgvector
- https://chartfox.org/
- https://github.com/AlmasB/FXGL
- https://github.com/bruin-data/ingestr
- https://github.com/baverman/sqlbind
- https://pgplayground.com/
- https://github.com/ytang07/ai_agents_cookbooks
- https://github.com/yerfor/GeneFacePlusPlus
- https://github.com/yerfor/Real3DPortrait
- https://pygments.org/demo/
- https://github.com/vectara/react-search
- https://github.com/deptofdefense/AndroidTacticalAssaultKit-CIV
- https://gist.github.com/loreanvictor/bddd8824c744024d338e935bd7e96707
- https://github.com/dotenv-org/dotenv-vault
- https://pql.dev/
- https://sql-workbench.com/
- https://slidesynth.com/
- https://github.com/slingdata-io/sling-cli
© 2020-2024 Tim Spann
FLaNK Stack 26 February 2024
FLaNK Stack 26 February 2024
26-February-2024
FLaNK Stack Weekly
Tim Spann @PaaSDev
https://www.youtube.com/@FLaNK-Stack
https://www.threads.net/@tspannhw
https://medium.com/@tspann/subscribe
https://www.cloudera.com/campaign/apache-nifi-for-dummies.html
https://ossinsight.io/analyze/tspannhw
CODE + COMMUNITY
Please join my meetup group NJ/NYC/Philly/Virtual.
http://www.meetup.com/futureofdata-princeton/
https://www.meetup.com/futureofdata-newyork/
https://www.meetup.com/futureofdata-philadelphia/
**This is Issue #126 **
https://github.com/tspannhw/FLiPStackWeekly
https://www.cloudera.com/solutions/dim-developer.html
Articles
Using Google Gemma https://medium.com/@tspann/google-gemma-for-real-time-lightweight-open-llm-inference-88efe98e580f
NYC Traffic?? (NiFi, Kafka, Flink) https://medium.com/@tspann/nyc-traffic-are-you-kidding-me-6d3fa853903b
Subways and Transit Updates in Real-Time https://medium.com/@tspann/subways-and-transit-updates-in-real-time-30c104c359ef
Open Source Data Infrastructure Meetup - Feb 2024 https://medium.com/@tspann/open-source-data-infrastructure-meetup-feb-2024-9e8048666828
https://sap1ens.com/blog/2024/02/18/customizing-flink-class-shadowing/
https://engineering.grab.com/attribution-platform
https://amistrongeryet.substack.com/p/why-are-llms-so-gullible
https://huggingface.co/blog/gemma
https://developer.nvidia.com/blog/build-an-llm-powered-data-agent-for-data-analysis/
https://thenewstack.io/the-rise-of-small-language-models/
https://www.infoq.com/news/2024/02/pinterest-pubsub-kafka-flink/
https://www.infoq.com/news/2024/01/doordash-service-mesh/
https://thenewstack.io/demo-use-webassembly-to-run-llms-on-your-own-device-with-wasmedge
https://www.eleuther.ai/releases
https://www.microsoft.com/en-us/research/blog/orca-2-teaching-small-language-models-how-to-reason/
https://www.baeldung.com/ops/docker-remove-dangling-unused-images
AI + More required for startup https://www.nfx.com/post/ai-like-water
https://explainextended.com/2023/12/31/happy-new-year-15/
https://medium.com/sids-tech-cafe/event-driven-systems-lessons-from-the-trenches-107c07b3fc1d
https://materializedview.io/p/from-samza-to-flink-a-decade-of-stream
https://exaspark.medium.com/the-ultimate-guide-to-postgresql-data-change-tracking-c3fa88779572
https://www.wired.com/story/17-tips-better-chatgpt-prompts
https://github.com/microsoft/generative-ai-for-beginners/
Videos
Continuous SQL with Kafka and Flink https://www.youtube.com/watch?v=0Fb8ggZlPrQ&ab_channel=stevecantrell
Building Real-time Pipelines: A Case Study by Transit Data https://www.youtube.com/watch?v=VjmC4J7KZgw&t=2s&ab_channel=Aiven
Unlocking Financial Data with Real-Time Pipelines (OSACon 2023) https://www.youtube.com/watch?v=Q7gF7m4yFi4&ab_channel=OSACon
The Never Landing Stream https://www.youtube.com/watch?v=M8Bp0tRGvV0
https://www.youtube.com/watch?v=gSvvBHBWq20
https://www.youtube.com/watch?v=ayAGiPd2zq4&t=1s
February 8, 2024 NYC Meetup
February 20, 2024 Virtual Meetup
https://www.slideshare.net/slideshows/dba-fundamentals-group-continuous-sql-with-kafka-and-flink/266403113 https://www.youtube.com/watch?v=0Fb8ggZlPrQ&ab_channel=stevecantrell
Feb 22, 2024 NYC Meetup
Events
Feb 28, 2024: NYC. Cloudera Meetup. Flink https://www.meetup.com/futureofdata-princeton/events/298661947/
Feb 29, 2024: Virtual. Conf42 Python. https://www.conf42.com/Python_2024_Tim_Spann_apache_nifi_2_processors
https://www.conf42.com/Python_2024_Karin_Wolok_nifi__kafka_risingwave_iceberg_llm
Soon, 2024: Princeton. TigerLabs New Location. Meetup. GenAI. https://www.meetup.com/applied-generative-artificial-intelligence-applications/
March 15, 2024: TCF Pro. Princeton, NJ. IT Professional Conference at Trenton Computer Festival IEEE Information Technology Professional Conference on Friday, March 15th, 2024 https://princetonacm.acm.org/tcfpro/
March 28, 2024: Pinot + NiFi + Flink + Kafka Meetup NYC https://www.meetup.com/real-time-analytics-meetup-ny/events/299290822/
April 2024: XtremeJ 2024. Virtual. https://xtremej.dev/2023/schedule/
April 11, 2024: Conf42 LLM. Virtual. https://www.conf42.com/llms2024
May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx
Cloudera Events https://www.cloudera.com/about/events.html
More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe
Code
- https://github.com/tspannhw/FLaNK-python-watsonx-processor
- https://github.com/thammuio/doc-genius-ai
- https://github.com/tspannhw/FLaNK-python-processors
Models
- https://github.com/ncbi/GeneGPT
- https://www.arxiv.org/abs/2402.03405
- https://huggingface.co/foduucom/stockmarket-pattern-detection-yolov8
- https://github.com/WongKinYiu/yolov9
Tools
- https://github.com/photopea/photopea
- https://redash.io/
- https://lookatme.readthedocs.io/en/latest/getting_started.html
- https://gist.github.com/johnloy/27dd124ad40e210e91c70dd1c24ac8c8
- https://prql-lang.org/
- https://fonts.google.com/selection
- https://www.kineticedge.io/blog/ktools-kafka-topic-truncate/
- https://htmx.org/
- https://deervo.itch.io/diskclick
- https://leanrada.com/htmz/
- https://groq.com/
- https://news.mit.edu/2024/tiny-tamper-proof-id-tag-can-authenticate-almost-anything-0218
- https://github.com/awslabs/llrt
- https://observablehq.com/framework/getting-started
- https://academy.datawrapper.de/article/384-how-to-create-small-multiple-line-charts
- https://github.com/enjalot/latent-scope
- https://github.com/IntelSoftware/Python-Loop-Replacement-with-NumPy-and-PyTorch
- https://dmarcchecker.app/
- https://github.com/gcarmix/HexWalk
- https://markmap.js.org/repl
- https://github.com/plantuml/plantuml
- https://predibase.com/blog/lora-land-fine-tuned-open-source-llms-that-outperform-gpt-4
- https://github.com/microsoft/UFO
- https://github.com/datadreamer-dev/datadreamer
- https://thealliance.ai/news
- https://engineering.fb.com/2022/03/10/security/code-verify/
- https://www.sciencedaily.com/releases/2024/02/240216135820.htm
- https://atuin.sh/
- https://github.com/simulaiofficial/simulai
- https://hyperdiv.io/
- https://github.com/OpenMOSS/AnyGPT
- https://github.com/Dashibase/lotion
- https://github.com/microsoft/JARVIS
- https://github.com/ariya/pico-jarvis
- https://github.com/ibis-project/ibis
- https://www.sivalabs.in/langchain4j-ai-services-tutorial/
- https://github.com/weaviate/weaviate-examples/tree/main
- https://github.com/weaviate/weaviate-examples/tree/main/clip-multi-modal-text-image-search
- https://huggingface.co/docs/transformers/model_doc/gptj
- https://github.com/EleutherAI/gpt-neox/
- https://github.com/weaviate-tutorials/DEMO-multimodal-search
- https://github.com/cloudera/CML_llm-hol
- https://github.com/Mozilla-Ocho/llamafile
- https://pagescms.org/
- https://github.com/erfanzar/EasyDeL
- https://github.com/bots-garden/pi-genai-stack
- https://spring.io/blog/2024/02/23/spring-ai-0-8-0-released
- https://github.com/Azure/PyRIT
- https://github.com/amithkoujalgi/ollama4j
- https://github.com/dustinblackman/oatmeal
- https://docs.spring.io/spring-ai/reference/api/clients/ollama-chat.html
- https://opensource.expediagroup.com/stream-registry/
- https://github.com/ExpediaGroup/beekeeper
- https://matklad.github.io/2021/02/06/ARCHITECTURE.md.html
- https://github.com/Frimkron/mud-pi
- https://pylint.readthedocs.io/en/latest/pyreverse.html
- https://electric-sql.com/
- https://medium.com/hashmapinc/nifi-nar-files-explained-14113f7796fd
- https://github.com/OpenCodeInterpreter/OpenCodeInterpreter
- https://github.com/tstack/lnav
- https://github.com/microsoft/FASTER
- https://github.com/ok-robot/ok-robot
- https://github.com/google/gemma.cpp
- https://github.com/Victormeriqui/Consol3
- https://github.com/chand1012/sq
- https://github.com/mukovnin/psfiles
Notable Tools
Postgresql + MySQL Cache https://github.com/readysettech/readyset
NVIDIA GPU LLM https://github.com/NVIDIA/TensorRT-LLM
Configuration Management Server https://caddyserver.com/features
Fast Text to Image https://fastsdxl.ai/
Very Interesting Remote tool for OBS https://vdo.ninja/
Commands Du Jour
docker system prune -a docker image prune -a docker system df docker ps docker logs name
© 2020-2024 Tim Spann