FLaNK-AIM Weekly for 06 May 2024

 

06-May-2024

https://www.youtube.com/@FLaNK-Stack

FLaNK / KNIFe AI / FLaNK-AIM Weekly

http://knifeai.org/

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

**This is Issue #136 **

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

New Releases

Articles

https://medium.com/@tspann/small-language-models-sml-for-the-win-ea0c6fee8061

https://medium.com/@tspann/maybe-four-smaller-open-llm-s-are-better-than-one-93f78fb69eb9

https://medium.com/@tspann/building-a-milvus-connector-for-nifi-34372cb3c7fa

https://medium.com/@tspann/searching-slack-from-apache-nifi-9ed562aa2397

https://medium.com/@tspann/events-streams-flows-and-maps-22a8d27cd9b4

https://medium.com/@tspann/storing-meetup-user-data-as-events-dad3b1dc89f5

https://medium.com/@tspann/real-time-in-boston-part-1-0f92d7da3496

NSA AI Security https://www.nsa.gov/Press-Room/Press-Releases-Statements/Press-Release-View/Article/3741371/nsa-publishes-guidance-for-strengthening-ai-system-security/

https://zilliz.com/learn/Sentence-Transformers-for-Long-Form-Text

https://zilliz.com/zilliz-cloud-pipelines

https://huggingface.co/BAAI/bge-large-en-v1.5

https://github.com/cloudevents/sdk-python/blob/main/samples/http-json-cloudevents/client.py

https://medium.com/@tspann/building-a-milvus-connector-for-nifi-34372cb3c7fa

https://docs.cloudera.com/machine-learning/cloud/release-notes/topics/ml-whats-new.html#ml_workspace_resource_tags

https://zilliz.com/blog/finding-right-fit-embedding-support-for-RAG-in-zilliz-cloud-pipelines-from-voyageai-openai-and-oss

https://hazelcast.com/glossary/streaming-data/

https://postgres.ai/blog/20220525-common-db-schema-change-mistakes

https://medium.com/cloudera-inc/consuming-rss-feeds-from-flink-sql-eaf33c1a5a23

https://medium.com/cloudera-inc/adding-generative-ai-results-to-sql-streams-513e1fd2a6af

https://www.linuxfoundation.org/press/lf-ai-data-foundation-launches-open-platform-for-enterprise-ai-opea

https://blog.mozilla.ai/local-llm-as-judge-evaluation-with-lm-buddy-prometheus-and-llamafile/

https://blog.mozilla.ai/open-source-in-the-age-of-llms/

https://www.pinecone.io/learn/structured-data/

https://medium.com/@stoty/a-bug-for-ages-fixing-time-zone-handling-in-apache-phoenix-e9934d7acd80

https://www.geeknarrator.com/blog/stream-processing/stream-processing-concepts

https://blog.cloudera.com/setting-up-and-getting-started-with-clouderas-new-sql-ai-assistant/

https://thenewstack.io/how-to-cure-llm-weaknesses-with-vector-databases/

https://dev.to/zilliz/exploring-bge-m3-and-splade-two-machine-learning-models-for-generating-sparse-embeddings-22p1

https://zilliz.com/learn/transforming-pdfs-into-insights-vectorizing-and-ingesting-with-zilliz-cloud-pipelines

https://zilliz.com/blog/how-to-evaluate-and-optimize-performance-of-milvus-storage

https://datavolo.io/2024/05/apache-nifi-designed-for-extension-at-scale/

Videos

Generative AI with Milvus https://www.youtube.com/watch?v=IfWIzKsoHnA

Four Models at Once https://youtu.be/xvNgsZyfo6A?si=zxwc9VcFc3o0vU3P

Search Slack https://www.youtube.com/watch?v=3ugppfb2kN8&t=5s&ab_channel=DatainMotion-HowToBeaStreamingEngineer

MBTA Transit Live with LLM https://www.youtube.com/watch?v=JGGY_uzQTdY&t=3s&pp=ygUOVGltIFNwYW5uIE5pRmk%3D

Events, Streams, Maps with Irish Rail https://www.youtube.com/watch?v=14CSQRfUWoE&t=684s&pp=ygUOVGltIFNwYW5uIE5pRmk%3D

FLaNK AI Channel https://www.youtube.com/@FLaNK-Stack

NiFi https://www.youtube.com/watch?v=m-ZoqHOYy_k

Slides

https://www.slideshare.net/slideshow/generative-ai-on-enterprise-cloud-with-nifi-and-milvus/267678399

https://www.slideshare.net/slideshow/conf42llmadding-generative-ai-to-realtime-streaming-pipelines/267269788

https://github.com/tspannhw/FLaNK-Milvus

https://medium.com/cloudera-inc/building-a-milvus-connector-for-nifi-34372cb3c7fa

https://www.slideshare.net/slideshow/generative-ai-on-enterprise-cloud-with-nifi-and-milvus/267678399

https://www.youtube.com/watch?v=ssoM5S87BBs

Events

May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx https://www.dbta.com/DataSummit/2024/Timothy-Spann.aspx

https://twitter.com/DBTADataSummit/status/1778393005646397636

May 21, 2024: Gen AI and Beyond with NiFi 2.0. Virtual.

May 30, 2024: Conf42: Machine learning https://www.conf42.com/Machine_Learning_2024_Tim_Spann_enriching_generative_events

June 12, 2024: Budapest Data + ML Forum. Virtual.  https://budapestml.hu/2024/en/speakers/

June 20, 2024: AI Camp Meetup. NYC.

Sept 24, 2024: JConf.Dev. Dallas. https://2024.jconf.dev/session/598816

Nov 5-7, 10-12, 2024: CloudX. Online/Santa Clara. https://www.developerweek.com/cloudx/

Nov 19, 2024: XtremePython. Online. https://xtremepython.dev/2024/

Cloudera Events https://www.cloudera.com/about/events.html

https://www.cloudera.com/events/cloudera-now-cdp.html?internal_keyplay=ALL&internal_campaign=FY25-Q1-AMER-WS-Cloudera-Now-Events-Page-P06&cid=701Hr000000tW6qIAE&internal_link=p06

More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

• https://github.com/tspannhw/FLaNK-python-processors
• https://github.com/tspannhw/FLANKAI-Boston
• https://github.com/tspannhw/FLaNK-IrelandTransit

Models

• https://github.com/ollama/ollama/releases/tag/v0.1.33-rc5
• https://github.com/vikhyat/moondream
• https://github.com/kermitt2/grobid
• https://huggingface.co/apple/OpenELM-3B-Instruct
• https://huggingface.co/namespace-Pt/Llama-3-8B-Instruct-80K-QLoRA

Tools

• https://krasjet.com/voice/pdf.tocgen/
• https://github.com/rcoh/angle-grinder
• https://www.brandur.org/logfmt
• https://iss.matteason.co.uk/
• https://echarts.apache.org/handbook/en/concepts/dataset
• https://www.chartjs.org/docs/latest/getting-started/usage.html
• https://js.cytoscape.org/
• https://github.com/apexcharts/apexcharts.js/
• https://github.com/andrewcourtice/ripl
• https://unovis.dev/gallery
• https://vega.github.io/vega-lite/
• https://observablehq.com/@observablehq/plot-gallery
• https://www.tremor.so/
• https://github.com/gyf304/dotenv
• https://wanix.sh/
• https://os-world.github.io/
• https://ollama.com/library/moondream
• https://forums.developer.nvidia.com/t/introducing-ollama-support-for-jetson-devices/289333
• https://www.youtube.com/@FLaNK-Stack
• https://github.com/kaytu-io/kaytu
• https://github.com/hashhar/jsonschema2sql
• https://threlte.xyz/
• https://github.com/kingjulio8238/memary
• https://zilliz.com/learn/how-to-enhance-the-performance-of-your-rag-pipeline
• https://medium.com/cloudera-inc/consuming-rss-feeds-from-flink-sql-eaf33c1a5a23
• https://huggingface.co/bigcode/starcoder2-15b-instruct-v0.1
• https://docs.pinecone.io/examples/notebooks
• https://github.com/JetBrains/xodus
• https://github.com/towhee-io/examples/tree/main/nlp/text_search
• https://github.com/openvinotoolkit/openvino_notebooks/tree/recipes/recipes
• https://github.com/zilliztech/kafka-connect-milvus
• https://github.com/timeplus-io/proton/tree/develop/examples/real-time-ai
• https://zilliz.com/what-is-gptcache
• https://github.com/zilliztech/VectorDBBench
• https://github.com/zilliztech/milvus_cli
• https://github.com/towhee-io/examples/blob/main/nlp/text_search/search_article_in_medium.ipynb
• https://seeed-studio.github.io/SenseCraft-Web-Toolkit/#/setup/process
• https://github.com/McGill-NLP/webllama
• https://github.com/Mozilla-Ocho/llamafile
• https://prometheus-eval.github.io/prometheus
• https://github.com/mozilla-ai/lm-buddy
• https://github.com/databonsai/databonsai
• https://github.com/cohere-ai/cohere-toolkit
• https://github.com/InternLM/lmdeploy
• https://github.com/TagStudioDev/TagStudio
• https://github.com/Axorax/tkforge
• https://github.com/hiyouga/LLaMA-Factory
• https://pub.towardsai.net/rag-2-0-finally-getting-rag-right-f74d0194a720
• https://github.com/brettdidonato/trendr-bot
• https://github.com/cloudera/CML_AMP_Intelligent-QA-Chatbot-with-NiFi-Pinecone-and-Llama2
• https://github.com/pydantic/logfire
• https://www.terminal.shop/
• https://github.com/penpot
• https://github.com/ytang07/ai_agents_cookbooks/tree/main/langchain
• https://voxel51.com/fiftyone/
• https://huggingface.co/apple/OpenELM-450M-Instruct?library=true
• https://zilliz.com/learn/effortless-ai-workflows-a-beginners-guide-to-hugging-face-and-pymilvus
• https://github.com/NVlabs/DoRA
• https://github.com/Blealtan/efficient-kan
• https://github.com/habuma/spring-ai-examples/tree/main/vector-store-loader
• https://github.com/eureka-research/DrEureka
• https://haystack.deepset.ai/integrations/milvus-document-store
• https://github.com/TencentARC/InstantMesh
• https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B
• https://kai-waehner.medium.com/genai-demo-with-kafka-flink-langchain-and-openai-8c9b7263770a
• https://medium.com/@amanatulla1606/llm-web-scraping-with-scrapegraphai-a-breakthrough-in-data-extraction-d6596b282b4d
• https://news.mit.edu/2024/mit-faculty-instructors-students-experiment-generative-ai-teaching-learning-0429
• https://zilliz.com/blog/spring-ai-and-milvus-using-milvus-as-spring-ai-vector-store

Cool Tool

Convert Spark SQL to Trino SQL https://github.com/linkedin/coral

Discount

Discount access to DataSummit 2024 https://secure.infotoday.com/RegForms/DataSummit/?Priority=24SPKR

© 2020-2024 Tim Spann https://www.youtube.com/@FLaNK-Stack FLaNK-AIM with LLAMA 3

FLaNK AI Weekly 18 March 2024

 

18-March-2024

FLaNK Stack Weekly

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

Congrats to my wife for being the youngest Leader of our local Elks!

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

**This is Issue #129 **

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

New Releases

https://cldr-steven-matison.github.io//blog/CEM-2.1.2-Release/

Articles

Image Processing with Custom Python and Apache NiFi 2.0 https://medium.com/@tspann/image-processing-with-custom-python-and-nifi-2-0-06eadc62c03c

Mixtral Deep Dive https://dzone.com/articles/mixtral-generative-sparse-mixture-of-experts-in-da

AI Augmented DevRel part 1 https://medium.com/@tspann/ai-augmented-devrel-part-1-4058af905a89

Next Level Flink with Nussknacker https://medium.com/@tspann/next-level-flink-with-nussknacker-fe7294e2ef21

Mixtral Generative Sparse Mixture of Experts in DataFlows https://medium.com/@tspann/mixtral-generative-sparse-mixture-of-experts-in-dataflows-59744f7d28a9

https://news.mit.edu/2024/researchers-enhance-peripheral-vision-ai-models-0308

https://medium.com/@1709deepesh/connecting-apache-nifi-to-microsoft-graph-reading-emails-with-invokehttp-processors-6d84db9fa157

https://community.cloudera.com/t5/Community-Articles/How-to-call-a-CML-Deployed-Model-From-Apache-NiFi-in-10/ta-p/374853

https://www.infoq.com/news/2024/03/java-22-so-far/

https://www.infoq.com/news/2024/03/lapce-rust-editor

https://www.infoq.com/news/2024/03/mistral-ai-aws/

https://www.infoq.com/news/2024/03/anthropic-claude-ai/

https://www.quantamagazine.org/new-breakthrough-brings-matrix-multiplication-closer-to-ideal-20240307/

https://venturebeat.com/ai/hugging-face-is-launching-an-open-source-robotics-project-led-by-former-tesla-scientist/?

https://dbos-project.github.io/

https://www.datanami.com/2024/03/07/cloudera-unveils-next-phase-of-open-data-lakehouse-to-unlock-enterprise-ai/?

https://www.decodable.co/blog/taxonomy-of-data-change-events

https://www.europarl.europa.eu/news/en/press-room/20240308IPR19015/artificial-intelligence-act-meps-adopt-landmark-law

https://medium.com/plain-simple-software/the-llm-app-stack-2024-eac28b9dc1e7

https://www.slideshare.net/JulienSIMON5/an-introduction-to-computer-vision-with-hugging-face

https://huggingface.co/learn/nlp-course/chapter1/2?fw=pt

https://huggingface.co/timm

https://github.com/huggingface/pytorch-image-models

https://www.slideshare.net/JulienSIMON5/an-introduction-to-computer-vision-with-hugging-face

https://www.infoq.com/news/2024/03/azure-openai-your-data-ga/

https://developers.redhat.com/articles/2024/03/13/kafka-tiered-storage-deep-dive?

https://community.cloudera.com/t5/What-s-New-Cloudera/Cloudera-DataFlow-adds-Change-Data-Capture-processors-flow/ba-p/381727

Videos

Streaming Traffic Cameras https://www.youtube.com/watch?v=85ECRGJBEQU&ab_channel=DatainMotion-HowToBeaStreamingEngineer

Python Processor https://www.youtube.com/watch?v=jF5FSY0xFiQ&t=9s&ab_channel=DatainMotion-HowToBeaStreamingEngineer

Preview of TCF Pro Talk https://youtu.be/ce9lhtbp48M?si=Svjb2-bIIPXLwXD1

Feb 22, 2024 NYC Meetup

https://www.slideshare.net/slideshows/2024-feb-ai-meetup-nyc-genaillmsmldata-codeless-generative-ai-pipelines/266444687

Feb 28, 2024 NYC Flink Meetup

https://www.slideshare.net/slideshows/2024-february-28-nyc-meetup-unlocking-financial-data-with-realtime-pipelines/266539528

Feb 29, 2024 Conf42 Python 2024

https://www.slideshare.net/slideshows/conf42python-using-apache-nifi-apache-kafka-risingwave-and-apache-iceberg-with-stock-data-and-llm/266521940

https://www.slideshare.net/slideshows/conf42pythonbuilding-apache-nifi-20-python-processors/266522007

https://www.youtube.com/watch?v=awxzG7laWx4&ab_channel=Conf42

https://www.youtube.com/watch?v=FD16_oZ65Ug&ab_channel=Conf42

March 11, 2024 Princeton 23 Orchard Event

https://www.slideshare.net/slideshows/2024-build-generative-ai-for-nonprofits/266748822

march 15, 2024 Trenton TCF

https://www.slideshare.net/slideshows/tcfpro24-building-realtime-generative-ai-pipelines/266807785

Events

March 27, 2024: Startup Grind. Jersey City https://www.startupgrind.com/events/details/startup-grind-princeton-presents-startup-grind-princeton-amp-nj-big-data-alliance-generative-ai-reverse-pitch/

March 28, 2024: Pinot + NiFi + Flink + Kafka Meetup NYC https://www.meetup.com/real-time-analytics-meetup-ny/events/299290822/

April 2, 2024: XtremeJ 2024. Virtual. https://xtremej.dev/2023/schedule/

April 8-11, 2024: NLIT Summit. Seattle. https://www.fbcinc.com/e/nlit/default.aspx 

April 11, 2024: Conf42 LLM. Virtual. https://www.conf42.com/llms2024

April 12, 2024: AI Max Conference. 23 Orchard Princeton https://www.startupgrind.com/events/details/startup-grind-princeton-presents-startup-grind-hosts-ai-max-summit/

April 2024: AI Meetup NJ https://www.meetup.com/nj-gai/

May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx

Cloudera Events https://www.cloudera.com/about/events.html

More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

• https://github.com/tspannhw/FLaNK-python-processors

Models

• https://github.com/urchade/GLiNER
• https://github.com/deepseek-ai/DeepSeek-VL
• https://github.com/sieve-community/fast-asd
• https://arxiv.org/abs/2403.09611
• https://github.com/xai-org/grok-1

Datasets

• https://hf.co/datasets

Tools

• https://github.com/echasnovski/mini.nvim/blob/main/readmes/mini-indentscope.md
• https://datavolo.io/2024/03/data-engineering-for-advanced-rag-small-to-big-with-pinecone-langchain-and-datavolo/
• https://docs.pinecone.io/docs/metadata-filtering
• https://python.langchain.com/docs/modules/data_connection/retrievers/parent_document_retriever
• https://docs.pinecone.io/reference/list
• https://colab.research.google.com/drive/1AvPpRvzLvGPMG3vVgcRDOvSdar_lsTks
• https://github.com/pi-ra/beesy-issue-tracker
• https://palindromicity.blogspot.com/2020/06/dotifi-generating-dot-files-from-apache.html
• https://github.com/flxzt/rnote
• https://github.com/teableio/teable
• https://localsend.org/#/
• https://github.com/leobeeson/llm_benchmarks
• https://www.cognition-labs.com/blog
• https://github.com/openvinotoolkit/openvino_notebooks/tree/recipes/recipes/defect_detection_anomalib
• https://brave.com/
• https://github.com/PeerDB-io/peerdb
• https://osm2pgsql.org/
• https://arc.net/
• https://github.com/truera/trulens
• https://github.com/has2k1/plotnine
• https://github.com/altair-viz/altair
• https://github.com/quarto-dev/quarto-cli
• https://github.com/tobymao/sqlglot
• https://github.com/quarkiverse/quarkus-langchain4j
• https://github.com/ELLA-Diffusion/ELLA
• https://github.com/skills-cogrammar/C7-Lecture-Backpack
• https://github.com/LucasPickering/slumber
• https://webhook.site/
• https://github.com/paveldedik/ludic
• https://github.com/bananaml/fructose
• https://github.com/betwixt-labs/bebop
• https://github.com/jafioti/luminal
• https://github.com/soorajshankar/logScreen
• https://github.com/flydelabs/flyde
• https://lite.ip2location.com/ip2location-lite
• https://github.com/getindata/flink-http-connector
• https://github.com/phospho-app/phospho
• https://github.com/phospho-app/fastassert
• https://github.com/developersdigest/llm-answer-engine
• https://docs.litellm.ai/docs/proxy/quick_start
• https://letsbuild.ai/
• https://github.com/albertan017/LLM4Decompile
• https://vector.dev/
• https://github.com/ArroyoSystems/arroyo
• https://github.com/stanfordnlp/pyvene
• https://www.mewho.com/titan/
• https://radicle.xyz/
• https://github.com/ianand/spreadsheets-are-all-you-need

Tips

https://www.datainmotion.dev/2020/05/one-minute-nifi-tip-calcite-sql-notes.html

Cool Tool

These are amazing diagrams and graphics.
https://drawify.com/templates/341/personal-user-manual

© 2020-2024 Tim Spann