FLaNK-AIM: 13 May 2024

 

13-May-2024

https://www.youtube.com/@FLaNK-Stack

FLaNK / KNIFe AI / FLaNK-AIM Weekly

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

**This is Issue #137 **

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

Articles

https://medium.com/@tspann/boston-wheres-my-bus-llm-streaming-to-the-rescue-586dfd019237

https://medium.com/@tspann/small-language-models-sml-for-the-win-ea0c6fee8061

https://medium.com/@tspann/maybe-four-smaller-open-llm-s-are-better-than-one-93f78fb69eb9

https://medium.com/@tspann/building-a-milvus-connector-for-nifi-34372cb3c7fa

https://medium.com/@tspann/searching-slack-from-apache-nifi-9ed562aa2397

https://milvus.io/blog/milvus-supports-apache-parquet-file-supports.md

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.26.0

https://www.cloudera.com/about/news-and-blogs/press-releases/2024-05-07-cloudera-announces-data-in-motion-products-will-be-made-available-as-kubernetes-operators-for-red-hat-openshift.html

https://www.jetson-ai-lab.com/research.html#meeting-schedule

https://jack-vanlightly.com/blog/2024/5/7/learning-and-reviewing-system-internals-tactics-and-psychology

https://blog.min.io/the-future-of-ai-is-open-source/

https://dagshub.com/blog/common-pitfalls-to-avoid-when-using-vector-databases/

https://medium.com/@adam.bellemare/preventing-and-fixing-bad-data-in-event-streams-part-1-27bf2a99b48e

https://www.linkedin.com/pulse/milvus-community-newsletter-beginner-rag-guides-learn-ok3bc/

https://spring.io/blog/2024/05/09/spring-ai-structured-output

https://openai.com/index/instruction-following/

https://blog.miguelgrinberg.com/post/how-llms-work-explained-without-math

https://www.dell.com/en-uk/blog/the-rise-of-the-small-language-models-slms/

https://www.junaideffendi.com/p/netflix-data-tech-stack

Videos

Generative AI with Milvus https://www.youtube.com/watch?v=IfWIzKsoHnA

Four Models at Once https://youtu.be/xvNgsZyfo6A?si=zxwc9VcFc3o0vU3P

Slides

https://www.slideshare.net/slideshow/generative-ai-on-enterprise-cloud-with-nifi-and-milvus/267678399

Events

May 21, 2024: Gen AI and Beyond with NiFi 2.0. Virtual.

May 30, 2024: Conf42: Machine learning https://www.conf42.com/Machine_Learning_2024_Tim_Spann_enriching_generative_events

June 12, 2024: Budapest Data + ML Forum. Virtual.  https://budapestml.hu/2024/en/speakers/

June 20, 2024: AI Camp Meetup. NYC.

Sept 24, 2024: JConf.Dev. Dallas. https://2024.jconf.dev/session/598816

Nov 5-7, 10-12, 2024: CloudX. Online/Santa Clara. https://www.developerweek.com/cloudx/

Nov 19, 2024: XtremePython. Online. https://xtremepython.dev/2024/

Cloudera Events https://www.cloudera.com/about/events.html

https://www.cloudera.com/events/cloudera-now-cdp.html?internal_keyplay=ALL&internal_campaign=FY25-Q1-AMER-WS-Cloudera-Now-Events-Page-P06&cid=701Hr000000tW6qIAE&internal_link=p06

More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

• https://github.com/tspannhw/FLaNK-python-processors
• https://github.com/tspannhw/FLANKAI-Boston
• https://github.com/tspannhw/FLaNK-IrelandTransit

Models

• https://github.com/ollama/ollama/releases/tag/v0.1.33-rc5
• https://github.com/NVlabs/RADIO

Tools

• https://github.com/judahpaul16/gpt-home
• https://github.com/teachingai/spring-ai-examples
• https://opea.dev/
• https://github.com/FalkorDB/FalkorDB
• https://github.com/NielsRogge/Transformers-Tutorials/blob/master/Idefics2/Fine_tune_Idefics2_for_JSON_extraction_use_cases_(PyTorch_Lightning).ipynb
• https://github.com/strapi/strapi
• https://github.com/traceloop/openllmetry
• https://research.ibm.com/blog/granite-code-models-open-source
• https://github.com/instructlab
• https://hurl.dev/docs/installation.html
• https://devenv.sh/
• https://gulpjs.com/
• https://github.com/context-labs/mactop
• https://dokploy.com/
• https://github.com/NimbleMarkets/ntcharts
• https://github.com/abi/secret-llama
• https://github.com/wandb/openui
• https://huggingface.co/spaces/chencws/stable-text-to-motion-framework/tree/main
• https://github.com/sato-team/Stable-Text-to-Motion-Framework
• https://github.com/IST-DASLab/PanzaMail
• https://github.com/huggingface/lerobot
• https://github.com/run-llama/llama_parse
• https://github.com/Efficient-Large-Model/VILA
• https://github.com/google-research/timesfm
• https://github.com/hugeicons/hugeicons-react
• https://github.com/galliumdata/adumbra
• https://bytewax.io/blog/stream-processing-using-python-on-a-raspberry-pi
• https://github.com/AdityaNG/kan-gpt
• https://github.com/meetbryce/open-source-slack-ai
• https://github.com/google-research/timesfm
• https://pyspread.gitlab.io/
• https://github.com/Upsonic/Tiger
• https://github.com/MHNightCat/superfile
• https://github.com/IceWhaleTech/CasaOS
• https://github.com/travisdoor/tine
• https://github.com/NickCulbertson/Teensy-Arcade-Synth

Cool Tool

• https://github.com/cloudera/CML_AMP_Deploy-Mistral7B-CML-Native-Model

New

https://ai.meta.com/blog/raft-llama-retrieval-augmented-generation-supervised-fine-tuning-microsoft/

© 2020-2024 Tim Spann https://www.youtube.com/@FLaNK-Stack FLaNK-AIM with LLAMA 3

Gen AI on Enterprise Cloud

FLaNK-AIM Weekly for 06 May 2024

 

06-May-2024

https://www.youtube.com/@FLaNK-Stack

FLaNK / KNIFe AI / FLaNK-AIM Weekly

http://knifeai.org/

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

**This is Issue #136 **

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

New Releases

Articles

https://medium.com/@tspann/small-language-models-sml-for-the-win-ea0c6fee8061

https://medium.com/@tspann/maybe-four-smaller-open-llm-s-are-better-than-one-93f78fb69eb9

https://medium.com/@tspann/building-a-milvus-connector-for-nifi-34372cb3c7fa

https://medium.com/@tspann/searching-slack-from-apache-nifi-9ed562aa2397

https://medium.com/@tspann/events-streams-flows-and-maps-22a8d27cd9b4

https://medium.com/@tspann/storing-meetup-user-data-as-events-dad3b1dc89f5

https://medium.com/@tspann/real-time-in-boston-part-1-0f92d7da3496

NSA AI Security https://www.nsa.gov/Press-Room/Press-Releases-Statements/Press-Release-View/Article/3741371/nsa-publishes-guidance-for-strengthening-ai-system-security/

https://zilliz.com/learn/Sentence-Transformers-for-Long-Form-Text

https://zilliz.com/zilliz-cloud-pipelines

https://huggingface.co/BAAI/bge-large-en-v1.5

https://github.com/cloudevents/sdk-python/blob/main/samples/http-json-cloudevents/client.py

https://medium.com/@tspann/building-a-milvus-connector-for-nifi-34372cb3c7fa

https://docs.cloudera.com/machine-learning/cloud/release-notes/topics/ml-whats-new.html#ml_workspace_resource_tags

https://zilliz.com/blog/finding-right-fit-embedding-support-for-RAG-in-zilliz-cloud-pipelines-from-voyageai-openai-and-oss

https://hazelcast.com/glossary/streaming-data/

https://postgres.ai/blog/20220525-common-db-schema-change-mistakes

https://medium.com/cloudera-inc/consuming-rss-feeds-from-flink-sql-eaf33c1a5a23

https://medium.com/cloudera-inc/adding-generative-ai-results-to-sql-streams-513e1fd2a6af

https://www.linuxfoundation.org/press/lf-ai-data-foundation-launches-open-platform-for-enterprise-ai-opea

https://blog.mozilla.ai/local-llm-as-judge-evaluation-with-lm-buddy-prometheus-and-llamafile/

https://blog.mozilla.ai/open-source-in-the-age-of-llms/

https://www.pinecone.io/learn/structured-data/

https://medium.com/@stoty/a-bug-for-ages-fixing-time-zone-handling-in-apache-phoenix-e9934d7acd80

https://www.geeknarrator.com/blog/stream-processing/stream-processing-concepts

https://blog.cloudera.com/setting-up-and-getting-started-with-clouderas-new-sql-ai-assistant/

https://thenewstack.io/how-to-cure-llm-weaknesses-with-vector-databases/

https://dev.to/zilliz/exploring-bge-m3-and-splade-two-machine-learning-models-for-generating-sparse-embeddings-22p1

https://zilliz.com/learn/transforming-pdfs-into-insights-vectorizing-and-ingesting-with-zilliz-cloud-pipelines

https://zilliz.com/blog/how-to-evaluate-and-optimize-performance-of-milvus-storage

https://datavolo.io/2024/05/apache-nifi-designed-for-extension-at-scale/

Videos

Generative AI with Milvus https://www.youtube.com/watch?v=IfWIzKsoHnA

Four Models at Once https://youtu.be/xvNgsZyfo6A?si=zxwc9VcFc3o0vU3P

Search Slack https://www.youtube.com/watch?v=3ugppfb2kN8&t=5s&ab_channel=DatainMotion-HowToBeaStreamingEngineer

MBTA Transit Live with LLM https://www.youtube.com/watch?v=JGGY_uzQTdY&t=3s&pp=ygUOVGltIFNwYW5uIE5pRmk%3D

Events, Streams, Maps with Irish Rail https://www.youtube.com/watch?v=14CSQRfUWoE&t=684s&pp=ygUOVGltIFNwYW5uIE5pRmk%3D

FLaNK AI Channel https://www.youtube.com/@FLaNK-Stack

NiFi https://www.youtube.com/watch?v=m-ZoqHOYy_k

Slides

https://www.slideshare.net/slideshow/generative-ai-on-enterprise-cloud-with-nifi-and-milvus/267678399

https://www.slideshare.net/slideshow/conf42llmadding-generative-ai-to-realtime-streaming-pipelines/267269788

https://github.com/tspannhw/FLaNK-Milvus

https://medium.com/cloudera-inc/building-a-milvus-connector-for-nifi-34372cb3c7fa

https://www.slideshare.net/slideshow/generative-ai-on-enterprise-cloud-with-nifi-and-milvus/267678399

https://www.youtube.com/watch?v=ssoM5S87BBs

Events

May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx https://www.dbta.com/DataSummit/2024/Timothy-Spann.aspx

https://twitter.com/DBTADataSummit/status/1778393005646397636

May 21, 2024: Gen AI and Beyond with NiFi 2.0. Virtual.

May 30, 2024: Conf42: Machine learning https://www.conf42.com/Machine_Learning_2024_Tim_Spann_enriching_generative_events

June 12, 2024: Budapest Data + ML Forum. Virtual.  https://budapestml.hu/2024/en/speakers/

June 20, 2024: AI Camp Meetup. NYC.

Sept 24, 2024: JConf.Dev. Dallas. https://2024.jconf.dev/session/598816

Nov 5-7, 10-12, 2024: CloudX. Online/Santa Clara. https://www.developerweek.com/cloudx/

Nov 19, 2024: XtremePython. Online. https://xtremepython.dev/2024/

Cloudera Events https://www.cloudera.com/about/events.html

https://www.cloudera.com/events/cloudera-now-cdp.html?internal_keyplay=ALL&internal_campaign=FY25-Q1-AMER-WS-Cloudera-Now-Events-Page-P06&cid=701Hr000000tW6qIAE&internal_link=p06

More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

• https://github.com/tspannhw/FLaNK-python-processors
• https://github.com/tspannhw/FLANKAI-Boston
• https://github.com/tspannhw/FLaNK-IrelandTransit

Models

• https://github.com/ollama/ollama/releases/tag/v0.1.33-rc5
• https://github.com/vikhyat/moondream
• https://github.com/kermitt2/grobid
• https://huggingface.co/apple/OpenELM-3B-Instruct
• https://huggingface.co/namespace-Pt/Llama-3-8B-Instruct-80K-QLoRA

Tools

• https://krasjet.com/voice/pdf.tocgen/
• https://github.com/rcoh/angle-grinder
• https://www.brandur.org/logfmt
• https://iss.matteason.co.uk/
• https://echarts.apache.org/handbook/en/concepts/dataset
• https://www.chartjs.org/docs/latest/getting-started/usage.html
• https://js.cytoscape.org/
• https://github.com/apexcharts/apexcharts.js/
• https://github.com/andrewcourtice/ripl
• https://unovis.dev/gallery
• https://vega.github.io/vega-lite/
• https://observablehq.com/@observablehq/plot-gallery
• https://www.tremor.so/
• https://github.com/gyf304/dotenv
• https://wanix.sh/
• https://os-world.github.io/
• https://ollama.com/library/moondream
• https://forums.developer.nvidia.com/t/introducing-ollama-support-for-jetson-devices/289333
• https://www.youtube.com/@FLaNK-Stack
• https://github.com/kaytu-io/kaytu
• https://github.com/hashhar/jsonschema2sql
• https://threlte.xyz/
• https://github.com/kingjulio8238/memary
• https://zilliz.com/learn/how-to-enhance-the-performance-of-your-rag-pipeline
• https://medium.com/cloudera-inc/consuming-rss-feeds-from-flink-sql-eaf33c1a5a23
• https://huggingface.co/bigcode/starcoder2-15b-instruct-v0.1
• https://docs.pinecone.io/examples/notebooks
• https://github.com/JetBrains/xodus
• https://github.com/towhee-io/examples/tree/main/nlp/text_search
• https://github.com/openvinotoolkit/openvino_notebooks/tree/recipes/recipes
• https://github.com/zilliztech/kafka-connect-milvus
• https://github.com/timeplus-io/proton/tree/develop/examples/real-time-ai
• https://zilliz.com/what-is-gptcache
• https://github.com/zilliztech/VectorDBBench
• https://github.com/zilliztech/milvus_cli
• https://github.com/towhee-io/examples/blob/main/nlp/text_search/search_article_in_medium.ipynb
• https://seeed-studio.github.io/SenseCraft-Web-Toolkit/#/setup/process
• https://github.com/McGill-NLP/webllama
• https://github.com/Mozilla-Ocho/llamafile
• https://prometheus-eval.github.io/prometheus
• https://github.com/mozilla-ai/lm-buddy
• https://github.com/databonsai/databonsai
• https://github.com/cohere-ai/cohere-toolkit
• https://github.com/InternLM/lmdeploy
• https://github.com/TagStudioDev/TagStudio
• https://github.com/Axorax/tkforge
• https://github.com/hiyouga/LLaMA-Factory
• https://pub.towardsai.net/rag-2-0-finally-getting-rag-right-f74d0194a720
• https://github.com/brettdidonato/trendr-bot
• https://github.com/cloudera/CML_AMP_Intelligent-QA-Chatbot-with-NiFi-Pinecone-and-Llama2
• https://github.com/pydantic/logfire
• https://www.terminal.shop/
• https://github.com/penpot
• https://github.com/ytang07/ai_agents_cookbooks/tree/main/langchain
• https://voxel51.com/fiftyone/
• https://huggingface.co/apple/OpenELM-450M-Instruct?library=true
• https://zilliz.com/learn/effortless-ai-workflows-a-beginners-guide-to-hugging-face-and-pymilvus
• https://github.com/NVlabs/DoRA
• https://github.com/Blealtan/efficient-kan
• https://github.com/habuma/spring-ai-examples/tree/main/vector-store-loader
• https://github.com/eureka-research/DrEureka
• https://haystack.deepset.ai/integrations/milvus-document-store
• https://github.com/TencentARC/InstantMesh
• https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B
• https://kai-waehner.medium.com/genai-demo-with-kafka-flink-langchain-and-openai-8c9b7263770a
• https://medium.com/@amanatulla1606/llm-web-scraping-with-scrapegraphai-a-breakthrough-in-data-extraction-d6596b282b4d
• https://news.mit.edu/2024/mit-faculty-instructors-students-experiment-generative-ai-teaching-learning-0429
• https://zilliz.com/blog/spring-ai-and-milvus-using-milvus-as-spring-ai-vector-store

Cool Tool

Convert Spark SQL to Trino SQL https://github.com/linkedin/coral

Discount

Discount access to DataSummit 2024 https://secure.infotoday.com/RegForms/DataSummit/?Priority=24SPKR

© 2020-2024 Tim Spann https://www.youtube.com/@FLaNK-Stack FLaNK-AIM with LLAMA 3