Data In Motion

https://developers.redhat.com/articles/2024/03/13/kafka-tiered-storage-deep-dive
https://huggingface.co/adept/fuyu-8b
https://llava.hliu.cc/
https://github.com/langchain-ai/langgraph/blob/main/examples/rag/langgraph_self_rag.ipynb
https://github.com/mapr-demos/gess
https://github.com/TracecatHQ/tracecat
https://github.com/datadreamer-dev/DataDreamer
https://huggingface.co/databricks/dbrx-instruct
https://github.com/openai/tiktoken
https://github.com/ora0600/genai-with-confluent
https://github.com/speedb-io/speedb
https://github.com/crate/crate
https://gpt4all.io/index.html
https://pypi.org/project/imposm.parser/
https://github.com/easyjailbreak/easyjailbreak?
https://github.com/databricks/megablocks
https://github.com/milvus-io/bootcamp/blob/master/bootcamp/RAG/zilliz_pipeline_rag.ipynb
https://github.com/DoMusic/Hybrid-Net
https://www.reddit.com/r/LocalLLaMA/comments/1bmvtyb/new_user_beginning_guide_from_total_noob_to/
https://github.com/run-llama/llama_index/tree/main/llama-index-packs/llama-index-packs-raft-dataset
https://github.com/FoundationVision/GLEE
https://github.com/zenml-io/zenml
https://github.com/lapce/lapdev
https://github.com/enisdenjo/graphql-ws
https://github.com/jasonppy/VoiceCraft
https://github.com/h2oai/enterprise-h2ogpte
https://github.com/sbcshop/Pitalk_4G_HAT_Software
https://github.com/OpenAccess-AI-Collective/axolotl/blob/main/docs/rlhf.qmd
https://www.modular.com/blog/the-next-big-step-in-mojo-open-source

New

Vector Db built on clickhouse https://github.com/myscale/myscaledb

Cool Tool - LLM Synthetic Data Generators

https://github.com/geraldyong/OpenAI_Synthetic/tree/main

https://github.com/quentinlintz/synthetic-data-generator

https://medium.com/@n-demia/how-to-prepare-test-data-via-openai-api-in-postman-7e378dde1f53

https://github.com/datadreamer-dev/DataDreamer

https://huggingface.co/collections/rbiswasfc/synthetic-data-generation-65ee68e821ddaff47073ed02

Flink Connectors (scroll down)

https://flink.apache.org/downloads/

Avro

Can't handle numbers bigger than 19 decimals

Throwback Article

https://community.cloudera.com/t5/Community-Articles/Ingesting-RDBMS-Data-As-New-Tables-Arrive-Automagically-into/ta-p/246214

https://docs.cloudera.com/csp-ce/latest/ce-overview/topics/csp-ce-overview.html

Discount

Discount access to DataSummit 2024 https://secure.infotoday.com/RegForms/DataSummit/?Priority=24SPKR

on April 01, 2024

Email This BlogThis!Share to Twitter Share to Facebook Share to Pinterest

Labels: Apache Flink, apache kafka, apache nifi, cloudera, genai, iot, llm, python, streaming, tspann

FLaNK Weekly for 12 February 2024

12-February-2024

FLaNK Stack Weekly

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

Get your new Apache NiFi for Dummies!

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

Trial: https://console.us-west-1.cdp.cloudera.com/trial/register.html#/

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

**This is Issue #124 **

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

Qualified Developers (Looking for work)

https://www.linkedin.com/in/satya-n99999/

Courses

https://cs50.harvard.edu/python/2022/

https://developers.google.com/machine-learning/crash-course

https://cognitiveclass.ai/courses/docker-essentials

https://www.scaler.com/topics/course/java-beginners/

https://online.stanford.edu/courses/soe-ydatabases0005-databases-relational-databases-and-sql

Articles

Open Source Data Infrastructure Meetup - Feb 2024 https://medium.com/@tspann/open-source-data-infrastructure-meetup-feb-2024-9e8048666828

Apache NiFi with Amazon Translate for ML Pipelines https://medium.com/@tspann/apache-nifi-and-amazon-translate-for-machine-learning-pipelines-dcfe4e61fc02

NiFi 2.0.0-M2 is Out! https://medium.com/@tspann/apache-nifi-2-0-0-m2-out-314a1d4c8b20

Apache NiFi Python extensions https://apex974.com/articles/nifi-2-python-extensions

Apache NiFi and Amazon Textract for Machine Learning https://medium.com/@tspann/apache-nifi-and-amazon-textract-for-machine-learning-e45f4af12e68

Using Apache NiFi API to Start and Stop NiFi Properties https://www.clearpeaks.com/using-the-nifi-api-to-start-and-stop-nifi-processors-from-a-nifi-flow/

Langchain from 0 to 1 https://fosdem.org/2024/events/attachments/fosdem-2024-2384-langchain-from-0-to-1-unveiling-the-power-of-llm-programming/slides/21698/LangChain_From_0_To_1_public_1_PpuSgEN.pdf

Local First AI with Postgres PgVector https://electric-sql.com/blog/2024/02/05/local-first-ai-with-tauri-postgres-pgvector-llama

RAG Details https://www.infoq.com/podcasts/retrieval-augmented-generation

ChromaDB in Java (LangChain4J) https://medium.com/@timju/chromadb-in-java-langchain4j-41ed910cd3e7

SQL Tutorial https://gvwilson.github.io/sql-tutorial

Gen AI to Edge https://developer.nvidia.com/blog/bringing-generative-ai-to-the-edge-with-nvidia-metropolis-microservices-for-jetson/

I want to know what small device can run Java for IoT and fit in a toothbrush. I want to develop stuff for that platform. https://www.zdnet.com/home-and-office/smart-home/3-million-smart-toothbrushes-were-just-used-in-a-ddos-attack-really/

https://www.unlogged.io/post/springboot-vs-quarkus-vs-micronaut

https://ionutbalosin.com/2024/02/jvm-performance-comparison-for-jdk-21/

https://thewritetoroam.com/2024/02/how-to-write-stuff-no-one-else-can

https://www.morling.dev/blog/filtering-process-output-with-tee/

Videos

Seven Videos on Real-Time Streaming https://medium.com/@tspann/seven-videos-on-real-time-streaming-02711320afa8

Unlocking Financial Data with Real-Time Pipelines (OSACon 2023) https://www.youtube.com/watch?v=Q7gF7m4yFi4&ab_channel=OSACon

Tips

https://stackoverflow.com/questions/65461182/apache-nifi-unable-to-start-the-flow-controller-because-the-tls-configuration-wa

February 8, 2024 Meetup

https://www.slideshare.net/slideshows/ny-open-source-data-meetup-feb-8-2024-building-realtime-pipelines-with-flank-a-case-study-with-transit-data/266227433

Events

Feb 2024: Webinar https://www.cloudera.com/about/events/webinars/stay-ahead-of-cyber-threats-by-utilizing-data-in-motion.html?utm_medium=virtual-event&utm_source=resources-module&keyplay=ALL&utm_campaign=FY25-Q1-CorporateWebinar-AMER-cyber-threats&cid=701Hr000001pXCQIA2

Feb 20, 2024: 12-1PM EST. Virtual. Azure Data Tech Groups: DBA Fundamentals Group https://www.meetup.com/dba-fundamentals-group/events/296855261/

Feb 28, 2024: NYC. Cloudera Meetup. Flink https://www.meetup.com/futureofdata-princeton/events/298661947/

Feb 29, 2024: Virtual. Conf42 Python. https://www.conf42.com/Python_2024_Tim_Spann_apache_nifi_2_processors

https://www.conf42.com/Python_2024_Karin_Wolok_nifi__kafka_risingwave_iceberg_llm

March 5, 2024: Princeton. Meetup. GenAI. https://www.meetup.com/applied-generative-artificial-intelligence-applications/

March 15, 2024: TCF Pro. Princeton, NJ. IT Professional Conference at Trenton Computer Festival IEEE Information Technology Professional Conference on Friday, March 15th, 2024 https://princetonacm.acm.org/tcfpro/

April 2024: XtremeJ 2024. Virtual. https://xtremej.dev/2023/schedule/

May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx

Cloudera Events https://www.cloudera.com/about/events.html

More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

https://github.com/tspannhw/FLaNK-python-watsonx-processor
https://github.com/kevinbtalbert/CML_AMP_NeMo-Guardrails-Chatbot

Models

https://wiki.rwkv.com/basic/play.html#rwkv-cpp-rwkv-cpp-cuda-project

Data

https://github.com/trending/python
https://github.com/trending/java?since=daily
https://github.com/josephmisiti/awesome-machine-learning
https://oceanservice.noaa.gov/education/tutorial_currents/04currents3.html
https://water.weather.gov/ahps2/hydrograph.php?gage=BKWN4&wfo=phi

Tools

https://github.com/rsrohan99/llamabot
https://github.com/google-deepmind/graphcast
https://open-meteo.com/en/docs/ensemble-api
https://github.com/Stell0/fosdem2024/
https://github.com/cfahlgren1/natural-sql
https://github.com/electric-sql/electric/tree/tauri-example-postgres/examples/tauri-postgres
https://github.com/QwenLM/Qwen1.5
https://github.com/nomic-ai/contrastors
https://checkip.amazonaws.com/
https://github.com/dvcoolarun/web2pdf
https://github.com/narfindustries/http-garden
https://github.com/kevingduck/ChatGPT-phone
https://github.com/SuperDuperDB/superduperdb
https://github.com/allegroai/clearml
https://github.com/mosaicml/streaming
https://github.com/Sanster/IOPaint
https://github.com/ververica/flink-cdc-connectors
https://github.com/grpc/grpc-java
https://github.com/in28minutes/master-spring-and-spring-boot
https://github.com/dapr/quickstarts/tree/master/tutorials/bindings
https://github.com/TheoKanning/openai-java
https://github.com/second-opinion-ai/second-opinion
https://rustpython.github.io/
https://github.com/kevinbtalbert/Healthcare-Demo
https://www.promptfoo.dev/docs/guides/mistral-vs-llama/
https://github.com/allenai/olmo
https://aitestkitchen.withgoogle.com/tools/image-fx
https://tabulator.info/docs/5.5/quickstart
https://app.getonboardai.com/chat/ike71usxhomcsoaalu3mi?repo=github%3A%3Alangchain-ai%2Flangchain
https://github.com/osrd-project/osrd
https://unstructured-io.github.io/unstructured/installation/full_installation.html
https://faiss.ai/
https://github.com/langchain4j/langchain4j-examples
https://thenewstack.io/linux-hide-your-shell-passwords-with-sshpass
https://github.com/CorentinJ/Real-Time-Voice-Cloning
https://github.com/neonbjb/tortoise-tts
https://github.com/coqui-ai/TTS
https://github.com/netease-youdao/EmotiVoice
https://github.com/lobehub/lobe-chat
https://github.com/FlowiseAI/Flowise
https://github.com/f/awesome-chatgpt-prompts
https://github.com/labring/FastGPT
https://github.com/eosphoros-ai/Awesome-Text2SQL
https://github.com/vanna-ai/vanna
https://github.com/sqlchat/sqlchat
https://github.com/chat2db/chat2db
https://github.com/Plachtaa/VALL-E-X
https://zellij.dev/screencasts/
https://github.com/zilliztech/attu
https://www.textualize.io/
https://getdeploying.com/reference/data-egress
https://github.com/ckampfe/jindex
https://github.com/openvinotoolkit/openvino_notebooks
ggerganov/llama.cpp#4167
https://lilianweng.github.io/posts/2023-10-25-adv-attack-llm/
https://github.com/InterLinked1/lbbs
https://www.mysticbbs.com/downloads.html
https://wiki.synchro.net/howto:raspbian_install
https://wiki.seeedstudio.com/Local_Voice_Chatbot/
https://learnopencv.com/yolo-loss-function-gfl-vfl-loss/
https://huggingface.co/blog/tgi-messages-api
https://github.com/7mind/sick
https://github.com/adamritter/fastgron
https://github.com/Textualize/toolong