Streaming Cameras with YOLOv8

 Apache NiFi, Python, YoLoV8, MinIO, S3, Images, Cameras, New York City

We can add a very easy to run Ultralytics YOLO v8 to hit against ingested camera’s from New York City. As you can see the code is really simple, we just need to load the pretrained model and call predict with some parameters and the image.

from ultralytics import YOLO
import sys
import io

import shutil
shutil.rmtree('runs/detect')

# Load a model
model = YOLO('yolov8n.pt') # pretrained YOLOv8n model

source = sys.argv[1]

results = model.predict(source, stream=False, save=True, imgsz=320, conf=0.5)

for r in results:
print(r.tojson())

Output

[
{
"name": "car",
"class": 2,
"confidence": 0.5163618922233582,
"box": {
"x1": 188.54917907714844,
"y1": 141.74185180664062,
"x2": 204.51304626464844,
"y2": 154.35519409179688
}
}
]
YOLOv9 added annotation

NiFi Flow

NiFi Detailed Steps

We invoke an HTTP URL from NY Open Data to get a list of all URLs. We send the metadata to a Kafka topic. We call the webcam URL to get the image. We save it to MinIO. We then save it local in a temporary file to get it analyzed by YOLOv8. We then retrieve the augmented image and send it to Slack.

Execute Shell Script Passing Argument to Python 3

META DATA

{
"Latitude" : 41.51472,
"Longitude" : -74.0733,
"ID" : "Skyline-9873",
"Name" : "I-87 MP 060.40 NB Just North of Interchange 17 (Newburgh/I-84)",
"DirectionOfTravel" : "Northbound",
"RoadwayName" : "I-87 - NYS Thruway",
"Url" : "https://511ny.org/map/Cctv/9873--43",
"VideoUrl" : "https://s58.nysdot.skyvdn.com:443/rtplive/TA_046/playlist.m3u8",
"Disabled" : false,
"Blocked" : false
}

RESOURCES

FLaNK Stack 26 February 2024

FLaNK Stack 26 February 2024 

26-February-2024

image

FLaNK Stack Weekly

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

image

**This is Issue #126 **

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

Articles

Using Google Gemma https://medium.com/@tspann/google-gemma-for-real-time-lightweight-open-llm-inference-88efe98e580f

NYC Traffic?? (NiFi, Kafka, Flink) https://medium.com/@tspann/nyc-traffic-are-you-kidding-me-6d3fa853903b

Subways and Transit Updates in Real-Time https://medium.com/@tspann/subways-and-transit-updates-in-real-time-30c104c359ef

Open Source Data Infrastructure Meetup - Feb 2024 https://medium.com/@tspann/open-source-data-infrastructure-meetup-feb-2024-9e8048666828

https://sap1ens.com/blog/2024/02/18/customizing-flink-class-shadowing/

https://www.projectpro.io/recipes/use-nifi-extract-and-parse-data-from-http-endpoints-and-store-data-persistent-storage

https://engineering.grab.com/attribution-platform

https://amistrongeryet.substack.com/p/why-are-llms-so-gullible

https://huggingface.co/blog/gemma

https://developer.nvidia.com/blog/build-an-llm-powered-data-agent-for-data-analysis/

https://thenewstack.io/the-rise-of-small-language-models/

https://www.infoq.com/news/2024/02/pinterest-pubsub-kafka-flink/

https://www.infoq.com/news/2024/01/doordash-service-mesh/

https://thenewstack.io/demo-use-webassembly-to-run-llms-on-your-own-device-with-wasmedge

https://www.eleuther.ai/releases

https://www.microsoft.com/en-us/research/blog/orca-2-teaching-small-language-models-how-to-reason/

https://www.baeldung.com/ops/docker-remove-dangling-unused-images

AI + More required for startup https://www.nfx.com/post/ai-like-water

https://explainextended.com/2023/12/31/happy-new-year-15/

https://medium.com/sids-tech-cafe/event-driven-systems-lessons-from-the-trenches-107c07b3fc1d

https://materializedview.io/p/from-samza-to-flink-a-decade-of-stream

https://exaspark.medium.com/the-ultimate-guide-to-postgresql-data-change-tracking-c3fa88779572

https://www.wired.com/story/17-tips-better-chatgpt-prompts

https://github.com/microsoft/generative-ai-for-beginners/

Videos

Continuous SQL with Kafka and Flink https://www.youtube.com/watch?v=0Fb8ggZlPrQ&ab_channel=stevecantrell

Building Real-time Pipelines: A Case Study by Transit Data https://www.youtube.com/watch?v=VjmC4J7KZgw&t=2s&ab_channel=Aiven

Unlocking Financial Data with Real-Time Pipelines (OSACon 2023) https://www.youtube.com/watch?v=Q7gF7m4yFi4&ab_channel=OSACon

The Never Landing Stream https://www.youtube.com/watch?v=M8Bp0tRGvV0

https://www.youtube.com/watch?v=gSvvBHBWq20

https://www.youtube.com/watch?v=ayAGiPd2zq4&t=1s

February 8, 2024 NYC Meetup

https://www.slideshare.net/slideshows/ny-open-source-data-meetup-feb-8-2024-building-realtime-pipelines-with-flank-a-case-study-with-transit-data/266227433

February 20, 2024 Virtual Meetup

https://www.slideshare.net/slideshows/dba-fundamentals-group-continuous-sql-with-kafka-and-flink/266403113 https://www.youtube.com/watch?v=0Fb8ggZlPrQ&ab_channel=stevecantrell

Feb 22, 2024 NYC Meetup

https://www.slideshare.net/slideshows/2024-feb-ai-meetup-nyc-genaillmsmldata-codeless-generative-ai-pipelines/266444687

Events

Feb 28, 2024: NYC. Cloudera Meetup. Flink https://www.meetup.com/futureofdata-princeton/events/298661947/

Feb 29, 2024: Virtual. Conf42 Python. https://www.conf42.com/Python_2024_Tim_Spann_apache_nifi_2_processors

https://www.conf42.com/Python_2024_Karin_Wolok_nifi__kafka_risingwave_iceberg_llm

Soon, 2024: Princeton. TigerLabs New Location. Meetup. GenAI. https://www.meetup.com/applied-generative-artificial-intelligence-applications/

March 15, 2024: TCF Pro. Princeton, NJ. IT Professional Conference at Trenton Computer Festival IEEE Information Technology Professional Conference on Friday, March 15th, 2024 https://princetonacm.acm.org/tcfpro/

March 28, 2024: Pinot + NiFi + Flink + Kafka Meetup NYC https://www.meetup.com/real-time-analytics-meetup-ny/events/299290822/

April 2024: XtremeJ 2024. Virtual. https://xtremej.dev/2023/schedule/

April 11, 2024: Conf42 LLM. Virtual. https://www.conf42.com/llms2024

May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx

Cloudera Events https://www.cloudera.com/about/events.html

More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

Models

Tools

Notable Tools

Postgresql + MySQL Cache https://github.com/readysettech/readyset

NVIDIA GPU LLM https://github.com/NVIDIA/TensorRT-LLM

Configuration Management Server https://caddyserver.com/features

Fast Text to Image https://fastsdxl.ai/

Very Interesting Remote tool for OBS https://vdo.ninja/

Commands Du Jour

docker system prune -a docker image prune -a docker system df docker ps docker logs name

© 2020-2024 Tim Spann