r/computervision • • 12h ago

Commercial Keeping grounding and extraction scores separate on plate reads

0 Upvotes

We used Unsiloed AI to extract plate numbers from OpenALPR benchmark photos, returning the read and its image coordinates together. This GIF shows twelve selected examples and how a review UI can bring a lower-scoring crop forward.

The enlarged CJ8P4V read has a grounding score of 0.46 and an extraction score of 0.74. The bars in the graphic use grounding scores. The two scores answer different questions, whether the value is supported at that image location and confidence in the extracted value itself.

All twelve displayed plate strings match the collection's reference labels, including the low-scoring one. These are selected examples, rather than an accuracy or calibration benchmark. A low score can route a correct read for review too.

For a production review queue, we'd evaluate the threshold against whole-plate errors on a separate labeled set, then measure both missed errors and review volume. The source coordinates let the reviewer inspect the plate rather than search the full image.

Source photos and labels · API score definitions


r/computervision • • 6h ago

Discussion I was tech chair for ICCV 2021. Here's my Defense of Academic Gatekeeping.

14 Upvotes

I was tech chair for ICCV 2021, and had "root access" to the whole process, so I saw (and scrubbed) its barnacly underbelly.

The ICCV organization tried to combat fraud, plagiarism, and cheating, but it was not a fair fight. There were at that point no effective mechanisms for detecting and punishing authors who tried to gain an unfair advantage.

Academic ML needs better policies. And we need to get this right because society needs better ways to build consensus and seek truth.

I wrote this essay to share some meta-science analysis I did over the last few weeks on academic ML and a few policy ideas.

My hope is to engage in discussion about academic ML policy. I believe that the only way we make progress on this is to sample and refine ideas together. So I ask: what are the policies you'd like to see adopted by academic journals and conferences?

Full essay: https://lacuna.tiptreesystems.com/paper/a-defense-of-academic-gatekeeping/art_221912b72e224a4393c508d4dd3ce5ed

(I'm now a postdoc-entrepreneur at Mila with Hugo Larochelle and Chris Pal, working on meta-science, multi-agent games, and AI for science.)


r/computervision • • 17h ago

Discussion Courses and resources

2 Upvotes

What’s the best resource to understand and really understand computer vision from scratch?
We talk logic, basics, solid fundamentals and everything, with hands on projects.
Or is there a plan to start learning to achieve the goal of being a CV expert?


r/computervision • • 19h ago

Help: Project Open-Source Gaze Assisted Communication—put your tokens to good use!

Post image
22 Upvotes

project Iris is an experimental browser-based tool exploring how gaze-assisted communication can work using everyday consumer technology. The goal is simple but important: Can we make gaze-assisted communication accessible without specialized hardware?

High-end eye-tracking systems improve accuracy with dual-camera setups, higher frame rates, and infrared illumination. But the cost and logistical complexity of getting those devices into the hands of people who need them can create significant barriers to access.

While this project has evolved over the past 18-months, I recently dug up the original Python file that first made me believe in project iris. With only ~500 lines of code, it's pretty cool to look back and see just how far things have come.

I'm sharing an updated version of that file here in hopes that someone (or something**) might find it interesting and want to contribute or expand upon the project!**

A couple of ways to get involved:

This is an open-ended project, and there's plenty left to figure out.


r/computervision • • 4h ago

Showcase 2-axis gimbal. Solo engineered. Master's thesis.

121 Upvotes

I'm building a 2-axis gimbal with a visible and a thermal camera for a small fixed-wing drone, as my master's thesis. The clip is the CAD model: on the left the gimbal as it looks, on the right the same unit with the housing made see-through, doing the same moves.

I'm posting here because the mechanics are mostly settled and the vision side is where I'd like input before I freeze the hardware.

What the cameras get from the gimbal:

  • Two cameras in a 67 mm ball, one above the other: a 5 MP visible camera and a 256×192 thermal camera, both arriving over USB
  • Two axes, roll and tilt: ±100° in roll, and from 25° up to 95° down in tilt
  • A magnetic encoder on each axis and two IMUs, one in the ball and one in the base, so the pointing angles are known at any time
  • Around 187 g for the whole nose

Two things I already know will matter for the image processing:

  • There is no pan axis. To look sideways the ball rolls and then tilts, so the picture rotates with the roll angle and has to be turned back in software.
  • The two lenses don't share an optical axis, and the thermal image is far lower in resolution, so lining up thermal and visible pixels will need calibration.

Questions for anyone who has worked with visible and thermal pairs, or with cameras on gimbals:

  • Is there something you wish the hardware had given you? Frame timestamps tied to the IMU, a sync line between the two cameras etc...
  • For registering thermal to visible at this resolution is a fixed calibration enough at long range or do you end up correcting per scene?

Thx!
- Amine Mehdi M.


r/computervision • • 8h ago

Help: Project Struggling to optimize YOLO on a heavily imbalanced, small-object dataset (Road Damage - RDD2022). What approach yields the best ROI?

3 Upvotes

Hi everyone,

I am working on a computer vision project for road damage detection using the RDD2022 dataset and YOLO (currently using YOLO11). I've hit a plateau with my baseline and suspect a few "bugs" in how the dataset is structured for deep learning. I am looking for advice on the highest-ROI optimization steps before I spend more GPU hours.

The Dataset (The "Bug"):

  • Size: ~38,000 images, 4 classes (Longitudinal, Transverse, Alligator cracks, Potholes).
  • Imbalance: Longitudinal cracks make up 47% of the objects; Potholes are only 11%.
  • Small Objects: Transverse cracks and potholes have a median box size of only 30px-39px.
  • Country Skew: Images come from 7 countries. Norway has massive 4,000px images (which become tiny boxes when resized to 640), Japan has 58% of all Alligator cracks, and India has 49% of all Potholes.
  • The Problem: When training at 640px, about 23.5% of Norway's boxes become smaller than 8px, rendering them nearly invisible to the model.

Current Results:

  • Model: YOLO11n (Nano).
  • Training: 100 epochs, 640 resolution, basic augmentations.
  • Metrics: Stagnated around mAP50 ~0.52 and mAP50-95 ~0.26.

My Proposed Approaches (What should I prioritize?):

  1. Subsampling & Resizing: Since 13.5GB is heavy, my professor suggested resizing images to max 640px to save space. Should I also subsample the dataset (e.g., drop 50% of background images) to speed up training, or will that hurt the model's ability to learn general features?
  2. Changing the Split: Currently at 80/10/10 (Train/Val/Test). My professor suggested moving to 60/20/20. Is this better for a noisy, multi-country dataset to get a more honest validation score?
  3. Model Size: Should I jump to YOLO11m (Medium) or YOLOv8m for better small-object detection? (Note: I tried a No-Augmentation YOLO11m run for 25 epochs and it was terrible, so I know I need augmentations back on).
  4. Resolution: Should I bite the bullet and train at 1024px to save Norway's tiny cracks, or will the trade-off in training speed not be worth it?
  5. Augmentation: What specific YOLO augmentations work best for long, thin objects like cracks?

Question for the community:
If you had limited GPU hours (Colab/Kaggle), what sequence of experiments would you run to break past the 0.52 mAP50 plateau? Should I focus on the resolution, the model size, or the data processing first?

Thanks in advance for any insights!


r/computervision • • 13h ago

Discussion Qwen3-VL-235B: Good at vision, horrible at structured output

Post image
3 Upvotes

Recently I benchmarked the Qwen3-VL-235B, and the result was very disappointing, as it showed only marginal improvements over the Qwen3-VL-2B.

The unusually high failure rate tipped me to investigate the results. Initially I suspected the benchmark or provider integration, but checking the saved responses were all pointing to the model direct output.

As Qwen3-VL-235B from our provider did not had a native json_schema, I had to use prompt to generate json_object in response, and surprisingly there were alarming amount of failures (For 100 samples per tasks):

TaskFailed responsesInvalid JSONValid JSON, invalid schema or valuesMain issue
Grounding (Box)53476All coordinates placed under x1, often with a missing closing bracket
Grounding (Point)50644Coordinate arrays under x, or an array replacing the point object
Detection261412Malformed coordinate objects, wrong types and out-of-range values
Attributes505Values outside the provided vocabulary

Classification, VQA and captioning had no invalid responses. Probably due to simpler task schema.

For example, instead of separate numeric x and y fields, one response was:

{"status": "found", "point": {"x": [0.55, 0.24]}}

This goes beyond a missing bracket. Making the JSON parseable still leaves the wrong structure, and automatically assigning those array elements to different fields would mean interpreting the answer.

And just to be clear most of these failures weren’t caused by the token cap as almost all ended normally, and failed point responses used just 20–28 output tokens against a 4,096-token limit.

The Workaround

The practical approach was correction retries: send validation errors back to the model and let it regenerate, keeping the original schema. No ground-truth answers or scores are provided as feedback:

task = Ground(PydanticAIBackend(model, retries=2), target="the leftmost cat")
results = task('cat-garden.png')

That allows up to two additional attempts after an invalid response. A fresh run on the same samples produced:

TaskMetricNo retriesWith retries
ClassificationTop-1 Acc96%96%
DetectionF1@0.516.67%28.45%
Grounding (Box)Acc@0.531%81%
Grounding (Point)Point-in-box Acc41%85%
AttributesAcc73.86%78.02%
VQAVQA Acc68.4%64.9%
CaptioningROUGE-L0.38410.3837

Failures remain included in the scores. The retry run had zero final invalid responses across 700 samples, but requests averaged 1.2 per sample overall, rising to 1.55 for box grounding and 1.47 for point grounding.

The Fix

Probably moving to a provider that support json_schema at inference would be the best choice. Sure I can use PydanticAI json_schema but it is not constrained decoding. I did a test run and it helped but then it was not a clear win, but rather a trade-off:

Native JSON Schema was 10% cheaper overall, but 47% slower than the normal prompted run, with Detection driving most of the slowdown:

ConfigurationInput tokens / sampleOutput tokens / sampleRequests / sampleEstimated cost / 1,000 samplesMean latency (s)
Normal812651$0.6041.441
Up to 2 retries1,058801.196$0.7751.812
Native JSON Schema4421161$0.5432.121

Compared with normal:

  • Retries: cost +28%, latency +26%, total tokens +30%.
  • JSON Schema: cost −10%, latency +47%, total tokens −36%.
  • Compared with retries, JSON Schema cost 30% less, but took 17% longer.

The detection difference was substantial:

DetectionNormalJSON SchemaChange
Output tokens / sample162499+208%
Estimated cost / 1,000 samples$1.169$1.728+48%
Mean latency (s)2.9157.554+159%

Closing

There’s clearly useful visual capability in Qwen3-VL-235B, particularly in grounding. But unreliable structured output can become a serious pain when you need usable results in an application: extra requests, validation logic, latency and cost.

If you are interested in comparison between variants of Qwen3-VL models, check [Revised] Qwen3-VL: 2B vs 235B on standard vision tasks, if you want a comparison between 2B and 235B.

Code, runs, benchmarks all available at https://github.com/Tenkei/vlm-tasks


r/computervision • • 7h ago

Help: Project Need advice on accurate golf ball tracking using computer vision

4 Upvotes

I'm trying to build a golf ball tracking app similar to Shot Tracer and SmoothSwing. I've experimented with several open-source AI models, but I haven't been able to achieve reliable tracking accuracy.

The ball is tiny and moves very fast, making it difficult to track consistently across video frames.

I'm looking for advice from people experienced in computer vision, object tracking or ML:

  • What approaches/models work best for tracking tiny, fast-moving objects?
  • Should I combine object detection with temporal tracking, optical flow or Kalman filters?
  • Are there any specific models, datasets or research papers worth exploring?

I'm not looking for a basic YOLO implementation. I'd like to understand how to build a genuinely reliable tracking pipeline.

Any suggestions or experiences would be appreciated!