Most machine vision solutions require too much setup to be practical
Traditional machine vision on a shop floor usually means one of two things: a dedicated industrial camera with specialized software and a custom model trained on your specific defect types, or a general-purpose camera feeding a cloud API that flags anomalies. The first option is expensive and slow to deploy. The second means sending production images off-site, which isn't acceptable everywhere.
The interesting development over the past couple of years is that Vision Language Models have gotten good enough to describe and reason about images in useful detail without any fine-tuning. You can give a VLM a frame from a camera feed and a description of what you're looking for, and it can often tell you what it sees and whether anything looks wrong. That capability, running locally through Ollama, was the core idea behind Vision_Inspect.
The goal was a system that a small team could set up in a few minutes using hardware they already have, pointed at any process, with no training data required. Define what you want to watch for in plain language, point a camera at it, and the system does the rest.
Camera feed, VLM inference, live alerts
The Python backend handles camera capture using OpenCV. On a configurable interval, it grabs a frame, runs it through the VLM via Ollama, and evaluates the response against the inspection criteria you've defined. Qwen2.5-VL is the default model, chosen for its strong visual grounding and reasonable inference speed on consumer hardware.
Inspection criteria are defined in a configuration file. You describe what a passing state looks like, what failure states to watch for, and the severity level for each failure type. The system prompt sent to the VLM includes these criteria, which means you can change what the system monitors without retraining anything.
The system supports multiple inspection zones per camera frame. You can define regions of interest and assign different criteria to each one. A single camera can simultaneously watch for surface defects in one region while monitoring assembly completeness in another.
Latency constraints and the prompt engineering that makes it work
Inference latency is the main constraint. A VLM running on a mid-range GPU takes somewhere between 1 and 5 seconds per frame depending on model size and image resolution. That rules out per-frame inspection on a high-speed line, but it's completely workable for slower processes, periodic checks, or lines where a 2-3 second detection window is acceptable.
Prompt engineering ended up being more important than I expected. The VLM's usefulness as an inspector depends heavily on how the criteria are framed. Too vague and the model hedges everything. Too prescriptive and it hallucinates specifics. The sweet spot was asking the model to describe what it sees first, then evaluate against the criteria, which produces more reliable verdicts than asking for a direct pass/fail judgment.
The frontend was migrated from Create React App to Vite during development. The WebSocket proxy configuration needed care to pass the upgrade header correctly alongside FastAPI's WebSocket endpoint, but once resolved the dev feedback loop became fast enough to iterate on UI and prompt changes in parallel.