AeroPark started as a weekend question. Parking enforcement still means someone walking a lot with a handheld device. Could a drone fly the same lot, read every plate it sees, and flag the cars that have not paid, within seconds of seeing them?
The honest answer after building it: yes, in a prototype, and the machine learning was about a fifth of the work. This post is the other four fifths. If you are building any real-time vision pipeline, the problems below will find you, whatever your model.
What it does
Drone footage comes in as a video stream. A YOLOv8 model fine-tuned on plates seen from above finds vehicles and plates in each frame. PaddleOCR reads the plate crop. The text is fuzzy-matched against a list of paid plates from a parking API, and anything visible that is not on the list is flagged. A Streamlit dashboard shows the feed, live telemetry, plate reads with confidence, and an analysis log.
Running those steps one after another in a loop gave me about 9 frames a second on a laptop GPU. The target was 30, because that is what the footage arrives at, and anything slower means either dropping frames blindly or falling further behind every second.
Three threads, two queues
The fix was not a faster model. It was separating the three jobs that have nothing to do with each other.
The reader does one thing: decode the next frame, resize it once to the model's input size, stamp it with the time, and push it to a queue. The queue holds two frames. If it is full, the reader throws away the oldest and pushes the new one. It never waits.
The inference thread takes the newest frame it can get, runs detection, crops the plates, runs OCR on the crops, matches the text, and pushes a result object to a second queue. It runs as fast as the GPU allows and does not care that it is missing frames.
The display thread draws at the stream rate. Each tick it takes the newest raw frame and the newest result, draws the boxes and text from the result onto the frame, and updates the dashboard. When no new result has arrived it reuses the last one.
That last rule is what makes the feed look like 30 FPS even when inference is running at 12. Boxes lag the video by a few frames, which nobody notices from a drone, while a stuttering feed is the first thing anyone notices.
class Reader(threading.Thread):
def run(self):
while self.running:
ok, frame = self.cap.read()
if not ok:
break
frame = cv2.resize(frame, self.size)
item = (time.monotonic(), frame)
try:
self.q.put_nowait(item)
except queue.Full:
self.q.get_nowait() # drop the oldest, keep the stream current
self.q.put_nowait(item)
Bounded queues with drop-oldest are the whole trick. Unbounded queues feel safer and are a slow memory leak that ends with the display showing footage from forty seconds ago.
OCR is where the time goes
YOLOv8 at the sizes I used runs in a few milliseconds per frame on a laptop GPU. PaddleOCR on each plate crop does not. With four or five vehicles in frame, OCR was most of the inference budget.
Three changes mattered:
- Do not OCR every frame. A plate that was read a moment ago at the same position with good confidence does not need reading again. I track detections across frames by box overlap and only run OCR when a track is new or its last read was weak.
- Batch the crops. All plate crops from one frame go to PaddleOCR in one call. The per-call overhead was larger than the per-crop cost.
- Crop with margin, then straighten. Aerial plates arrive at an angle. A slightly larger crop and a cheap perspective correction raised OCR confidence more than any model setting.
Making confidence mean something
The dashboard shows a confidence next to every plate read, and the first version lied. OCR reported 0.9 on reads that were wrong in one character, because each character was confident on its own.
The score that felt honest combines three signals: the detector's box confidence, OCR's mean character confidence, and a format check against the plate pattern for the region. A read that fails the format check is capped low no matter what OCR says. A read that matches the paid list exactly gets a bump. Fuzzy matches within one edit are shown but marked, and never flagged as unpaid on their own.
This was the difference between a dashboard that looked impressive and one you could imagine someone acting on. A confidence score that people can trust is a product feature, not a number from a library.
Matching against the paid list
The parking API returns the plates that have paid. Matching plate text against it sounds trivial and is not, because OCR confuses 0 and O, 1 and I, 8 and B, and drones see plates partly covered by a roof rack.
The matcher normalises both sides to a canonical form, swapping the usual look-alike characters to one representative, then does an exact lookup, then a one-edit lookup. Only an exact normalised match clears a car. Everything else is a "check" state, which is the state a human would want.
What I would tell myself at the start
- Build the three-thread skeleton on day one with a dummy model that sleeps for 50 ms. Get the display smooth before the model exists.
- Measure per stage. One line printing milliseconds per stage every second told me more than any profiler.
- Decide what happens when you fall behind before you fall behind. Drop-oldest was right here. For a recording pipeline it would be wrong.
- Treat confidence as something you design for the person reading it.
AeroPark is not a product and I have not published the code; it was built to learn real-time vision end to end, and it did that. If you are working on something in the same space, email me. I would rather compare notes on the hard parts than publish half a prototype.