Mean average precision is a useful research metric and a poor procurement metric. It says nothing about the camera angle in your warehouse or the glare on your forecourt at 4 pm.
What to measure instead
- Event-level precision and recall for the alerts you will actually send.
- Performance by condition: night, rain, crowding, distance from camera.
- Latency and stability over time, not a single evaluation run.
- Operator agreement: do people confirm the system's events?
Build an evaluation set from day one
A few hundred labelled clips from your own cameras are worth more than any public benchmark. They also become the regression suite that protects you when models are updated.
All insights3Cubicles Team