Benchmarking

Drive Rekor Scout® on all CPU cores to benchmark speed for various video resolutions.

Example Benchmarks

Benchmarks demonstrating the power of Rekor Scout are presented below. If you find yourself thinking, “These numbers look too good to be true”, we understand. Don't take our word for it.

‍Try it out yourself!

Plate number accuracy

Benchmark	License plate state identified	Correct in 1 of 10 estimates	Correct on 1st estimate
OpenALPR Open Source	0.00%	65.69%	42.16%
Rekor Scout®	96.83%	100.00%	99.02%

CPU hardware video performance

Benchmarks were conducted using version 4.1.13 of the Rekor Scout Agent.

System Profile	720p	1080p	4K	AWS Instance Type
I7-6700 CPU@3.40GHZ	17.3 FPS	11.7 FPS	6.9 FPS
i7- 870 CPU@ 2.93GHz	26.2 FPS	16.0 FPS	8.8 FPS
i7-9750H CPU @2.60GHz	28.4 FPS	19.7 FPS	4.2 FPS
Xeon E5506 CPU@ 2.13GHz	29.5 FPS	19.9 FPS	10.8 FPS
Xeon Platinum 8275CL CPU @ 3.00GHz	105.5 FPS	66.2 FPS	29.2 FPS	c5.2xlarge
Xeon Platinum 8488C	129.2 FPS	81.3 FPS	36.6 FPS	c5n.2xlarge
Intel Xeon CPU E5-2686 v4 @ 2.30GHz	31.4 FPS	19.6 FPS	8.7 FPS	t2.large
Intel Xeon Platinum 8151 CPU @ 3.40GHz	31.1 FPS	18.9 FPS	8.2 FPS	z1d.large
AMD EPYC 7571	15.1 FPS	9.6 FPS	4.9 FPS	t3a.large
Intel Xeon Platinum 8124M CPU @ 3.00GHz	148.7 FPS	112.3 FPS	70.9 FPS	c5.9xlarge
Intel Xeon Platinum 8175M CPU @ 2.50GHz	88.2 FPS	57.2 FPS	26.2 FPS	m5.2xlarge
Intel Xeon Platinum 8259CL CPU @ 2.50GHz	23.8 FPS	14.6 FPS	6.4 FPS	t3.large
Intel Xeon CPU E5-2666 v3 @ 2.90GHz	24.7 FPS	15.0 FPS	6.7 FPS	c4.large
Intel Xeon CPU E7-8880 v3 @ 2.30GHz	36.2 FPS	22.3 FPS	10.3 FPS	x1e.xlarge
Intel Xeon Platinum 8175M CPU @ 2.50GHz	124.3 FPS	87.2 FPS	44.6 FPS	m5.4xlarge

NVIDIA GPU video performance

Benchmarks were conducted using version 4.1.13 of the Rekor Scout Agent with NVIDIA GPU Acceleration enabled.

System Profile	720p	1080p	4K
GTX 1660 Ti	41.2 FPS	28.7 FPS	8.2 FPS
T400	26.6 FPS	16.1 FPS	8.8 FPS
NVIDIA Jetson Xavier NX	20.1 FPS	13.2 FPS	6.8 FPS

Prerequisites

• Rekor Scout® Basic or Pro license/subscription

• Ubuntu 22.04, Ubuntu 20.04, Ubuntu 18.04, Windows 10, or Windows 11

• Python (2 or 3)

Installation

Generic

1. Download the Vehicle Recognition SDK
2. Clone this repository git clone https://github.com/openalpr/speed_benchmark.git

3. Install the Python requirements pip install -r requirements.txt

Docker

docker run -it --rm -v /etc/openalpr:/etc/openalpr/ openalpr/commercial-agent /bin/bash
apt update && apt install -y curl python-pip git
git clone https://github.com/openalpr/speed_benchmark.git
cd speed_benchmark/
pip install -r requirements.txt
bash <(curl https://deb.openalpr.com/install)  # Select SDK

Usage

1. View all command line options by running python speed_benchmark.py -h

2. Select your desired resolution(s) - vga, 720p, 1080p, and/or 4k

3. Benchmark using the default flags (1 stream and no minimum CPU threshold) by running python speed_benchmark.py

4. Check the average CPU utilization (see sample output below). Resolutions with utilization of less than 95% are bottlenecked on decoding the video stream (typical for higher resolutions). These should be rerun with additional streams for a better estimate of maximum performance

5. Set the --thres to a non-zero value. This causes the program to add streams until the threshold CPU utilization is achieved. We recommend using 90 < thres < 95. On large systems where the CPU utilization for a single stream is much lower than your desired threshold, you can reduce the granularity of the search by setting --steps > 1

6. Estimate the number of cameras for a given total FPS value by using the following per-camera rules of thumb

   • Low Speed (under 25 mph): 5-10 fps

   • Medium Speed (25-45 mph): 10-15 fps

   • High Speed (over 45 mph): 15-30 fps

Sample Output

Using default options

user@ubuntu:~/git/speed-bench$ python speed_benchmark.py
Initializing...
	Operating system: Linux
	CPU model: Intel Core i7-8750H CPU @ 2.20GHz
	OpenALPR version: 2.7.101
	Runtime data: /usr/share/openalpr/runtime_data
	OpenALPR configuration: /usr/share/openalpr/config/openalpr.defaults.conf
Downloading benchmark videos...
	Downloaded vga
	Downloaded 720p
	Downloaded 1080p
	Downloaded 4k
Testing with 1 stream(s)...
	Processing vga
	Processing 720p
	Processing 1080p
	Processing 4k
	Lowest average CPU usage 81.4%
+---------------------------------------------------------+
|        OpenALPR Speed: 1 stream(s) on 12 threads        |
+------------+-----------+-----------+-----------+--------+
| Resolution | Total FPS | CPU (Avg) | CPU (Max) | Frames |
+------------+-----------+-----------+-----------+--------+
|    vga     |    52.9   |    81.4   |    99.4   |  479   |
|    720p    |    49.6   |    84.9   |    99.5   |  479   |
|   1080p    |    44.4   |    88.8   |   100.0   |  479   |
|     4k     |    23.8   |    93.7   |   100.0   |  479   |
+------------+-----------+-----------+-----------+--------+
Saving results to /home/user/git/speed_benchmark/speed-bench-20190618.csv

Starting with 3 streams and incrementing by 2 each time 95% CPU utilization is not achieved

user@ubuntu:~/git/speed-bench$ python speed_benchmark.py --thres 95 --streams 3 --step 2
Initializing...
	Operating system: Linux
	CPU model: Intel Core i7-8750H CPU @ 2.20GHz
	OpenALPR version: 2.7.101
	Runtime data: /usr/share/openalpr/runtime_data
	OpenALPR configuration: /usr/share/openalpr/config/openalpr.defaults.conf
Downloading benchmark videos...
	Found local vga
	Found local 720p
	Found local 1080p
	Found local 4k
Testing with 3 stream(s)...
	Processing vga
	Processing 720p
	Processing 1080p
	Processing 4k
	Lowest average CPU usage 93.2%
Testing with 5 stream(s)...
	Processing vga
	Processing 720p
	Processing 1080p
	Processing 4k
	Lowest average CPU usage 95.3%
+---------------------------------------------------------+
|        OpenALPR Speed: 5 stream(s) on 12 threads        |
+------------+-----------+-----------+-----------+--------+
| Resolution | Total FPS | CPU (Avg) | CPU (Max) | Frames |
+------------+-----------+-----------+-----------+--------+
|    vga     |    66.5   |    95.3   |   100.0   |  479   |
|    720p    |    61.3   |    96.2   |   100.0   |  479   |
|   1080p    |    54.1   |    97.3   |   100.0   |  479   |
|     4k     |    29.5   |    99.2   |   100.0   |  479   |
+------------+-----------+-----------+-----------+--------+
Saving results to /home/user/git/speed_benchmark/speed-bench-20190618.csv