Skip to content

C++ STL Map and HashMap Benchmark vs. CrossDB

Test Tool

This tool will use auto-commit transaction to test single CRUD performance for CrossDB Prepared STMT.

Bench Tool

Bench Test Framework

STL Map&HashMap Bench Driver

CrossDB Bench Driver

The test scripts will conduct five rounds of testing and select the average value.

crossdb/bench/basic$ ./bench-stlmap -q -r 5 -n 1000
crossdb/bench/basic$ ./bench-stlmap -q -r 5 -n 10000
crossdb/bench/basic$ ./bench-stlmap -q -r 5 -n 100000
crossdb/bench/basic$ ./bench-stlmap -q -r 5 -n 1000000
crossdb/bench/basic$ ./bench-stlmap -q -r 5 -n 10000000

crossdb/bench/basic$ ./bench-crossdb -q -r 5 -n 1000
crossdb/bench/basic$ ./bench-crossdb -q -r 5 -n 10000
crossdb/bench/basic$ ./bench-crossdb -q -r 5 -n 100000
crossdb/bench/basic$ ./bench-crossdb -q -r 5 -n 1000000
crossdb/bench/basic$ ./bench-crossdb -q -r 5 -n 10000000

Note

  • To ensure fairness in testing, both STL Map and HashMap (unordered_map) use pthread read-write locks, as CrossDB is thread-safe with read-write locks by default.
  • std::shared_mutex is not used because, in a single-threaded context, the compiler optimizes the code and omits the lock.
  • CrossDB will support a lockless mode in the future, and the benchmark tool will offer an option to configure the lock mode.

Note

Test results will vary depending on the CPU, OS, compiler, and system load. Even on the same server, the results will differ each time.

Test Server

CPU         : 11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz, Cache size 16384 KB
OS          : Ubuntu 24.04
CrossDB     : 0.8.0
Compiler    : gcc 13.2.0 with -O2 optimization
c++ std     : c++17

In-Memory DB Test with INT

1,000 Rows

Sequential Access Test

DB Access Insert QPS Query QPS Update QPS Delete QPS
STL Map 10,442,591 21,776,205 27,627,912 19,777,677
CrossDB STMT 17,773,584 28,868,027 22,767,213 23,593,441
STL HashMap 14,284,640 44,081,055 46,764,986 38,358,974

Random Access Test

DB Access Insert QPS Query QPS Update QPS Delete QPS
STL Map 9,841,660 15,349,472 17,730,115 9,580,807
CrossDB STMT 20,171,189 28,834,741 22,076,943 21,973,314
STL HashMap 14,975,036 42,500,933 48,155,156 36,146,519

10,000 Rows

Sequential Access Test

DB Access Insert QPS Query QPS Update QPS Delete QPS
STL Map 8,113,054 16,625,624 19,813,725 22,953,185
CrossDB STMT 14,752,474 28,119,037 22,175,662 22,469,934
STL HashMap 18,570,206 44,608,531 46,079,259 39,167,902

Random Access Test

DB Access Insert QPS Query QPS Update QPS Delete QPS
STL Map 8,629,238 8,951,170 9,888,694 6,962,080
CrossDB STMT 16,550,826 26,314,983 21,294,530 21,212,114
STL HashMap 19,087,491 30,989,837 43,464,636 29,057,718

100,000 Rows

Sequential Access Test

DB Access Insert QPS Query QPS Update QPS Delete QPS
STL Map 5,145,771 15,298,169 17,967,132 22,018,252
CrossDB STMT 19,700,788 28,480,655 21,548,692 23,305,607
STL HashMap 20,788,054 49,810,428 53,449,199 46,489,166

Random Access Test

DB Access Insert QPS Query QPS Update QPS Delete QPS
STL Map 5,815,344 5,163,883 5,531,051 4,941,685
CrossDB STMT 20,411,675 22,748,379 18,789,522 19,783,731
STL HashMap 21,936,850 26,063,150 37,778,169 23,640,037

1,000,000 Rows

Sequential Access Test

DB Access Insert QPS Query QPS Update QPS Delete QPS
STL Map 2,674,488 12,008,803 13,194,472 14,606,515
CrossDB STMT 18,477,686 27,987,661 21,844,773 23,105,578
STL HashMap 20,543,816 48,752,370 49,362,932 45,854,590

Random Access Test

DB Access Insert QPS Query QPS Update QPS Delete QPS
STL Map 3,655,302 1,868,468 2,006,574 1,829,826
CrossDB STMT 18,610,416 6,568,603 6,631,045 6,341,988
STL HashMap 22,427,027 7,474,584 10,213,001 6,075,652

10,000,000 Rows

Sequential Access Test

DB Access Insert QPS Query QPS Update QPS Delete QPS
STL Map 1,722,853 7,860,464 7,879,262 9,947,688
CrossDB STMT 17,671,148 26,329,139 20,146,554 20,992,326
STL HashMap 20,558,213 48,890,089 49,051,956 45,776,292

Random Access Test

DB Access Insert QPS Query QPS Update QPS Delete QPS
STL Map 2,154,541 1,035,817 1,041,939 1,094,936
CrossDB STMT 18,768,313 4,923,951 4,299,394 4,622,262
STL HashMap 22,269,633 6,651,748 8,840,238 4,838,402

Comments