With the surge in the number of Database-as-a-Service providers in the market today, many DBaaS providers claim to offer competitive database performance for your applications. The only true way of evaluating the performance of the database is by running performance benchmarks for the databases running on the cloud. In this article, we run performance benchmarks to evaluate, analyze, and compare the performance of MySQL database engines running on the AWS cloud for Tessell. We have used the Sysbench benchmarking tool for the benchmarking process.
Before starting the benchmarking process, you need to get your environment ready.
Prepare your environment
To prepare the environment for the benchmarking process, perform the following high-level tasks:
Provision a Tessell RDS MySQL instance to benchmark the performance testing. We are using the Tessell shape βtesl_8h_aβ on the AWS cloud. This shape comes with 8 vCPUs and 64 GB RAM. While provisioning the instance, note down the username and password to connect to your database instance.
Launch or create an Amazon EC2 instance to install the Sysbench benchmarking tool, and set up the load. It is recommended that you create the instance in the same Virtual Private Cloud (VPC) as your Tessell RDS instance to keep the latency minimum.
Set up the security groups for the client and server machines in a way that the client machine can connect to the server machine over the database port TCP:3306. For more information, see Default security groups for your VPC.
The following diagram shows the recommended environment for running the benchmarking process. The VPC located in the AWS cloud contains the Tessell RDS MySQL instance and the Sysbench client installed in the Amazon EC2 instance.
Provision the Sysbench client machine
Firstly, provision the client Linux machine to install the Sysbench benchmarking tool. For our test, we provisioned the Amazon Linux instance with the following configuration details:
Copied to clipboard!
Image: Amazon Linux 2 Kernel 5.10 AMI 2.0.20221210.1 x86_64 HVM gp2 Shape: m5.2xlarge
VPC: Same as the DB Service
Download Sysbench and configure MySQL
Once Sysbench is installed we need to configure following MySQL server items:
Login to MySQL server via any client and follow below steps.
Copied to clipboard!
https://github.com/akopytov/sysbench
Sysbench can be downloaded and configured from the repository link below for respective OS.
Database Name to run the Test: CREATE DATABASE sysbench
Run the benchmark
Β Β Β Β Β Β Β Β Β Β 1. Load the instance with desired database size Β Β Β Β Β Β Β Β Β Β Β Β Β β’ Below Settings will load 100 Million Rows with 26 GB Data. For every 100 Million Rows it is 26 GB Data. Load per the benchmark requirements. Max Limits - Table size: 999 Million
Β Β Β Β Β Β Β Β Β 2. Run Write-Only Load Test and capture results. Β Β Β Β Β Β Β Β Β Β Β Β Β β’ Adjust the no of threads, table size, time as per the test requirements. Max Limits - Threads: 1000, Time: 9999 seconds, Table_size: 999 Million, report-interval: This option outputs the ongoing test runs every n seconds defined.
When the performance test completes, We get the following TPCC based output.
QPS - Questions/Query Per Second
TPS - Transactions Per Second
Tessell MySQL Instance High Performance Shape - 8 vCPUs, 64 GB Memory
Β Β Β Β Β Β Β Β 1. Read Only Load
Copied to clipboard!
SQL statistics:
ββ queries performed:
ββββ read: 107064216
ββββ write: 0
ββββ other: 15294888
ββββ total: 122359104
ββ transactions: 7647444 (4246.62 per sec.)
ββ queries: 122359104 (67945.90 per sec.)
ββ ignored errors: 0 (0.00 per sec.)
ββ reconnects: 0 (0.00 per sec.)
General statistics:
ββ total time: 1800.2757s
ββ total number of events: 7647444
Latency (ms):
ββ min: 2.41
ββ avg: 235.41
ββ max: 4034.71
ββ 95th percentile: 253.35
ββ sum: 600093627.96
Threads fairness:
ββ events (avg/stddev): 2549.1480/66.30
ββ execution time (avg/stddev): 1800.0936/0.06
Β Β Β Β Β Β Β 2. Read Write Load
Copied to clipboard!
SQL statistics:
ββ queries performed:
ββββ read: 96958176
ββββ write: 27702336
ββββ other: 13851168
ββββ total: 138511680
ββ transactions: 13753764 (7690.98 per sec.)
ββ queries: 138511680 (76950.93 per sec.)
ββ ignored errors: 0 (0.00 per sec.)
ββ reconnects: 0 (0.00 per sec.)
General statistics:
ββ total time: 1800.1590s
ββ total number of events: 13753764
Latency (ms):
ββ min: 13.62
ββ avg: 129.97
ββ max: 1437.25
ββ 95th percentile: 231.53
ββ sum: 300032956.05
Threads fairness:
ββ events (avg/stddev): 2308.5280/85.96
ββ execution time (avg/stddev): 1800.0330/0.03
RDS for MySQL - 8 vCPUs, 64 GB Memory
Β Β Β Β Β Β 1. Read Only Load
Copied to clipboard!
SQL statistics:
ββ queries performed:
ββββ read: 8778868
ββββ write: 0
ββββ other: 1254124
ββββ total: 10032992
ββ transactions: 627062 (343.92 per sec.)
ββ queries: 10032992 (5502.76 per sec.)
ββ ignored errors: 0 (0.00 per sec.)
ββ reconnects: 0 (0.00 per sec.)
General statistics:
ββ total time: 1823.2650s
ββ total number of events: 627062
Latency (ms):
ββ min: 104.37
ββ avg: 2871.45
ββ max: 60337.62
ββ 95th percentile: 6026.41
ββ sum: 1800577096.12
Threads fairness:
ββ events (avg/stddev): 627.0620/5.62
ββ execution time (avg/stddev): 1800.5771/0.78
Β Β Β Β Β Β 2. Read Write Load
Copied to clipboard!
SQL statistics:
ββ queries performed:
ββββ read: 62039418
ββββ write: 17725548
ββββ other: 8862774
ββββ total: 88627740
ββ transactions: 4431387 (2461.48 per sec.)
ββ queries: 88627740 (49229.59 per sec.)
ββ ignored errors: 0 (0.00 per sec.)
ββ reconnects: 0 (0.00 per sec.)
General statistics:
ββ total time: 1800.2921s
ββ total number of events: 4431387
Latency (ms):
ββ min: 20.13
ββ avg: 406.20
ββ max: 2377.74
ββ 95th percentile: 623.33
ββ sum: 1800027924.89
Threads fairness:
ββ events (avg/stddev): 4431.3870/108.58
ββ execution time (avg/stddev): 1800.0279/0.03
Conclusion
It is, therefore, safe to conclude that Tessell RDS produced 321% higher IOPS as compared to AWS RDS for the identical Sysbench workload at the same cost. The image below depicts the visual comparison of the performance benchmark results that Tessell RDS and AWS RDS produce: