With the surge in the number of Database-as-a-Service providers in the market today, many DBaaS providers claim to offer competitive database performance for your applications. The only true way of evaluating the performance of the database is by running performance benchmarks for the databases running on the cloud. In this article, we run performance benchmarks to evaluate, analyze, and compare the performance of MySQL database engines running on the AWS cloud for Tessell. We have used the Sysbench benchmarking tool for the benchmarking process.
Before starting the benchmarking process, you need to get your environment ready.
Prepare your environment
To prepare the environment for the benchmarking process, perform the following high-level tasks:
Provision a Tessell RDS MySQL instance to benchmark the performance testing. We are using the Tessell shape โtesl_8h_aโ on the AWS cloud. This shape comes with 8 vCPUs and 64 GB RAM. While provisioning the instance, note down the username and password to connect to your database instance.
Launch or create an Amazon EC2 instance to install the Sysbench benchmarking tool, and set up the load. It is recommended that you create the instance in the same Virtual Private Cloud (VPC) as your Tessell RDS instance to keep the latency minimum.
Set up the security groups for the client and server machines in a way that the client machine can connect to the server machine over the database port TCP:3306. For more information, see Default security groups for your VPC.
The following diagram shows the recommended environment for running the benchmarking process. The VPC located in the AWS cloud contains the Tessell RDS MySQL instance and the Sysbench client installed in the Amazon EC2 instance.
Provision the Sysbench client machine
Firstly, provision the client Linux machine to install the Sysbench benchmarking tool. For our test, we provisioned the Amazon Linux instance with the following configuration details:
Copied to clipboard!
Image: Amazon Linux 2 Kernel 5.10 AMI 2.0.20221210.1 x86_64 HVM gp2 Shape: m5.2xlarge
VPC: Same as the DB Service
Download Sysbench and configure MySQL
Once Sysbench is installed we need to configure following MySQL server items:
Login to MySQL server via any client and follow below steps.
Copied to clipboard!
https://github.com/akopytov/sysbench
Sysbench can be downloaded and configured from the repository link below for respective OS.
Database Name to run the Test: CREATE DATABASE sysbench
Run the benchmark
ย ย ย ย ย ย ย ย ย ย 1. Load the instance with desired database size ย ย ย ย ย ย ย ย ย ย ย ย ย โข Below Settings will load 100 Million Rows with 26 GB Data. For every 100 Million Rows it is 26 GB Data. Load per the benchmark requirements. Max Limits - Table size: 999 Million
ย ย ย ย ย ย ย ย ย 2. Run Write-Only Load Test and capture results. ย ย ย ย ย ย ย ย ย ย ย ย ย โข Adjust the no of threads, table size, time as per the test requirements. Max Limits - Threads: 1000, Time: 9999 seconds, Table_size: 999 Million, report-interval: This option outputs the ongoing test runs every n seconds defined.
When the performance test completes, We get the following TPCC based output.
QPS - Questions/Query Per Second
TPS - Transactions Per Second
Tessell MySQL Instance High Performance Shape - 8 vCPUs, 64 GB Memory
ย ย ย ย ย ย ย ย 1. Read Only Load
Copied to clipboard!
SQL statistics:
โโ queries performed:
โโโโ read: 107064216
โโโโ write: 0
โโโโ other: 15294888
โโโโ total: 122359104
โโ transactions: 7647444 (4246.62 per sec.)
โโ queries: 122359104 (67945.90 per sec.)
โโ ignored errors: 0 (0.00 per sec.)
โโ reconnects: 0 (0.00 per sec.)
General statistics:
โโ total time: 1800.2757s
โโ total number of events: 7647444
Latency (ms):
โโ min: 2.41
โโ avg: 235.41
โโ max: 4034.71
โโ 95th percentile: 253.35
โโ sum: 600093627.96
Threads fairness:
โโ events (avg/stddev): 2549.1480/66.30
โโ execution time (avg/stddev): 1800.0936/0.06
ย ย ย ย ย ย ย 2. Read Write Load
Copied to clipboard!
SQL statistics:
โโ queries performed:
โโโโ read: 96958176
โโโโ write: 27702336
โโโโ other: 13851168
โโโโ total: 138511680
โโ transactions: 13753764 (7690.98 per sec.)
โโ queries: 138511680 (76950.93 per sec.)
โโ ignored errors: 0 (0.00 per sec.)
โโ reconnects: 0 (0.00 per sec.)
General statistics:
โโ total time: 1800.1590s
โโ total number of events: 13753764
Latency (ms):
โโ min: 13.62
โโ avg: 129.97
โโ max: 1437.25
โโ 95th percentile: 231.53
โโ sum: 300032956.05
Threads fairness:
โโ events (avg/stddev): 2308.5280/85.96
โโ execution time (avg/stddev): 1800.0330/0.03
RDS for MySQL - 8 vCPUs, 64 GB Memory
ย ย ย ย ย ย 1. Read Only Load
Copied to clipboard!
SQL statistics:
โโ queries performed:
โโโโ read: 8778868
โโโโ write: 0
โโโโ other: 1254124
โโโโ total: 10032992
โโ transactions: 627062 (343.92 per sec.)
โโ queries: 10032992 (5502.76 per sec.)
โโ ignored errors: 0 (0.00 per sec.)
โโ reconnects: 0 (0.00 per sec.)
General statistics:
โโ total time: 1823.2650s
โโ total number of events: 627062
Latency (ms):
โโ min: 104.37
โโ avg: 2871.45
โโ max: 60337.62
โโ 95th percentile: 6026.41
โโ sum: 1800577096.12
Threads fairness:
โโ events (avg/stddev): 627.0620/5.62
โโ execution time (avg/stddev): 1800.5771/0.78
ย ย ย ย ย ย 2. Read Write Load
Copied to clipboard!
SQL statistics:
โโ queries performed:
โโโโ read: 62039418
โโโโ write: 17725548
โโโโ other: 8862774
โโโโ total: 88627740
โโ transactions: 4431387 (2461.48 per sec.)
โโ queries: 88627740 (49229.59 per sec.)
โโ ignored errors: 0 (0.00 per sec.)
โโ reconnects: 0 (0.00 per sec.)
General statistics:
โโ total time: 1800.2921s
โโ total number of events: 4431387
Latency (ms):
โโ min: 20.13
โโ avg: 406.20
โโ max: 2377.74
โโ 95th percentile: 623.33
โโ sum: 1800027924.89
Threads fairness:
โโ events (avg/stddev): 4431.3870/108.58
โโ execution time (avg/stddev): 1800.0279/0.03
Conclusion
It is, therefore, safe to conclude that Tessell RDS produced 321% higher IOPS as compared to AWS RDS for the identical Sysbench workload at the same cost. The image below depicts the visual comparison of the performance benchmark results that Tessell RDS and AWS RDS produce: