Performance Benchmarking Disk IO

Tool Used: sysbench, Phoronix Test Suite

First Instance ECS on Cloud:

root@iZj6c5vmzbgujq3xi2m45aZ:~# phoronix-test-suite system-info

Phoronix Test Suite v5.2.1
System Information

Hardware:
Processor: Intel Xeon E5-2682 v4 @ 2.50GHz (1 Core), Motherboard: Alibaba Cloud ECS, Chipset: Intel 440FX- 82441FX PMC, Memory: 1 x 1024 MB RAM, Disk: 40GB, Graphics: Cirrus Logic GD 5446, Network: Red Hat Virtio device

Software:
OS: Ubuntu 18.04, Kernel: 4.15.0-45-generic (x86_64), File-System: ext4, Screen Resolution: 1024x768, System Layer: KVM

Second similar Instance on Azure Cloud:

root@testbenchmark:~# phoronix-test-suite system-info

Phoronix Test Suite v5.2.1
System Information

Hardware:
Processor: Intel Xeon Platinum 8171M @ 2.10GHz (1 Core), Motherboard: Microsoft Virtual Machine v7.0, Chipset: Intel 440BX/ZX/DX, Memory: 1 x 1024 MB Microsoft, Disk: 32GB Virtual Disk + 4GB Virtual Disk, Graphics: Microsoft Hyper-V virtual VGA

Software:
OS: Ubuntu 18.04, Kernel: 5.0.0-1035-azure (x86_64), File-System: ext4, Screen Resolution: 1152x864, System Layer: Microsoft Hyper-V Server



ECS
Azure VM
Processor
Intel Xeon E5-2682 v4 @ 2.50GHz (1 Core)
Intel Xeon Platinum 8171M @ 2.10GHz (1 Core)
Motherboard
Alibaba Cloud ECS
Microsoft Virtual Machine v7.0
Chipset
Intel 440FX- 82441FX PMC
Intel 440BX/ZX/DX
Memory
1 x 1024 MB RAM
1 x 1024 MB Microsoft
Disk
40GB
32GB Virtual Disk + 4GB Virtual Disk
Graphics
Cirrus Logic GD 5446
Microsoft Hyper-V virtual VGA
Network
Red Hat Virtio device

OS
Ubuntu 18.04
Ubuntu 18.04
Kernel
4.15.0-45-generic (x86_64)
5.0.0-1035-azure (x86_64)
File-System
ext4
ext4
Screen Resolution
1024x768
1152x864
System Layer
KVM
Microsoft Hyper-V Server

NOTE: The test and comparison here is very high level and the purpose here is to show how I do usually benchmarking and reporting.

Fileio workload

When using fileio, we will need to create a set of test files to work on. The recommended way is that the file size is larger than the available memory to ensure that file caching does not influence the workload too much. In our case we have 1 GB of RAM so I have taken file size of 10 GB for this test.
As this is I/O benchmarking, we can tell sysbench which kind of workload we want to run: sequential reads, writes or random reads, writes, or a combination. In this below example, random read/write is used (rndrw). The duration of the test is given through the --time option (in seconds).
The output of a run is shown below on Alibaba Cloud: In First Step we created files and last step the test files are removed.

root@iZj6c5vmzbgujq3xi2m45aZ:~# sysbench fileio --file-total-size=10G prepare
sysbench 1.0.11 (using system LuaJIT 2.1.0-beta3)

128 files, 81920Kb each, 10240Mb total
Creating files for the test...
Extra file open flags: 0
Creating file test_file.0
Creating file test_file.1
Creating file test_file.2
……
……
Creating file test_file.127
10737418240 bytes written in 73.47 seconds (139.37 MiB/sec).

root@iZj6c5vmzbgujq3xi2m45aZ:~# sysbench fileio --file-total-size=10G --file-test-mode=rndrw --time=300 --max-requests=0 run
sysbench 1.0.11 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time


Extra file open flags: 0
128 files, 80MiB each
10GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads...


Threads started!

File operations:
    reads/s:                      1329.33
    writes/s:                     886.22
    fsyncs/s:                     2835.76

Throughput:
    read, MiB/s:                  20.77
    written, MiB/s:               13.85

General statistics:
    total time:                          300.0137s
    total number of events:              1515475

Latency (ms):
         min:                                  0.00
         avg:                                  0.20
         max:                                 29.13
         95th percentile:                      0.52
         sum:                             298177.24

Threads fairness:
    events (avg/stddev):           1515475.0000/0.00
    execution time (avg/stddev):   298.1772/0.00

root@iZj6c5vmzbgujq3xi2m45aZ:~# sysbench fileio --file-total-size=10G cleanup
sysbench 1.0.11 (using system LuaJIT 2.1.0-beta3)
Removing test files...

The important part to look at is the information regarding the operations:

File operations:
    reads/s:                      1329.33
    writes/s:                     886.22
    fsyncs/s:                     2835.76

Throughput:
    read, MiB/s:                  20.77
    written, MiB/s:               13.85

These numbers can be compared with runs on different file systems, other systems, etc.

On Azure Cloud similar test:

root@testbenchmark:~# sysbench fileio --file-total-size=10G prepare
sysbench 1.0.11 (using system LuaJIT 2.1.0-beta3)

128 files, 81920Kb each, 10240Mb total
Creating files for the test...
Extra file open flags: 0
Creating file test_file.0
Creating file test_file.1
Creating file test_file.127
10737418240 bytes written in 773.51 seconds (13.24 MiB/sec).

root@testbenchmark:~# sysbench fileio --file-total-size=10G --file-test-mode=rndrw --time=300 --max-requests=0 run
sysbench 1.0.11 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time

Extra file open flags: 0
128 files, 80MiB each
10GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads...

Threads started!

File operations:
    reads/s:                      263.18
    writes/s:                     175.45
    fsyncs/s:                     561.05

Throughput:
    read, MiB/s:                  4.11
    written, MiB/s:               2.74

General statistics:
    total time:                          300.0195s
    total number of events:              299928

Latency (ms):
         min:                                  0.00
         avg:                                  1.00
         max:                                 99.29
         95th percentile:                      3.68
         sum:                             299674.75

Threads fairness:
    events (avg/stddev):           299928.0000/0.00
    execution time (avg/stddev):   299.6748/0.00

root@testbenchmark:~# sysbench fileio --file-total-size=10G cleanup
sysbench 1.0.11 (using system LuaJIT 2.1.0-beta3)

Removing test files...

Result: To create 10 GB file in Alibaba it took 73 sec with written speed 139 MiB/sec while in Azure cloud it took 773 second with written speed 13 MiB/sec although Premium SSD was chosen on Azure cloud. Similarly clearly File Operation, Throughput etc is much better in Alibaba cloud.

On Azure VM:


On ECS:



Another Disk test using Phoronix Test Suite. PTS is the most comprehensive testing and benchmarking platform available for Linux.

hasnain@benchmarktest:~$ phoronix-test-suite info pts/disk

Phoronix Test Suite v5.2.1
Disk Test Suite

Run Identifier: pts/disk-1.3.1
Suite Version: 1.3.1
Maintainer: Michael Larabel
Suite Type: Disk
Unique Tests: 7
Suite Description: This test suite is designed to contain real-world disk and file-system tests.

pts/disk-1.3.1
  * pts/sqlite
  * pts/fs-mark
  * pts/compilebench
  * pts/iozone
  * pts/dbench
  * pts/postmark
  * pts/fio


# No of test available
hasnain@benchmarktest:~$ phoronix-test-suite list-available-tests | wc -l
376

# To get no of test for Disk
hasnain@benchmarktest:~$ phoronix-test-suite list-available-tests | grep Disk
pts/aio-stress                 - AIO-Stress                          Disk
pts/blogbench                  - BlogBench                           Disk
pts/compilebench               - Compile Bench                       Disk
pts/dbench                     - Dbench                              Disk
pts/fio                        - Flexible IO Tester                  Disk
pts/fs-mark                    - FS-Mark                             Disk
pts/hdparm-read                - hdparm Timed Disk Reads             Disk
pts/ior                        - IOR                                 Disk
pts/iozone                     - IOzone                              Disk
pts/pjdfstest                  - Pjdfstest                           Disk
pts/postmark                   - PostMark                            Disk
pts/sqlite                     - SQLite                              Disk
pts/tiobench                   - Threaded I/O Tester                 Disk
pts/unpack-linux               - Unpacking The Linux Kernel          Disk

hasnain@benchmarktest:~$ phoronix-test-suite benchmark pts/unpack-linux

This test measures how long it takes to extract the .tar.xz Linux kernel package.





For other performance benchmarking refer:


Post a Comment

Thanks for your comment !
I will review your this and will respond you as soon as possible.