Performance Benchmarking Memory

Tool Used: sysbench, Phoronix Test Suite

First Instance ECS on Cloud:

root@iZj6c5vmzbgujq3xi2m45aZ:~# phoronix-test-suite system-info

Phoronix Test Suite v5.2.1

System Information

Hardware:

Processor: Intel Xeon E5-2682 v4 @ 2.50GHz (1 Core), Motherboard: Alibaba Cloud ECS, Chipset: Intel 440FX- 82441FX PMC, Memory: 1 x 1024 MB RAM, Disk: 40GB, Graphics: Cirrus Logic GD 5446, Network: Red Hat Virtio device

Software:

OS: Ubuntu 18.04, Kernel: 4.15.0-45-generic (x86_64), File-System: ext4, Screen Resolution: 1024x768, System Layer: KVM

Second similar Instance on Azure Cloud:

root@testbenchmark:~# phoronix-test-suite system-info

Phoronix Test Suite v5.2.1

System Information

Hardware:

Processor: Intel Xeon Platinum 8171M @ 2.10GHz (1 Core), Motherboard: Microsoft Virtual Machine v7.0, Chipset: Intel 440BX/ZX/DX, Memory: 1 x 1024 MB Microsoft, Disk: 32GB Virtual Disk + 4GB Virtual Disk, Graphics: Microsoft Hyper-V virtual VGA

Software:

OS: Ubuntu 18.04, Kernel: 5.0.0-1035-azure (x86_64), File-System: ext4, Screen Resolution: 1152x864, System Layer: Microsoft Hyper-V Server

	ECS	Azure VM
Processor	Intel Xeon E5-2682 v4 @ 2.50GHz (1 Core)	Intel Xeon Platinum 8171M @ 2.10GHz (1 Core)
Motherboard	Alibaba Cloud ECS	Microsoft Virtual Machine v7.0
Chipset	Intel 440FX- 82441FX PMC	Intel 440BX/ZX/DX
Memory	1 x 1024 MB RAM	1 x 1024 MB Microsoft
Disk	40GB	32GB Virtual Disk + 4GB Virtual Disk
Graphics	Cirrus Logic GD 5446	Microsoft Hyper-V virtual VGA
Network	Red Hat Virtio device
OS	Ubuntu 18.04	Ubuntu 18.04
Kernel	4.15.0-45-generic (x86_64)	5.0.0-1035-azure (x86_64)
File-System	ext4	ext4
Screen Resolution	1024x768	1152x864
System Layer	KVM	Microsoft Hyper-V Server

NOTE: The test and comparison here is very high level and the purpose here is to show how I do usually benchmarking and reporting.

Memory workload

Test1:

When using the memory test in sysbench, the benchmark application will allocate a memory buffer and then read or write from it, each time for the size of a pointer (so 32bit or 64bit), and each execution until the total buffer size has been read from or written to. This is then repeated until the provided volume (--memory-total-size) is reached. We can provide multiple threads (--num-threads), different sizes in buffer (--memory-block-size) and the type of requests (read or write, sequential or random).

On ECS:

root@iZj6c5vmzbgujq3xi2m45aZ:~# sysbench memory --threads=4 run

sysbench 1.0.11 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:

Number of threads: 4

Initializing random number generator from current time

Running memory speed test with the following options:

block size: 1KiB

total size: 102400MiB

operation: write

scope: global

Initializing worker threads...

Threads started!

Total operations: 35340155 (3533220.67 per second)

34511.87 MiB transferred (3450.41 MiB/sec)

General statistics:

total time: 10.0002s

total number of events: 35340155

Latency (ms):

min: 0.00

avg: 0.00

max: 1415.56

95th percentile: 0.00

sum: 19612.98

Threads fairness:

events (avg/stddev): 8835038.7500/522232.48

execution time (avg/stddev): 4.9032/0.28

The important number to compare is the throughput and operations per second:

Total operations: 35340155 (3533220.67 per second)

34511.87 MiB transferred (3450.41 MiB/sec)

On Azure VM:

root@testbenchmark:~# sysbench memory --threads=4 run

sysbench 1.0.11 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:

Number of threads: 4

Initializing random number generator from current time

Running memory speed test with the following options:

block size: 1KiB

total size: 102400MiB

operation: write

scope: global

Initializing worker threads...

Threads started!

Total operations: 34568406 (3456219.26 per second)

33758.21 MiB transferred (3375.21 MiB/sec)

General statistics:

total time: 10.0001s

total number of events: 34568406

Latency (ms):

min: 0.00

avg: 0.00

max: 29.84

95th percentile: 0.00

sum: 17935.95

Threads fairness:

events (avg/stddev): 8642101.5000/40252.48

execution time (avg/stddev): 4.4840/0.21

Result: total time is approximately same in both machine.

Test2:

pts/ramspeed: This benchmark tests the system memory (RAM) performance

Memory Test Configuration

1: Copy

2: Scale

3: Add

4: Triad

5: Average

6: Test All Options

Type: 6

1: Integer

2: Floating Point

3: Test All Options

Benchmark: 3

On Alibaba Cloud: Test Results Uploaded To:
https://openbenchmarking.org/result/2004047-KH-ALIBABAEC61

On Azure Cloud: Test Results Uploaded To:
https://openbenchmarking.org/result/2004045-KH-AZUREVMME88

Result:
https://openbenchmarking.org/result/2004047-KH-ALIBABAEC61,%202004045-KH-AZUREVMME88

	ECS RAM	Azure VM RAM
RAMspeed SMP - Type: Add - Benchmark: Integer	7288.58	9559.93
RAMspeed SMP - Type: Copy - Benchmark: Integer	6364.63	9921.63
RAMspeed SMP - Type: Scale - Benchmark: Integer	6325.14	9516.27
RAMspeed SMP - Type: Triad - Benchmark: Integer	7044.71	9709.39
RAMspeed SMP - Type: Average - Benchmark: Integer	6824.29	9789.98
RAMspeed SMP - Type: Add - Benchmark: Floating Point	6972.92	9586.6
RAMspeed SMP - Type: Copy - Benchmark: Floating Point	6489.8	9446.13
RAMspeed SMP - Type: Scale - Benchmark: Floating Point	6195.74	9607.01
RAMspeed SMP - Type: Triad - Benchmark: Floating Point	6394.26	9599.64
RAMspeed SMP - Type: Average - Benchmark: Floating Point	2346.58	9684.32