Sunday, July 25, 2021

disk tests between HDD servers

SERVER-ID 40143   

~# dd if=/dev/zero of=/root/testfile bs=4k count=10000 oflag=direct
10000+0 records in
10000+0 records out
40960000 bytes (41 MB, 39 MiB) copied, 0.446108 s, 91.8 MB/s
~# dd if=/dev/zero of=/root/testfile bs=1M count=1000 oflag=direct
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 4.92345 s, 213 MB/s
~# fio -filename=/root/test_randwrite -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=4k -size=10M -numjobs=10 -runtime=10 -group_reporting -name=mytest
mytest: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.12
Starting 10 threads
mytest: Laying out IO file (1 file / 10MiB)
Jobs: 9 (f=9): [w(8),_(1),w(1)][55.6%][w=6224KiB/s][w=1556 IOPS][eta 00m:08s]
mytest: (groupid=0, jobs=10): err= 0: pid=24501: Sun Jul 25 18:53:39 2021
write: IOPS=1533, BW=6132KiB/s (6279kB/s)(59.9MiB/10010msec); 0 zone resets
clat (usec): min=47, max=781215, avg=6432.29, stdev=45635.14
lat (usec): min=47, max=781216, avg=6433.16, stdev=45635.14
clat percentiles (usec):
| 1.00th=[ 178], 5.00th=[ 231], 10.00th=[ 249], 20.00th=[ 277],
| 30.00th=[ 302], 40.00th=[ 326], 50.00th=[ 371], 60.00th=[ 453],
| 70.00th=[ 693], 80.00th=[ 1123], 90.00th=[ 1467], 95.00th=[ 1762],
| 99.00th=[267387], 99.50th=[362808], 99.90th=[583009], 99.95th=[658506],
| 99.99th=[759170]
bw ( KiB/s): min= 8, max= 4456, per=10.56%, avg=647.79, stdev=777.62, samples=188
iops : min= 2, max= 1114, avg=161.94, stdev=194.40, samples=188
lat (usec) : 50=0.01%, 100=0.03%, 250=10.20%, 500=52.54%, 750=9.03%
lat (usec) : 1000=5.70%
lat (msec) : 2=18.89%, 4=1.36%, 10=0.21%, 20=0.06%, 50=0.03%
lat (msec) : 100=0.07%, 250=0.72%, 500=0.96%, 750=0.18%, 1000=0.01%
cpu : usr=0.25%, sys=1.81%, ctx=30705, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,15346,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=6132KiB/s (6279kB/s), 6132KiB/s-6132KiB/s (6279kB/s-6279kB/s), io=59.9MiB (62.9MB), run=10010-10010msec
Disk stats (read/write):
sda: ios=0/15187, merge=0/33, ticks=0/9174, in_queue=8432, util=83.67%


SERVER-ID 40396

~# dd if=/dev/zero of=/root/testfile bs=4k count=500 oflag=dsync
\500+0 records in
500+0 records out
2048000 bytes (2.0 MB, 2.0 MiB) copied, 71.0946 s, 28.8 kB/s
~# dd if=/dev/zero of=/root/testfile bs=1M count=50 oflag=dsync
50+0 records in
50+0 records out
52428800 bytes (52 MB, 50 MiB) copied, 6.77477 s, 7.7 MB/s~# dd if=/dev/zero of=/root/testfile bs=1M count=1000 oflag=direct
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 7.87863 s, 133 MB/s
~# dd if=/dev/zero of=/root/testfile bs=4k count=10000 oflag=direct
10000+0 records in
10000+0 records out
40960000 bytes (41 MB, 39 MiB) copied, 1.713 s, 23.9 MB/s

 

~# fio -filename=/root/test_randwrite -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=4k -size=10M -numjobs=10 -runtime=10 -group_reporting -name=mytest
mytest: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.12
Starting 10 threads
mytest: Laying out IO file (1 file / 10MiB)
Jobs: 10 (f=10): [w(10)][100.0%][w=156KiB/s][w=39 IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=10): err= 0: pid=15239: Sun Jul 25 18:54:02 2021
write: IOPS=49, BW=198KiB/s (203kB/s)(2040KiB/10292msec); 0 zone resets
clat (usec): min=193, max=10131k, avg=199604.88, stdev=1234842.93
lat (usec): min=193, max=10131k, avg=199605.71, stdev=1234842.92
clat percentiles (usec):
| 1.00th=[ 200], 5.00th=[ 215], 10.00th=[ 233],
| 20.00th=[ 269], 30.00th=[ 310], 40.00th=[ 359],
| 50.00th=[ 545], 60.00th=[ 8586], 70.00th=[ 13173],
| 80.00th=[ 27919], 90.00th=[ 76022], 95.00th=[ 92799],
| 99.00th=[ 9328133], 99.50th=[10133439], 99.90th=[10133439],
| 99.95th=[10133439], 99.99th=[10133439]
bw ( KiB/s): min= 8, max= 504, per=69.61%, avg=137.83, stdev=132.79, samples=29
iops : min= 2, max= 126, avg=34.38, stdev=33.19, samples=29
lat (usec) : 250=16.47%, 500=33.33%, 750=2.16%, 1000=0.20%
lat (msec) : 2=0.59%, 4=0.59%, 10=14.51%, 20=7.25%, 50=7.45%
lat (msec) : 100=12.94%, 250=0.98%, 500=0.20%, 750=0.39%, 1000=0.39%
cpu : usr=0.02%, sys=0.04%, ctx=1035, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,510,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=198KiB/s (203kB/s), 198KiB/s-198KiB/s (203kB/s-203kB/s), io=2040KiB (2089kB), run=10292-10292msec
Disk stats (read/write):
sda: ios=0/512, merge=0/42, ticks=0/14112, in_queue=6744, util=63.97%
~# uname -a
Linux WebVPN-Master-Fiber 4.19.0-9-amd64 #1 SMP Debian 4.19.118-2 (2020-04-29) x86_64 GNU/Linux