Performance/Cost Ratio for Silicon Storage Media
Apr 07, 2022
May 03, 2022
6
minutes
Contents
Disk performances between SSD, USB and SDcard. Typically done for raspberry pi and usb Linux.
TL;DR
Considering the overall cost/performance ratio, we recommend using USB 3.1 based USB drives to run the operating systems without breaking your wallet.
Introduction
What we are using for our benchmarking measurements is a tool called fio. Debian based platforms can install it using apt install fio. For other distros, you can find the prebuilt binaries on the official repo at https://github.com/axboe/fio.
Also note that we are using few presets in advance to our little test, you can research more about why we are using that here
SD Card Benchmarks
test-sdcard: (g=0): rw=write, bs=(R) 2300B-2300B, (W) 2300B-2300B, (T) 2300B-2300B, ioengine=sync, iodepth=1
fio-3.25
Starting 1 process
test-sdcard: (groupid=0, jobs=1): err= 0: pid=784: Tue May 3 15:24:58 2022
write: IOPS=633, BW=1422KiB/s (1456kB/s)(21.0MiB/15843msec); 0 zone resets
clat (usec): min=12, max=220209, avg=272.13, stdev=2233.24
lat (usec): min=12, max=220212, avg=272.86, stdev=2233.27
clat percentiles (usec):
| 1.00th=[ 14], 5.00th=[ 16], 10.00th=[ 16], 20.00th=[ 16],
| 30.00th=[ 27], 40.00th=[ 30], 50.00th=[ 396], 60.00th=[ 404],
| 70.00th=[ 408], 80.00th=[ 420], 90.00th=[ 437], 95.00th=[ 453],
| 99.00th=[ 537], 99.50th=[ 562], 99.90th=[ 611], 99.95th=[ 1090],
| 99.99th=[26346]
bw ( KiB/s): min= 4, max= 1572, per=100.00%, avg=1426.65, stdev=318.43, samples=31
iops : min= 2, max= 700, avg=635.42, stdev=141.78, samples=31
lat (usec) : 20=24.80%, 50=18.94%, 100=0.09%, 500=54.25%, 750=1.86%
lat (msec) : 2=0.02%, 4=0.01%, 50=0.02%, 250=0.01%
fsync/fdatasync/sync_file_range:
sync (usec): min=758, max=411420, avg=1298.35, stdev=4484.25
sync percentiles (usec):
| 1.00th=[ 783], 5.00th=[ 799], 10.00th=[ 799], 20.00th=[ 807],
| 30.00th=[ 832], 40.00th=[ 840], 50.00th=[ 865], 60.00th=[ 1352],
| 70.00th=[ 1385], 80.00th=[ 1401], 90.00th=[ 1483], 95.00th=[ 1958],
| 99.00th=[ 5276], 99.50th=[ 5342], 99.90th=[ 11994], 99.95th=[ 12649],
| 99.99th=[130548]
cpu : usr=1.19%, sys=8.02%, ctx=31377, majf=0, minf=32
IO depths : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,10029,0,0 short=10029,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=1422KiB/s (1456kB/s), 1422KiB/s-1422KiB/s (1456kB/s-1456kB/s), io=21.0MiB (23.1MB), run=15843-15843msec
Disk stats (read/write):
mmcblk0: ios=5561/10120, merge=0/44, ticks=2264/70639, in_queue=72902, util=99.51%
USB Drive Benchmarks
test-usb: (g=0): rw=write, bs=(R) 2300B-2300B, (W) 2300B-2300B, (T) 2300B-2300B, ioengine=sync, iodepth=1
fio-3.25
Starting 1 process
test-usb: Laying out IO file (1 file / 22MiB)
test-usb: (groupid=0, jobs=1): err= 0: pid=758: Tue May 3 13:51:08 2022
write: IOPS=456, BW=1026KiB/s (1051kB/s)(21.0MiB/21952msec); 0 zone resets
clat (usec): min=6, max=11446, avg=34.31, stdev=134.58
lat (usec): min=7, max=11446, avg=34.97, stdev=134.59
clat percentiles (usec):
| 1.00th=[ 8], 5.00th=[ 15], 10.00th=[ 15], 20.00th=[ 17],
| 30.00th=[ 24], 40.00th=[ 31], 50.00th=[ 33], 60.00th=[ 35],
| 70.00th=[ 36], 80.00th=[ 41], 90.00th=[ 50], 95.00th=[ 55],
| 99.00th=[ 60], 99.50th=[ 67], 99.90th=[ 143], 99.95th=[ 1926],
| 99.99th=[ 4686]
bw ( KiB/s): min= 718, max= 1154, per=100.00%, avg=1027.67, stdev=72.01, samples=43
iops : min= 320, max= 514, avg=457.72, stdev=32.01, samples=43
lat (usec) : 10=3.02%, 20=23.42%, 50=64.98%, 100=8.36%, 250=0.14%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.03%, 4=0.02%, 10=0.01%, 20=0.01%
fsync/fdatasync/sync_file_range:
sync (usec): min=853, max=10840, avg=2146.64, stdev=1136.83
sync percentiles (usec):
| 1.00th=[ 873], 5.00th=[ 889], 10.00th=[ 938], 20.00th=[ 963],
| 30.00th=[ 971], 40.00th=[ 996], 50.00th=[ 2704], 60.00th=[ 2966],
| 70.00th=[ 2999], 80.00th=[ 3032], 90.00th=[ 3097], 95.00th=[ 3130],
| 99.00th=[ 4817], 99.50th=[ 5211], 99.90th=[ 6783], 99.95th=[8225],
| 99.99th=[10552]
cpu : usr=0.61%, sys=6.40%, ctx=21106, majf=0, minf=28
IO depths : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,10029,0,0 short=10029,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=1026KiB/s (1051kB/s), 1026KiB/s-1026KiB/s (1051kB/s-1051kB/s), io=21.0MiB (23.1MB), run=21952-21952msec
Disk stats (read/write):
sda: ios=0/21489, merge=0/11280, ticks=0/19990, in_queue=19990, util=99.68%
SSD Benchmarks
test-ssd: (g=0): rw=write, bs=(R) 2300B-2300B, (W) 2300B-2300B, (T) 2300B-2300B, ioengine=sync, iodepth=1
fio-3.25
Starting 1 process
test-ssd: (groupid=0, jobs=1): err= 0: pid=921: Tue May 3 13:45:06 2022
write: IOPS=1410, BW=3168KiB/s (3244kB/s)(21.0MiB/7111msec); 0 zone resets
clat (usec): min=6, max=612, avg=150.36, stdev=124.81
lat (usec): min=7, max=613, avg=150.77, stdev=124.81
clat percentiles (usec):
| 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 8], 20.00th=[ 8],
| 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 235], 60.00th=[ 245],
| 70.00th=[ 251], 80.00th=[ 265], 90.00th=[ 285], 95.00th=[ 293],
| 99.00th=[ 306], 99.50th=[ 314], 99.90th=[ 334], 99.95th=[ 343],
| 99.99th=[ 433]
bw ( KiB/s): min= 2924, max= 3333, per=100.00%, avg=3173.50, stdev=153.29, samples=14
iops : min= 1302, max= 1484, avg=1413.14, stdev=68.26, samples=14
lat (usec) : 10=24.40%, 20=16.51%, 50=2.91%, 100=0.01%, 250=24.60%
lat (usec) : 500=31.56%, 750=0.01%
fsync/fdatasync/sync_file_range:
sync (usec): min=454, max=8992, avg=553.49, stdev=161.68
sync percentiles (usec):
| 1.00th=[ 469], 5.00th=[ 494], 10.00th=[ 494], 20.00th=[ 498],
| 30.00th=[ 498], 40.00th=[ 502], 50.00th=[ 510], 60.00th=[ 529],
| 70.00th=[ 562], 80.00th=[ 578], 90.00th=[ 635], 95.00th=[ 832],
| 99.00th=[ 906], 99.50th=[ 930], 99.90th=[ 947], 99.95th=[ 1057],
| 99.99th=[ 8848]
cpu : usr=1.34%, sys=15.08%, ctx=41364, majf=0, minf=29
IO depths : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,10029,0,0 short=10029,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=3168KiB/s (3244kB/s), 3168KiB/s-3168KiB/s (3244kB/s-3244kB/s), io=21.0MiB (23.1MB), run=7111-7111msec
Disk stats (read/write):
sda: ios=5455/19426, merge=0/2, ticks=1253/4964, in_queue=9749, util=98.68%
