基础操作 [root@iZ116haf49sZ bin]# ll /usr/local/bin/fio -rwxr-xr-x 1 root root 3113575 Nov 12 21:48 /usr/local/bin/fio [root@iZ116haf49sZ bin]# /usr/local/bin/fio -h fio-2.1.10 /usr/local/bin/fio [options] [job options] <job file(s)> --debug=options Enable
安装 [root@iZ116haf49sZ fio]# less README [root@iZ116haf49sZ fio]# ./configure Operating system Linux CPU x86_64 Big endian no Compiler gcc Cross compile no Sta
在Linux环境中，了解存储/磁盘I/O性能对于评估系统性能和优化存储子系统非常重要。通过测试存储/磁盘I/O性能，我们可以确定磁盘的读写速度、延迟和吞吐量等指标。本文将介绍几种常用的方法来测试Linux机器中的存储/磁盘I/O性能。
磁盘压测工具理论上都有损坏文件系统的可能，如果盘里有数据，压测前一定先做快照，压测完毕后回滚快照还原回去，确保不因压测磁盘丢数据。
前言 fio 是一款非常经典的开源磁盘io测试工具 fio is an I/O tool meant to be used both for benchmark and stress/hardware verification. It has support for 19 different types of I/O engines (sync, mmap, libaio, posixaio, SG v3, splice, null, network, syslet, guasi, solarisaio,
项目需要使用的主板有很多性能需要经过测试之后才能用于开发使用，因此将Linux上一些常用的tools移植进板子进行测试。
Fio（Flexible I/O Tester） 是一款由 Jens Axboe 开发的用于测评和压力/硬件验证的自由开源的软件。它支持 19 种不同类型的 I/O 引擎 (sync、mmap、libaio、posixaio、SG v3、splice、null、network、 syslet、guasi、solarisaio，以及更多)， I/O 优先级（针对较新的 Linux 内核），I/O 速度，fork 的任务或线程任务等等。它能够在块设备和文件上工作。
是否使用 direct io，测试过程不使用OS 自带的buffer，使测试磁盘的结果更真实。Linux读写的时候，内核维护了缓存，数据先写到缓存，后面在后台写到SSD。读的时候也优先读缓存里的数据。这样速度可以加快，但是一旦掉电，缓存里的数据就没有了。所以有一种模式叫做direct io，跳过缓存，直接读写SSD。
前面系列已经讲完了硬件选型、部署、调优，在上线之前呢需要进行性能存储测试，本章主要讲述下测试Ceph的几种常用工具，以及测试方法。
https://www.cs.nmsu.edu/~pfeiffer/fuse-tutorial/
有关Windows磁盘性能压测，笔者还是强烈推荐使用微软自己开源的压测工具DiskSpd。当然，如果要使用其他磁盘性能压测工具也是可以的，比如：IOMeter（老牌经典）、FIO（更适合Linux）等。
FIO是测试IOPS的非常好的工具，用来对硬件进行压力测试和验证。磁盘IO是检查磁盘性能的重要指标，可以按照负载情况分成照顺序读写，随机读写两大类。
Fio（Flexible I/O Tester） 是一款由 Jens Axboe 开发的用于测评和压力/硬件验证的自由开源的软件。
1.缓冲 I/O，是指利用标准库缓存来加速文件的访问，而标准库内部再通过系统调度访问文件。
通过讲解如何优雅扩容云硬盘，我们了解了云盘连接到服务器上的具体操作过程。那么，如何进一步了解已挂载硬盘的实际性能呢？你或许会疑惑，测试硬盘性能，为什么不能用Linux系统自带的dd工具呢？而且不少人之前都这么用的：
参考博客： http://lilinji.blog.51cto.com/5441000/1569623
介绍：FIO是测试IOPS的非常好的工具，用来对硬件进行压力测试和验证，支持多种不同的I/O引擎，包括:sync,mmap, libaio, posixaio, SG v3, splice, null, network, syslet, guasi, solarisaio 等等
目前主流的第三方IO测试工具有fio、iometer和Orion，这三种工具各有千秋。
randrepeat=boolForrandomIOworkloads,seedthegeneratorinapredictablewaysothatresultsarerepeatableacrossrepetitions.Defaultstotrue.randseed=intSeedtherandomnumbergeneratorsbasedonthisseedvalue,tobeabletocontrolwhatsequenceofoutputisbeinggenerated.Ifnotset,therandomsequencedependsontherandrepeatsetting.fallocate=strWhetherpre-allocationisperformedwhenlayingdownfiles.Acceptedvaluesare:noneDonotpre-allocatespaceposixPre-allocateviaposix_fallocate()keepPre-allocateviafallocate()withFALLOC_FL_KEEP_SIZEset0Backward-compatiblealiasfor'none'1Backward-compatiblealiasfor'posix'Maynotbeavailableonallsupportedplatforms.'keep'isonlyavailableonLinux.IfusingZFSonSolaristhismustbesetto'none'becauseZFSdoesn't support it. Default: 'posix'. fadvise_hint=bool By default, fio will use fadvise() to advise the kernel on what IO patterns it is likely to issue. Sometimes you want to test specific IO patterns without telling the kernel about it, in which case you can disable this option. If set, fio will use POSIX_FADV_SEQUENTIAL for sequential IO and POSIX_FADV_RANDOM for random IO. fadvise_stream=int Notify the kernel what write stream ID to place these writes under. Only supported on Linux. Note, this option may change going forward. size=int The total size of file io for this job. Fio will run until this many bytes has been transferred, unless runtime is limited by other options (such as 'runtime', for instance, or increased/decreased by 'io_size'). Unless specific nrfiles and filesize options are given, fio will divide this size between the available files specified by the job. If not set, fio will use the full size of the given files or devices. If the files do not exist, size must be given. It is also possible to give size as a percentage between 1 and 100. If size=20% is given, fio will use 20% of the full size of the given files or devices. io_size=int io_limit=int Normally fio operates within the region set by 'size', which means that the 'size' option sets both the region and size of IO to
云硬盘是一种高可用、高可靠、低成本、可定制化的网络块存储，可作为云服务器的独立可扩展硬盘使用。它提供数据块级别的数据存储，采用三副本的分布式机制，为云服务器提供数据可靠性保证。云硬盘提供以下 SSD 云硬盘、高性能云硬盘及普通云硬盘三种云硬盘类型，不同的硬盘类型、性能、特点和价格均不同。
我能找到的唯一方法是fio-status实用程序,但它的目的是输出人类可读的文本,而不是机器可解析的文本.我可以刮它,但那很脏.
上篇文章 「什么？相同型号物理机 容器性能不如虚拟机？」 ，给我们的经验教训，就是上线前，基准测试的重要性，这篇文章着重介绍一下「Linux 性能基准测试工具及测试方法」
sync_file_range=str:valUsesync_file_range()forevery'val'numberofwriteoperations.Fiowilltrackrangeofwritesthathavehappenedsincethelastsync_file_range()call.'str'cancurrentlybeoneormoreof:wait_beforeSYNC_FILE_RANGE_WAIT_BEFOREwriteSYNC_FILE_RANGE_WRITEwait_afterSYNC_FILE_RANGE_WAIT_AFTERSoifyoudosync_file_range=wait_before,write:8,fiowoulduseSYNC_FILE_RANGE_WAIT_BEFORE|SYNC_FILE_RANGE_WRITEforevery8writes.Alsoseethesync_file_range(2)manpage.ThisoptionisLinuxspecific.overwrite=boolIftrue,writestoafilewillalwaysoverwriteexistingdata.Ifthefiledoesn't already exist, it will be created before the write phase begins. If the file exists and is large enough for the specified write phase, nothing will be done. end_fsync=bool If true, fsync file contents when a write stage has completed. fsync_on_close=bool If true, fio will fsync() a dirty file on close. This differs from end_fsync in that it will happen on every file close, not just at the end of the job. rwmixread=int How large a percentage of the mix should be reads. rwmixwrite=int How large a percentage of the mix should be writes. If both rwmixread and rwmixwrite is given and the values do not add up to 100%, the latter of the two will be used to override the first. This may interfere with a given rate setting, if fio is asked to limit reads or writes to a certain rate. If that is the case, then the distribution may be skewed. random_distribution=str:float By default, fio will use a completely uniform random distribution when asked to perform random IO. Sometimes it is useful to skew the distribution in specific ways, ensuring that some parts of the data is more hot than others. fio includes the following distribution models: random Uniform random distribution zipf Zipf distribution pareto Pareto distribution When using a zipf or pareto distribution, an input value is also needed to define the access pattern. For zipf, this is the zipf theta. For pareto, it'stheparetopower.Fioincludesatestprogram,genzipf,thatcanbeusedvi
马上年底了，各种云评测陆续放了出来，最近看到有一些评测也引起了争议，第三方评测数据可以作为参考，真正要使用云，将业务放到云上，还是要自己来做一些评测，一方面自己跑的数据可信，一方面自己最了解业务需求，知道测试的时候应该重点关注那些指标。
前言： 简单看了一下glusterfs，使用单节点构造glusterfs环境，导出的路径是是本地SSD在分区上。用qemu挂载glusterfs上的卷，用FIO测试IOPS，测试结果不理想。 大致分析了一下，怀疑fuse会导致性能下降。 分析： 1，libfuse & fuse 为了方便测试和便于分析问题，使用了libfuse。代码地址https://github.com/libfuse/libfuse 编译libfuse比较麻烦，不支持Makefile，需要用meson编译，而且meson的版本要求比较高，不能用apt-get直接安装。操作方法就是下载高版本的meson包，在meson包里面执行python3 setup.py install。 除了用户态的libfuse之外，还需要kernel支持。作者在Ubuntu1804上测试，fuse已经被编译到kernel中。在config文件（内核配置文件即ls /boot/config-`uname -r`）中CONFIG_FUSE_FS。如果是kmod的方式编译，执行modprobe fuse。
本文介绍了作者常用的 4 个 Linux 监控工具，希望可以帮助读者提高生产力。
广义上Cache的同步方式有两种，即Write Through（写穿）和Write back（写回）. 从名字上就能看出这两种方式都是从写操作的不同处理方式引出的概念（纯读的话就不存在Cache一致性了，不是么）。对应到Linux的Page Cache上所谓Write Through就是指write(2)操作将数据拷贝到Page Cache后立即和下层进行同步的写操作，完成下层的更新后才返回。而Write back正好相反，指的是写完Page Cache就可以返回了。Page Cache到下层的更新操作是异步进行的。
对每个人而言，真正的职责只有一个：找到自我。然后在心中坚守其一生，全心全意，永不停息。所有其它的路都是不完整的，是人的逃避方式，是对大众理想的懦弱回归，是随波逐流，是对内心的恐惧 ——赫尔曼·黑塞《德米安》
身为一个运维开发人员，如果你不知道眼下当前服务器底层操作系统中正在发生什么，那就有点合眼摸象了。其实，你可以根据相应数据做出一定的推测，但是要做到这一点，就需要原始数据，并且数据要有一定的实时性。
前言 其实这个专题很久很久之前就想写了，但是一直因为各种原因拖着没动笔。 因为没有资格，也没有钱在一线城市买房 (😂😂😂); 但是在要结婚之前，婚房又是刚需。我和太太最终一起在一线城市周边的某二线城市买了房。再之后，一起装修，她负责非电相关，我负责电 网相关的装修。家庭组网，家庭实验室就这么一步一步随着家庭的组建而组建了起来： 1.家庭有线无线组网2.智能家居3.NAS4.公网 IP 和 IPv65.Wake Online (WOL)6.家庭网络安全 (😂看了防火墙日志，才知道被攻击频率能有多高）7.玩转
FIO是测试IOPS的非常好的工具，用来对硬件进行压力测试和验证，支持13种不同的I/O引擎，包括:sync,mmap, libaio, posixaio, SG v3, splice, null, network, syslet, guasi, solarisaio 等等。他可以通过多线程或进程模拟各种io操作。
块是文件系统的抽象，而非磁盘的属性，一般是 Sector Size 的倍数；扇区大小则是磁盘的物理属性，它是磁盘设备寻址的最小单元。另外，内核中要求 Block_Size = Sector_Size * (2的n次方)，且 Block_Size <= 内存的 Page_Size (页大小)。
Ceph，作为一个高度可扩展的分布式存储系统，已经成为云计算和大数据时代的关键基石。随着企业和组织对数据存储的需求日益增长，Ceph 通过其强大的特性，如可靠性、伸缩性和性能，满足了这些需求。然而，随着集群规模的扩大和工作负载的多样性，如何确保资源的有效分配和性能隔离成为了一个重要议题。在这个背景下，Ceph 的 Quality of Service (QoS) 功能显得尤为重要。
bs_is_seq_randIfthisoptionisset,fiowillusethenormalread,writeblocksizesettingsassequential,randominstead.AnyrandomreadorwritewillusetheWRITEblocksizesettings,andanysequentialreadorwritewillusetheREADblocksizesetting.zero_buffersIfthisoptionisgiven,fiowillinittheIObufferstoallzeroes.Thedefaultistofillthemwithrandomdata.refill_buffersIfthisoptionisgiven,fiowillrefilltheIObuffersoneverysubmit.Thedefaultistoonlyfillitatinittimeandreusethatdata.Onlymakessenseifzero_buffersisn't specified, naturally. If data verification is enabled, refill_buffers is also automatically enabled. scramble_buffers=bool If refill_buffers is too costly and the target is using data deduplication, then setting this option will slightly modify the IO buffer contents to defeat normal de-dupe attempts. This is not enough to defeat more clever block compression attempts, but it will stop naive dedupe of blocks. Default: true. buffer_compress_percentage=int If this is set, then fio will attempt to provide IO buffer content (on WRITEs) that compress to the specified level. Fio does this by providing a mix of random data and a fixed pattern. The fixed pattern is either zeroes, or the pattern specified by buffer_pattern. If the pattern option is used, it might skew the compression ratio slightly. Note that this is per block size unit, for file/disk wide compression level that matches this setting, you'llalsowanttosetrefill_buffers.buffer_compress_chunk=intSeebuffer_compress_percentage.Thissettingallowsfiotomanagehowbigtherangesofrandomdataandzeroeddatais.Withoutthisset,fiowillprovidebuffer_compress_percentageofblocksizerandomdata,followedbytheremainingzeroed.Withthissettosomechunksizesmallerthantheblocksize,fiocanalternaterandomandzeroeddatathroughouttheIObuffer.buffer_pattern=strIfset,fiowillfilltheiobufferswiththispattern.Ifnotset,thecontentsofiobuffersisdefinedbytheotheroptionsrelatedtobuffercontents.Thesettingcanbeanypatternofbytes,andcanbeprefixedwith0xforhexvalues.Itmayalsobeastring,wherethestringmustthenbewrappedwit
合成测试程序根据统计的真实负载发生规律，如请求的读写比例，大小，频率和分布等信息。建立响应的io存取模型。在测试时产生符合存取模型的io请求序列。发送给存储系统。这类程序包括 IOMeter,IOZone 和 Bonnie++。
本篇文章是性能篇的最后一篇文章，算是一个学习笔记吧，当中的例子也是从别的文章里面摘录的，主要用来讲解如何使用和查看对应的指标。这一篇主要介绍文件系统，说的更加具体点其实是磁盘这个点。
SLI（Service Level Indicator）：服务等级指标，其实就是我们选择哪些指标来衡量我们的稳定性。
; -- end job file -- Here we have no global section, as we only have one job defined anyway. We want to use async io here, with a depth of4foreach file. We also increased the buffer size used to32KB and define numjobs to4to fork 4 identical jobs. The result is4 processes each randomly writing to their own 64MB file. Instead ofusing the above job file, you could have given the parameters on the command line. For this case, you would specify: $ fio --name=random-writers --ioengine=libaio --iodepth=4 --rw=randwrite --bs=32k --direct=0 --size=64m --numjobs=4When fio is utilized as a basis ofany reasonably large test suite, it might be desirable toshare a setof standardized settings across multiple job files. Instead of copy/pasting such settings, anysection may pull in an external .fio file with'includefilename' directive, asin the following example: ;-- start job file including.fio -- [global] filename=/tmp/test filesize=1m include glob-include.fio [test] rw=randread bs=4k time_based=1 runtime=10 include test-include.fio ; -- end job file including.fio -- ; -- start job file glob-include.fio -- thread=1 group_reporting=1 ; -- end job file glob-include.fio -- ; -- start job file test-include.fio -- ioengine=libaio iodepth=4 ; -- end job file test-include.fio -- Settings pulled into a section apply to that section only (except global section). Include directives may be nested in that any included file may contain further include directive(s). Include files may not contain [] sections. 4.1 Environment variables ------------------------- fio also supports environment variable expansion in job files. Any sub-string of the form "${VARNAME}" as part of an option value (in other words, on the right of the `='),willbeexpandedtothevalueoftheenvironmentvariablecalledVARNAME.Ifnosuchenvironmentvariableisdefined,orVARNAMEistheemptystring,theemptystringwillbesubstituted.Asanexample,let's look at a sample fio invocation and job file: $ SIZE=64m NUMJOBS=4 fio jobfile.fio ; -- start job file -- [random-writers] rw=randwr
可以直接使用了 [root@iZ116haf49sZ fio]# which fio /usr/local/bin/fio [root@iZ116haf49sZ fio]# fio -h fio-2.2.11-15-g236d fio [options] [job options] <job file(s)> --debug=options Enable debug logging. May be one/more of: process,file,io,mem,blktrace,verify
2、fio测试建议在空闲的、未保存重要数据的硬盘上进行，并在测试完后重新制作文件系统。请不要在业务数据硬盘上测试，避免底层文件系统元数据损坏导致数据损坏。
fio是一个适应性非常强的软件，基本上能够模拟所有的IO请求，是目前最全面的一款测试软件，之前在看德国电信的一篇分享的时候，里面就提到了，如果需要测试存储性能，尽量只用一款软件，这样从上层测试到底层去，才能更好的去比较差别
FIO 工具使用：https://www.cnblogs.com/xuyaowen/p/fio-usage.html
rdma The RDMA I/O engine supports both RDMA memory semantics (RDMA_WRITE/RDMA_READ) and channel semantics (Send/Recv) for the InfiniBand, RoCE and iWARP protocols. falloc IO engine that does regular fallocate to simulate data transfer as fio ioengine. DDIR_READ does fallocate(,mode = keep_size,) DDIR_WRITE does fallocate(,mode = 0) DDIR_TRIM does fallocate(,mode = punch_hole) e4defrag IO engine that does regular EXT4_IOC_MOVE_EXT ioctls to simulate defragment activity in request to DDIR_WRITE event rbd IO engine supporting direct access to Ceph Rados Block Devices (RBD) via librbd without the need to use the kernel rbd driver. This ioengine defines engine specific options. gfapi Using Glusterfs libgfapi sync interface to direct access to Glusterfs volumes without options. gfapi_async Using Glusterfs libgfapi async interface to direct access to Glusterfs volumes without having to go through FUSE. This ioengine defines engine specific options. libhdfs Read and write through Hadoop (HDFS). The 'filename' option is used to specify host, port of the hdfs name-node to connect. This engine interprets offsets a little differently. In HDFS, files once created cannot be modified. So random writes are not possible. To imitate this, libhdfs engine expects bunch of small files to be created over HDFS, and engine will randomly pick a file out of those files based on the offset generated by fio backend. (see the example job file to create such files, use rw=write option). Please note, you might want to set necessary environment variables to work with hdfs/libhdfs properly. mtd Read, write and erase an MTD character device (e.g., /dev/mtd0). Discards are treated as erases. Depending on the underlying device type, the I/O may have to go in a certain pattern, e.g., on NAND, writing sequentially to erase blocks and discarding before overwriting. The w
The write state is relatively small, on the order of hundreds of bytes to single kilobytes. It contains information on the number of completions done, the last X completions, etc. A trigger is invoked either through creation ('touch') of a specified file in the system, or through a timeout setting. If fio is run with --trigger-file=/tmp/trigger-file, then it will continually check for the existence of /tmp/trigger-file. When it sees this file, it will fire off the trigger (thus saving state, and executing the trigger command). For client/server runs, there'sbothalocalandremotetrigger.Iffioisrunningasaserverbackend,itwillsendthejobstatesbacktotheclientforsafestorage,thenexecutetheremotetrigger,ifspecified.Ifalocaltriggerisspecified,theserverwillstillsendbackthewritestate,buttheclientwillthenexecutethetrigger.10.1Verificationtriggerexample---------------------------------Letssaywewanttorunapowercuttestontheremotemachine'server'.Ourwriteworkloadisinwrite-test.fio.Wewanttocutpowerto'server'atsomepointduringtherun,andwe'll run this test from the safety or our local machine, 'localbox'. On the server, we'llstartthefiobackendnormally:server#fio--serverandontheclient,we'll fire off the workload: localbox$ fio --client=server --trigger-file=/tmp/my-trigger --trigger-remote="bash -c \"echo b > /proc/sysrq-triger\"" We set /tmp/my-trigger as the trigger file, and we tell fio to execute echo b > /proc/sysrq-trigger on the server once it has received the trigger and sent us the write state. This will work, but it'snot_really_cuttingpowertotheserver,it's merely abruptly rebooting it. If we have a remote way of cutting power to the server through IPMI or similar, we could do that through a local trigger command instead. Lets assume we have a script that does IPMI reboot of a given hostname, ipmi-reboot. On localbox, we could then have run fio with a local trigger instead: localbox$ fio --client=server --trigger-file=/tmp/my-trigger --trigger="ipmi-reboot server" For this case, fio would wait for the server to send us th
附： 下面是关于 fio 的详细用法与各参数解释，在源码包的 HOWTO 文档里 [root@iZ116haf49sZ fio]# cat HOWTO Table of contents ----------------- 1. Overview 2. How fio works 3. Running fio 4. Job file format 5. Detailed list of parameters 6. Normal output 7. Terse output 8. Trace file f
rpmbuild --rebuild iomemory-vsl-3.2.4.1086-1.0.el6.src.rpm
领取专属 10元无门槛券
手把手带您无忧上云
云服务器
ICP备案
对象存储
即时通信 IM
腾讯会议
