实战spdk perf

原创

存储之厨 2018-12-20 22:47:34 博主文章分类：存储相关 ©著作权

文章标签 nvme pcie 文章分类 运维

©著作权归作者所有：来自51CTO博客作者存储之厨的原创作品，谢绝转载，否则将追究法律责任

spdk的perf能够充分展现nvme 的高性能，那么怎么让这个利器发挥作用呢？下面分享了100% 验证的完整步骤：

下载代码并解决编译依赖

SPDK的安装在最近的版本里面已经简化了，可以参考https://github.com/spdk/spdk，会自动把对应的DPDK版本拉下来。

基本步骤：

下载代码 Code

git clone https://github.com/spdk/spdk cd spdk git submodule update --init Prerequisites

The dependencies can be installed automatically by scripts/pkgdep.sh. ./scripts/pkgdep.sh

Build 二进制

./configure make

确定待测试盘的pcie bdv

比如我要测/dev/nvme6n1, 由于perf需要指定这个设备对应的PCIE的BDF地址，那么改怎么确定它呢？ [root@test]# ls -alrt /dev/nvme6n1 brw-rw---- 1 root disk 259, 0 Dec 20 17:26 /dev/nvme6n1 [root@test]# cd/sys/dev/block/259:0/device && pwd /sys/dev/block/259:0/device [root@test]# ls -alrt total 0 -rw-r--r-- 1 root root 4096 Dec 19 23:05 uevent drwxr-xr-x 3 root root 0 Dec 19 23:05 . lrwxrwxrwx 1 root root 0 Dec 19 23:05 subsystem -> ../../../../../../class/nvme drwxr-xr-x 7 root root 0 Dec 19 23:05 nvme6n1 drwxr-xr-x 3 root root 0 Dec 19 23:05 .. lrwxrwxrwx 1 root root 0 Dec 20 21:31 device -> ../../../0000:b1:00.0

加载DPDK huge page

....................... cd /root/spdk-bundle/dpdk/dpdk-16.11/tools ./dpdk_setup.sh ....................... Step 2: Setup linuxapp environment

[16] Insert IGB UIO module [17] Insert VFIO module [18] Insert KNI module [19] Setup hugepage mappings for non-NUMA systems [20] Setup hugepage mappings for NUMA systems [21] Display current Ethernet/Crypto device settings [22] Bind Ethernet/Crypto device to IGB UIO module [23] Bind Ethernet/Crypto device to VFIO module [24] Setup VFIO permissions

Step 3: Run test application for linuxapp environment

[25] Run test application ($RTE_TARGET/app/test) [26] Run testpmd application in interactive mode ($RTE_TARGET/app/testpmd)

Step4: Other tools

[27] List hugepage info from /proc/meminfo

Step 5: Uninstall and system cleanup

[28] Unbind devices from IGB UIO or VFIO driver [29] Remove IGB UIO module [30] Remove VFIO module [31] Remove KNI module [32] Remove hugepage mappings

[33] Exit Script

Option: 20 创建huge page然后退出。

卸载测试盘默认linux 的nvme 驱动

cd /root/spdk-bundle/spdk/spdkv1707/scripts 为了避免卸掉其他可能正在使用的nvme盘的驱动，参考下面改动直接指定： function linux_iter_pci { #Argument is the class code #TODO: More specifically match against only class codes in the grep #step. #lspci -mm -n -D | grep $1 | tr -d '"' | awk -F " " '{print $1}' echo "0000:b1:00.0" } 然后： ./setup.sh