【verbs】ibv_reg_mr()-程序员宅基地

描述

ibv_reg_mr() 注册一个与保护域(pd)关联的内存区域 (MR)。通过这样做，允许 RDMA 设备向该内存读取和写入数据。执行此注册需要一些时间，因此当需要快速响应时，不建议在数据路径中执行内存注册。
每次成功注册都会产生一个具有唯一(在特定 RDMA 设备内)lkey 和 rkey 值的 MR。
MR 的起始地址是 addr，它的大小是 length。可以注册的块的最大大小限制为 device_attr.max_mr_size。调用它的进程空间中每个的虚拟内存地址都可以注册，包括但不限于：
本地内存(变量或数组)
全局内存(变量或数组)
动态分配的内存(使用 malloc() 或 mmap())
共享内存

来自text segment的地址

注册的内存缓冲区不必是页面对齐的。没有任何方法可以知道可以为特定设备注册的内存总大小是多少。
参数 access 描述了 RDMA 设备所需的内存访问属性。它是 0 或以下一个或多个标志的按位或：

V_ACCESS_LOCAL_WRITE	Enable Local Write Access: Memory Region can be used in Receive Requests or in IBV_WR_ATOMIC_CMP_AND_SWP orIBV_WR_ATOMIC_FETCH_AND_ADD to write locally the remote content value
IBV_ACCESS_REMOTE_WRITE	Enable Remote Write Access: Memory Region can be access from remote context using IBV_WR_RDMA_WRITE orIBV_WR_RDMA_WRITE_WITH_IMM
IBV_ACCESS_REMOTE_READ	Enable Remote Read Access: Memory Region can be access from remote context using IBV_WR_RDMA_READ
IBV_ACCESS_REMOTE_ATOMIC	Enable Remote Atomic Operation Access (if supported): Memory Region can be access from remote context using IBV_WR_ATOMIC_CMP_AND_SWP orIBV_WR_ATOMIC_FETCH_AND_ADD
IBV_ACCESS_MW_BIND	Enable Memory Window Binding

如果设置了 IBV_ACCESS_REMOTE_WRITE 或 IBV_ACCESS_REMOTE_ATOMIC，则也必须设置 IBV_ACCESS_LOCAL_WRITE，因为只有在允许本地写入时才允许远程写入。
MR 始终启用本地读取访问，即可以使用 IBV_WR_SEND、IBV_WR_SEND_WITH_IMM、IBV_WR_RDMA_WRITE、IBV_WR_RDMA_WRITE_WITH_IMM 在本地读取内存区域。
内存注册请求的权限可以是该内存块的操作系统权限的全部或子集。例如：只读内存不能注册有写权限(本地或远程)。
一个特定的进程可以注册一个或多个内存区域。

参数

Name	Direction	Description
pd	in	Protection Domain that was returned from ibv_alloc_pd()
addr	in	The start address of the virtual contiguous memory block
length	in	Size of the memory block to register, in bytes. This value must be at least 1 and less than dev_cap.max_mr_size
access	in	Requested access permissions for the memory region

返回值

Value

Description

MR

Pointer to the newly allocated Memory Region.
This pointer also contains the following fields:

lkey	The value that will be used to refer to this MR using a local access
rkey	The value that will be used to refer to this MR using a remote access

Those values may be equal, but this isn't always guaranteed.

NULL

On failure, errno indicates the failure reason:

EINVAL	Invalid access value
ENOMEM	Not enough resources (either in operating system or in RDMA device) to complete this operation

Examples

Register a MR to allow only local read and write access and deregister it:

struct ibv_pd *pd;
struct ibv_mr *mr;
 
mr = ibv_reg_mr(pd, buf, size, IBV_ACCESS_LOCAL_WRITE);
if (!mr) {
	fprintf(stderr, "Error, ibv_reg_mr() failed\n");
	return -1;
}
 
if (ibv_dereg_mr(mr)) {
	fprintf(stderr, "Error, ibv_dereg_mr() failed\n");
	return -1;
}

Register a MR to allow Remote read and write to it:

mr = ibv_reg_mr(pd, buf, size, IBV_ACCESS_LOCAL_WRITE |
		IBV_ACCESS_REMOTE_WRITE | IBV_ACCESS_REMOTE_READ);
if (!mr) {
	fprintf(stderr, "Error, ibv_reg_mr() failed\n");
	return -1;
}

原文：https://www.rdmamojo.com/2012/09/07/ibv_reg_mr/