DAT_EP_POST_RDMA_READ(3DAT)DAT_EP_POST_RDMA_READ(3DAT)NAME
dat_ep_post_rdma_read - transfer all data to the local data buffer
SYNOPSIS
cc [ flag... ] file... -ldat [ library... ]
#include <dat/udat.h>
DAT_RETURN
dat_ep_post_rdma_read (
IN DAT_EP_HANDLE ep_handle,
IN DAT_COUNT num_segments,
IN DAT_LMR_TRIPLET *local_iov,
IN DAT_DTO_COOKIE user_cookie,
IN DAT_RMR_TRIPLET *remote_buffer,
IN DAT_COMPLETION_FLAGS completion_flags
)
PARAMETERS
ep_handle
Handle for an instance of the Endpoint.
num_segments
Number of lmr_triplets in local_iov.
local_iov
I/O Vector that specifies the local buffer to fill.
user_cookie
User-provided cookie that is returned to the Con‐
sumer at the completion of the RDMA Read. Can be
NULL.
remote_buffer
A pointer to an RMR Triplet that specifies the
remote buffer from which the data is read.
completion_flags
Flags for posted RDMA Read. The default DAT_COMPLE‐
TION_DEFAULT_FLAG is 0x00. Other values are as fol‐
lows:
Completion Suppression
DAT_COMPLETION_SUP‐
PRESS_FLAG
0x01
Suppress suc‐
cessful Com‐
pletion.
Notification of Completion
DAT_COMPLE‐
TION_UNSIGNALLED_FLAG
0x04
Non-notifica‐
tion comple‐
tion. Local
Endpoint must
be configured
for Notifica‐
tion Suppres‐
sion.
Barrier Fence
DAT_COMPLETION_BAR‐
RIER_FENCE_FLAG
0x08
Request for
Barrier
Fence.
DESCRIPTION
The dat_ep_post_rdma_read() function requests the transfer of all the
data specified by the remote_buffer over the connection of the ep_han‐
dle Endpoint into the local_iov.
The num_segments parameter specifies the number of segments in the
local_iov. The local_iov segments are filled in the I/O Vector order
until the whole message is received. This ensures that all the "front"
segments of the local_iov I/O Vector are completely filled, only one
segment is partially filled, if needed, and all segments that follow it
are not filled at all.
The user_cookie allows Consumers to have unique identifiers for each
DTO. These identifiers are completely under user control and are
opaque to the Provider. There is no requirement on the Consumer that
the value user_cookie should be unique for each DTO. The user_cookie is
returned to the Consumer in the Completion event for the posted RDMA
Read.
A Consumer must not modify the local_iov or its content until the DTO
is completed. When a Consumer does not adhere to this rule, the behav‐
ior of the Provider and the underlying Transport is not defined.
Providers that allow Consumers to get ownership of the local_iov but
not the memory it specifies back after the dat_ep_post_rdma_read()
returns should document this behavior and also specify its support in
Provider attributes. This behavior allows Consumers full control of the
local_iov after dat_ep_post_rdma_read() returns. Because this behavior
is not guaranteed by all Providers, portable Consumers should not rely
on this behavior. Consumers should not rely on the Provider copying
local_iov information.
The completion of the posted RDMA Read is reported to the Consumer
asynchronously through a DTO Completion event based on the specified
completion_flags value. The value of DAT_COMPLETION_UNSIGNALLED_FLAG is
only valid if the Endpoint Request Completion Flags DAT_COMPLE‐
TION_UNSIGNALLED_FLAG. Otherwise, DAT_INVALID_PARAMETER is returned.
The DAT_SUCCESS return of the dat_ep_post_rdma_read() is at least the
equivalent of posting an RDMA Read operation directly by native Trans‐
port. Providers should avoid resource allocation as part of
dat_ep_post_rdma_read() to ensure that this operation is nonblocking
and thread safe for an UpCall.
The operation is valid for the Endpoint in the DAT_EP_STATE_CONNECTED
and DAT_EP_STATE_DISCONNECTED states. If the operation returns success‐
fully for the Endpoint in the DAT_EP_STATE_DISCONNECTED state, the
posted RDMA Read is immediately flushed to request_evd_handle.
RETURN VALUES
DAT_SUCCESS
The operation was successful.
DAT_INSUFFICIENT_RESOURCES
The operation failed due to resource lim‐
itations.
DAT_INVALID_PARAMETER
Invalid parameter. For example, one of
the IOV segments pointed to a memory out‐
side its LMR.
DAT_INVALID_HANDLE
The ep_handle parameter is invalid.
DAT_INVALID_STATE
A parameter is in an invalid state. End‐
point was not in the DAT_EP_STATE_CON‐
NECTED or DAT_EP_STATE_DISCONNECTED
state.
DAT_LENGTH_ERROR
The size of the receiving buffer is too
small for sending buffer data. The size
of the local buffer is too small for the
data of the remote buffer.
DAT_PROTECTION_VIOLATION
Protection violation for local or remote
memory access. Protection Zone mismatch
between either an LMR of one of the
local_iov segments and the local Endpoint
or the rmr_context and the remote End‐
point.
DAT_PRIVILEGES_VIOLATION
Privileges violation for local or remote
memory access. Either one of the LMRs
used in local_iov is invalid or does not
have the local write privileges, or
rmr_context does not have the remote read
privileges.
USAGE
For best RDMA Read operation performance, the Consumer should align
each buffer segment of local_iov to the Optimal Buffer Alignment
attribute of the Provider. For portable applications, the Consumer
should align each buffer segment of local_iov to the DAT_OPTIMAL_ALIGN‐
MENT.
If connection was established without outstanding RDMA Read attributes
matching on Endpoints on both sides (outstanding RDMA Read outgoing on
one end is larger than the outstanding RDMA Read incoming on the other
end), connection is broken when the number of incoming RDMA Read
exceeds the outstanding RDMA Read incoming attribute of the Endpoint.
The Consumer can use its own flow control to ensure that it does not
post more RDMA Reads then the remote EP outstanding RDMA Read incoming
attribute is. Thus, they do not rely on the underlying Transport
enforcing it.
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
┌────────────────────┬───────────────────────────┐
│ ATTRIBUTE TYPE │ ATTRIBUTE VALUE │
├────────────────────┼───────────────────────────┤
│Interface Stability │ Standard: uDAPL, 1.1, 1.2 │
├────────────────────┼───────────────────────────┤
│MT-Level │ Unsafe │
└────────────────────┴───────────────────────────┘
SEE ALSOlibdat(3LIB), attributes(5)
Jul 16, 2004 DAT_EP_POST_RDMA_READ(3DAT)