[Devel] [PATCH RH7] fs/fuse kio: adjust rdma connection parameters

[Kui Liu]

 I believe 0 values mean no retry based on test. With both values as 0,  It kept receiving IBV_WC_RETRY_EXC_ERR for RDMA READ operations, which aborts the connection. Changing to 7 (which I find to be commonly used value in several examples from RDMA Core library), the error goes away and we can at least have stable connection in such RoCE network. 


-----Original Message-----
From: Alexey Kuznetsov <kuznet at acronis.com <mailto:kuznet at acronis.com>>
Date: Thursday, 17 August 2023 at 11:52 PM
To: Kui Liu <Kui.Liu at acronis.com <mailto:Kui.Liu at acronis.com>>
Cc: Devel <devel at openvz.org <mailto:devel at openvz.org>>
Subject: Re: [Devel] [PATCH RH7] fs/fuse kio: adjust rdma connection parameters


Ack.


What 0 values did mean? No retry or some default value?


On Thu, Aug 17, 2023 at 11:47 PM Kui Liu <Kui.Liu at acronis.com <mailto:Kui.Liu at acronis.com>> wrote:
>
>
>
> In RoCE network, packet loss and dealy due to congestion can happen
>
> quite often. We need to tolerate such event. So increase retry_count
>
> and rnr_retry_count to 7 to allow NIC to retry operations when an
>
> error happens, instead of returning the error directly which causes
>
> the connection to be aborted.
>
>
>
> Signed-off-by: Liu Kui <Kui.Liu at acronis.com <mailto:Kui.Liu at acronis.com>>
>
> ---
>
> fs/fuse/kio/pcs/pcs_rdma_conn.c | 4 ++--
>
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
>
>
> diff --git a/fs/fuse/kio/pcs/pcs_rdma_conn.c b/fs/fuse/kio/pcs/pcs_rdma_conn.c
>
> index 4db903151de0..7339b1466d3a 100644
>
> --- a/fs/fuse/kio/pcs/pcs_rdma_conn.c
>
> +++ b/fs/fuse/kio/pcs/pcs_rdma_conn.c
>
> @@ -44,8 +44,8 @@ conn_param_init(struct rdma_conn_param *cp, struct pcs_rdmaio_conn_req *cr,
>
> cp->initiator_depth = min_t(int, U8_MAX, cmid->device->attrs.max_qp_init_rd_atom);
>
>
>
> cp->flow_control = 1; /* does not matter */
>
> - cp->retry_count = 0; /* # retransmissions when no ACK received */
>
> - cp->rnr_retry_count = 0; /* # RNR retransmissions */
>
> + cp->retry_count = 7; /* # retransmissions when no ACK received */
>
> + cp->rnr_retry_count = 7; /* # RNR retransmissions */
>
> }
>
>
>
> static int pcs_rdma_cm_event_handler(struct rdma_cm_id *cmid,
>
> --
>
> 2.32.0 (Apple Git-132)
>
> _______________________________________________
> Devel mailing list
> Devel at openvz.org <mailto:Devel at openvz.org>
> https://lists.openvz.org/mailman/listinfo/devel <https://lists.openvz.org/mailman/listinfo/devel>