1*4882a593SmuzhiyunWhat: /sys/class/rtrs-client 2*4882a593SmuzhiyunDate: Feb 2020 3*4882a593SmuzhiyunKernelVersion: 5.7 4*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 5*4882a593SmuzhiyunDescription: When a user of RTRS API creates a new session, a directory entry with 6*4882a593Smuzhiyun the name of that session is created under /sys/class/rtrs-client/<session-name>/ 7*4882a593Smuzhiyun 8*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/add_path 9*4882a593SmuzhiyunDate: Feb 2020 10*4882a593SmuzhiyunKernelVersion: 5.7 11*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 12*4882a593SmuzhiyunDescription: RW, adds a new path (connection) to an existing session. Expected format is the 13*4882a593Smuzhiyun following:: 14*4882a593Smuzhiyun 15*4882a593Smuzhiyun <[source addr,]destination addr> 16*4882a593Smuzhiyun *addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ] 17*4882a593Smuzhiyun 18*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/max_reconnect_attempts 19*4882a593SmuzhiyunDate: Feb 2020 20*4882a593SmuzhiyunKernelVersion: 5.7 21*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 22*4882a593SmuzhiyunDescription: Maximum number reconnect attempts the client should make before giving up 23*4882a593Smuzhiyun after connection breaks unexpectedly. 24*4882a593Smuzhiyun 25*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/mp_policy 26*4882a593SmuzhiyunDate: Feb 2020 27*4882a593SmuzhiyunKernelVersion: 5.7 28*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 29*4882a593SmuzhiyunDescription: Multipath policy specifies which path should be selected on each IO: 30*4882a593Smuzhiyun 31*4882a593Smuzhiyun round-robin (0): 32*4882a593Smuzhiyun select path in per CPU round-robin manner. 33*4882a593Smuzhiyun 34*4882a593Smuzhiyun min-inflight (1): 35*4882a593Smuzhiyun select path with minimum inflights. 36*4882a593Smuzhiyun 37*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/ 38*4882a593SmuzhiyunDate: Feb 2020 39*4882a593SmuzhiyunKernelVersion: 5.7 40*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 41*4882a593SmuzhiyunDescription: Each path belonging to a given session is listed here by its source and 42*4882a593Smuzhiyun destination address. When a new path is added to a session by writing to 43*4882a593Smuzhiyun the "add_path" entry, a directory <src@dst> is created. 44*4882a593Smuzhiyun 45*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/state 46*4882a593SmuzhiyunDate: Feb 2020 47*4882a593SmuzhiyunKernelVersion: 5.7 48*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 49*4882a593SmuzhiyunDescription: RO, Contains "connected" if the session is connected to the peer and fully 50*4882a593Smuzhiyun functional. Otherwise the file contains "disconnected" 51*4882a593Smuzhiyun 52*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect 53*4882a593SmuzhiyunDate: Feb 2020 54*4882a593SmuzhiyunKernelVersion: 5.7 55*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 56*4882a593SmuzhiyunDescription: Write "1" to the file in order to reconnect the path. 57*4882a593Smuzhiyun Operation is blocking and returns 0 if reconnect was successful. 58*4882a593Smuzhiyun 59*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect 60*4882a593SmuzhiyunDate: Feb 2020 61*4882a593SmuzhiyunKernelVersion: 5.7 62*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 63*4882a593SmuzhiyunDescription: Write "1" to the file in order to disconnect the path. 64*4882a593Smuzhiyun Operation blocks until RTRS path is disconnected. 65*4882a593Smuzhiyun 66*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path 67*4882a593SmuzhiyunDate: Feb 2020 68*4882a593SmuzhiyunKernelVersion: 5.7 69*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 70*4882a593SmuzhiyunDescription: Write "1" to the file in order to disconnected and remove the path 71*4882a593Smuzhiyun from the session. Operation blocks until the path is disconnected 72*4882a593Smuzhiyun and removed from the session. 73*4882a593Smuzhiyun 74*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name 75*4882a593SmuzhiyunDate: Feb 2020 76*4882a593SmuzhiyunKernelVersion: 5.7 77*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 78*4882a593SmuzhiyunDescription: RO, Contains the the name of HCA the connection established on. 79*4882a593Smuzhiyun 80*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port 81*4882a593SmuzhiyunDate: Feb 2020 82*4882a593SmuzhiyunKernelVersion: 5.7 83*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 84*4882a593SmuzhiyunDescription: RO, Contains the port number of active port traffic is going through. 85*4882a593Smuzhiyun 86*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr 87*4882a593SmuzhiyunDate: Feb 2020 88*4882a593SmuzhiyunKernelVersion: 5.7 89*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 90*4882a593SmuzhiyunDescription: RO, Contains the source address of the path 91*4882a593Smuzhiyun 92*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr 93*4882a593SmuzhiyunDate: Feb 2020 94*4882a593SmuzhiyunKernelVersion: 5.7 95*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 96*4882a593SmuzhiyunDescription: RO, Contains the destination address of the path 97*4882a593Smuzhiyun 98*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all 99*4882a593SmuzhiyunDate: Feb 2020 100*4882a593SmuzhiyunKernelVersion: 5.7 101*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 102*4882a593SmuzhiyunDescription: RW, Read will return usage help, write 0 will clear all the statistics. 103*4882a593Smuzhiyun 104*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration 105*4882a593SmuzhiyunDate: Feb 2020 106*4882a593SmuzhiyunKernelVersion: 5.7 107*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 108*4882a593SmuzhiyunDescription: RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's 109*4882a593Smuzhiyun not the case, the processing of an I/O response could be processed on a 110*4882a593Smuzhiyun different CPU than where it was originally submitted. This file shows 111*4882a593Smuzhiyun how many interrupts where generated on a non expected CPU. 112*4882a593Smuzhiyun 113*4882a593Smuzhiyun "from:" 114*4882a593Smuzhiyun is the CPU on which the IRQ was expected, but not generated. 115*4882a593Smuzhiyun "to:" 116*4882a593Smuzhiyun is the CPU on which the IRQ was generated, but not expected. 117*4882a593Smuzhiyun 118*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects 119*4882a593SmuzhiyunDate: Feb 2020 120*4882a593SmuzhiyunKernelVersion: 5.7 121*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 122*4882a593SmuzhiyunDescription: Contains 2 unsigned int values, the first one records number of successful 123*4882a593Smuzhiyun reconnects in the path lifetime, the second one records number of failed 124*4882a593Smuzhiyun reconnects in the path lifetime. 125*4882a593Smuzhiyun 126*4882a593SmuzhiyunWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma 127*4882a593SmuzhiyunDate: Feb 2020 128*4882a593SmuzhiyunKernelVersion: 5.7 129*4882a593SmuzhiyunContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 130*4882a593SmuzhiyunDescription: Contains statistics regarding rdma operations and inflight operations. 131*4882a593Smuzhiyun The output consists of 6 values:: 132*4882a593Smuzhiyun 133*4882a593Smuzhiyun <read-count> <read-total-size> <write-count> \ 134*4882a593Smuzhiyun <write-total-size> <inflights> <failovered> 135