IB/srp: Add multichannel support

Improve performance by using multiple RDMA/RC channels per SCSI
host for communication with an SRP target. About the
implementation:
- Introduce a loop over all channels in the code that uses
  target->ch.
- Set the SRP_MULTICHAN_MULTI flag during login for the creation
  of the second and subsequent channels.
- RDMA completion vectors are chosen such that RDMA completion
  interrupts are handled by the CPU socket that submitted the I/O
  request. As one can see in this patch it has been assumed if a
  system contains n CPU sockets and m RDMA completion vectors
  have been assigned to an RDMA HCA that IRQ affinity has been
  configured such that completion vectors [i*m/n..(i+1)*m/n) are
  bound to CPU socket i with 0 <= i < n.
- Modify srp_free_ch_ib() and srp_free_req_data() such that it
  becomes safe to invoke these functions after the corresponding
  allocation function failed.
- Add a ch_count sysfs attribute per target port.

Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>

diff --git a/Documentation/ABI/stable/sysfs-driver-ib_srp b/Documentation/ABI/stable/sysfs-driver-ib_srp
index b9688de..7049a2b 100644 (file)
--- a/Documentation/ABI/stable/sysfs-driver-ib_srp
+++ b/Documentation/ABI/stable/sysfs-driver-ib_srp
@@ -55,12 +55,12 @@ Description:        Interface for making ib_srp connect to a new target.
                  only safe with partial memory descriptor list support enabled
                  (allow_ext_sg=1).
                * comp_vector, a number in the range 0..n-1 specifying the
-                 MSI-X completion vector. Some HCA's allocate multiple (n)
-                 MSI-X vectors per HCA port. If the IRQ affinity masks of
-                 these interrupts have been configured such that each MSI-X
-                 interrupt is handled by a different CPU then the comp_vector
-                 parameter can be used to spread the SRP completion workload
-                 over multiple CPU's.
+                 MSI-X completion vector of the first RDMA channel. Some
+                 HCA's allocate multiple (n) MSI-X vectors per HCA port. If
+                 the IRQ affinity masks of these interrupts have been
+                 configured such that each MSI-X interrupt is handled by a
+                 different CPU then the comp_vector parameter can be used to
+                 spread the SRP completion workload over multiple CPU's.
                * tl_retry_count, a number in the range 2..7 specifying the
                  IB RC retry count.
                * queue_size, the maximum number of commands that the
@@ -88,6 +88,13 @@ Description: Whether ib_srp is allowed to include a partial memory
                descriptor list in an SRP_CMD when communicating with an SRP
                target.
 
+What:          /sys/class/scsi_host/host<n>/ch_count
+Date:          April 1, 2015
+KernelVersion: 3.19
+Contact:       linux-rdma@vger.kernel.org
+Description:   Number of RDMA channels used for communication with the SRP
+               target.
+
 What:          /sys/class/scsi_host/host<n>/cmd_sg_entries
 Date:          May 19, 2011
 KernelVersion: 2.6.39
@@ -95,6 +102,12 @@ Contact:    linux-rdma@vger.kernel.org
 Description:   Maximum number of data buffer descriptors that may be sent to
                the target in a single SRP_CMD request.
 
+What:          /sys/class/scsi_host/host<n>/comp_vector
+Date:          September 2, 2013
+KernelVersion: 3.11
+Contact:       linux-rdma@vger.kernel.org
+Description:   Completion vector used for the first RDMA channel.
+
 What:          /sys/class/scsi_host/host<n>/dgid
 Date:          June 17, 2006
 KernelVersion: 2.6.17

diff --cc drivers/infiniband/ulp/srp/ib_srp.c
Simple merge

diff --cc drivers/infiniband/ulp/srp/ib_srp.h
Simple merge