1*4882a593Smuzhiyun.. SPDX-License-Identifier: GPL-2.0 2*4882a593Smuzhiyun 3*4882a593Smuzhiyun=========================================== 4*4882a593SmuzhiyunHOWTO for multiqueue network device support 5*4882a593Smuzhiyun=========================================== 6*4882a593Smuzhiyun 7*4882a593SmuzhiyunSection 1: Base driver requirements for implementing multiqueue support 8*4882a593Smuzhiyun======================================================================= 9*4882a593Smuzhiyun 10*4882a593SmuzhiyunIntro: Kernel support for multiqueue devices 11*4882a593Smuzhiyun--------------------------------------------------------- 12*4882a593Smuzhiyun 13*4882a593SmuzhiyunKernel support for multiqueue devices is always present. 14*4882a593Smuzhiyun 15*4882a593SmuzhiyunBase drivers are required to use the new alloc_etherdev_mq() or 16*4882a593Smuzhiyunalloc_netdev_mq() functions to allocate the subqueues for the device. The 17*4882a593Smuzhiyununderlying kernel API will take care of the allocation and deallocation of 18*4882a593Smuzhiyunthe subqueue memory, as well as netdev configuration of where the queues 19*4882a593Smuzhiyunexist in memory. 20*4882a593Smuzhiyun 21*4882a593SmuzhiyunThe base driver will also need to manage the queues as it does the global 22*4882a593Smuzhiyunnetdev->queue_lock today. Therefore base drivers should use the 23*4882a593Smuzhiyunnetif_{start|stop|wake}_subqueue() functions to manage each queue while the 24*4882a593Smuzhiyundevice is still operational. netdev->queue_lock is still used when the device 25*4882a593Smuzhiyuncomes online or when it's completely shut down (unregister_netdev(), etc.). 26*4882a593Smuzhiyun 27*4882a593Smuzhiyun 28*4882a593SmuzhiyunSection 2: Qdisc support for multiqueue devices 29*4882a593Smuzhiyun=============================================== 30*4882a593Smuzhiyun 31*4882a593SmuzhiyunCurrently two qdiscs are optimized for multiqueue devices. The first is the 32*4882a593Smuzhiyundefault pfifo_fast qdisc. This qdisc supports one qdisc per hardware queue. 33*4882a593SmuzhiyunA new round-robin qdisc, sch_multiq also supports multiple hardware queues. The 34*4882a593Smuzhiyunqdisc is responsible for classifying the skb's and then directing the skb's to 35*4882a593Smuzhiyunbands and queues based on the value in skb->queue_mapping. Use this field in 36*4882a593Smuzhiyunthe base driver to determine which queue to send the skb to. 37*4882a593Smuzhiyun 38*4882a593Smuzhiyunsch_multiq has been added for hardware that wishes to avoid head-of-line 39*4882a593Smuzhiyunblocking. It will cycle though the bands and verify that the hardware queue 40*4882a593Smuzhiyunassociated with the band is not stopped prior to dequeuing a packet. 41*4882a593Smuzhiyun 42*4882a593SmuzhiyunOn qdisc load, the number of bands is based on the number of queues on the 43*4882a593Smuzhiyunhardware. Once the association is made, any skb with skb->queue_mapping set, 44*4882a593Smuzhiyunwill be queued to the band associated with the hardware queue. 45*4882a593Smuzhiyun 46*4882a593Smuzhiyun 47*4882a593SmuzhiyunSection 3: Brief howto using MULTIQ for multiqueue devices 48*4882a593Smuzhiyun========================================================== 49*4882a593Smuzhiyun 50*4882a593SmuzhiyunThe userspace command 'tc,' part of the iproute2 package, is used to configure 51*4882a593Smuzhiyunqdiscs. To add the MULTIQ qdisc to your network device, assuming the device 52*4882a593Smuzhiyunis called eth0, run the following command:: 53*4882a593Smuzhiyun 54*4882a593Smuzhiyun # tc qdisc add dev eth0 root handle 1: multiq 55*4882a593Smuzhiyun 56*4882a593SmuzhiyunThe qdisc will allocate the number of bands to equal the number of queues that 57*4882a593Smuzhiyunthe device reports, and bring the qdisc online. Assuming eth0 has 4 Tx 58*4882a593Smuzhiyunqueues, the band mapping would look like:: 59*4882a593Smuzhiyun 60*4882a593Smuzhiyun band 0 => queue 0 61*4882a593Smuzhiyun band 1 => queue 1 62*4882a593Smuzhiyun band 2 => queue 2 63*4882a593Smuzhiyun band 3 => queue 3 64*4882a593Smuzhiyun 65*4882a593SmuzhiyunTraffic will begin flowing through each queue based on either the simple_tx_hash 66*4882a593Smuzhiyunfunction or based on netdev->select_queue() if you have it defined. 67*4882a593Smuzhiyun 68*4882a593SmuzhiyunThe behavior of tc filters remains the same. However a new tc action, 69*4882a593Smuzhiyunskbedit, has been added. Assuming you wanted to route all traffic to a 70*4882a593Smuzhiyunspecific host, for example 192.168.0.3, through a specific queue you could use 71*4882a593Smuzhiyunthis action and establish a filter such as:: 72*4882a593Smuzhiyun 73*4882a593Smuzhiyun tc filter add dev eth0 parent 1: protocol ip prio 1 u32 \ 74*4882a593Smuzhiyun match ip dst 192.168.0.3 \ 75*4882a593Smuzhiyun action skbedit queue_mapping 3 76*4882a593Smuzhiyun 77*4882a593Smuzhiyun:Author: Alexander Duyck <alexander.h.duyck@intel.com> 78*4882a593Smuzhiyun:Original Author: Peter P. Waskiewicz Jr. <peter.p.waskiewicz.jr@intel.com> 79