BNX2X(4) BNX2X(4) NAME bnx2x - Broadcom NetXtreme II BCM55771x series 10-Gigabit Ethernet device driver SYNOPSIS insmod bnx2x.ko [int_mode=1] insmod bnx2x.ko [int_mode=2] insmod bnx2x.ko [disable_tpa=1] insmod bnx2x.ko [multi_mode=2 pri_map=0x11110000 qs_per_cos=0x404] insmod bnx2x.ko [multi_mode=4 pri_map=0x22221100 qs_per_cos=0x10101 [cos_min_rate=0x30201]] modprobe bnx2x [int_mode=1] modprobe bnx2x [int_mode=2] modprobe bnx2x [disable_tpa=1] modprobe bnx2x [multi_mode=2 pri_map=0x11110000 qs_per_cos=0x404] modprobe bnx2x [multi_mode=4 pri_map=0x22221100 qs_per_cos=0x10101 [cos_min_rate=0x30201]] DESCRIPTION bnx2x is the network device driver for the Broadcom NetXtreme II BCM55771x series PCIE 10-Gigabit Ethernet Network Interface Card (NIC). The driver has been tested on 2.6.x kernels starting from 2.6.9. Refer to the README.TXT from the driver package on how to compile and install the driver. Refer to various Linux documentations on how to configure network pro- tocol and address. DRIVER DEPENDENCIES The driver uses library functions in the crc32 and zlib_inflate libraries. On most kernels, these libraries are already built into the kernel. In some cases, it may be necessary to load these library mod- ules before the driver or unresolved symbol errors will appear. Using modprobe will resolve the dependencies automatically. In rare cases where the crc32 and zlib_inflate libraries are not enabled in the kernel, it will be necessary to compile the kernel again with the libraries enabled. The driver uses also library functions in the crc32c library. On new kernels, this library is already built into the kernel. In some old kernels, it may be necessary to load this library module before the driver or unresolved symbol errors will appear. Using modprobe will resolve the dependencies automatically. DRIVER SETTINGS The bnx2x driver settings can be queried and changed using ethtool. The latest ethtool can be downloaded from http://source- forge.net/projects/gkernel if it is not already installed. See the ethtool man page for more information. ethtool settings do not persist across reboot or module reload. The ethtool commands can be put in a startup script such as /etc/rc.local to preserve the settings across a reboot. On Red Hat distributions, "ethtool -s" parameters can be speci- fied in the ifcfg-ethx scripts using the ETHTOOL_OPTS keyword. The specified ethtool parameters will be set during ifup. Example: /etc/sysconfig/network-scripts/ifcfg-eth0: ETHTOOL_OPTS="wol g speed 100 duplex half autoneg off" PARAMETERS Several optional parameters can be supplied as a command line argument to the insmod or modprobe command. These parameters can also be set in modprobe.conf. See the man page for more information. The optional parameter int_mode is used to force using an interrupt mode other than MSI-X. By default, the driver will try to enable MSI-X if it is supported by the kernel. In case MSI-X is not attainable, the driver will try to enable MSI if it is supported by the kernel. In case MSI is not attainable, the driver will use legacy INTx mode. Set the int_mode parameter to 1 as shown below to force using the legacy INTx mode on all NetXtreme II NICs in the system. insmod bnx2x.ko int_mode=1 or modprobe bnx2x int_mode=1 Set the int_mode parameter to 2 as shown below to force using MSI mode on all NetXtreme II NICs in the system. insmod bnx2x.ko int_mode=2 or modprobe bnx2x int_mode=2 The optional parameter disable_tpa can be used to disable the Transpar- ent Packet Aggregation (TPA) feature. By default, the driver will aggregate TCP packets, but if a user would like to disable this advanced feature - it can be done. Set the disable_tpa parameter to 1 as shown below to disable the TPA feature on all NetXtreme II NICs in the system. insmod bnx2x.ko disable_tpa=1 or modprobe bnx2x disable_tpa=1 Use ethtool (if available) to disable TPA (LRO) for a specific NetX- treme II NIC. The optional parameter multi_mode is for use on systems that support multi queue networking. Multi queue networking on the receive side only is supported on kernels starting from 2.6.24. And multi queue network- ing on the transmit side also is supported on kernels starting from 2.6.27. By default, multi_mode parameter is set to 0 on old kernels that do not support multi queue networking. On new kernels multi_mode parameter is set to 1. Thus, on kernels starting from 2.6.24, the driver will allocate on the receive side one queue per-CPU and up to the HW limit and on the transmit side only one queue. On kernels start- ing from 2.6.27, the driver will allocate on both, receive and trans- mit, sides one queue per-CPU and up to the HW limit. The multi_mode optional parameter can also be used to enable SAFC (Ser- vice Aware Flow Control) by differentiating the traffic to up to 3 CoS (Class of Service) in the HW according to the VLAN PRI value or accord- ing to the IP DSCP value (least 3 bits). The optional parameter pri_map is used to map the VLAN PRI value or the IP DSCP value to a different or same CoS in the HW. This 32 bits parameter is evaluated by the driver as a 8 values of 4 bits each. Each nibble sets the desired HW queue number for that priority. For example, set the pri_map parameter to 0x22221100 to map priority 0 and 1 to CoS 0, map priority 2 and 3 to CoS 1, and map priority 4 to 7 to CoS 2. Another example, set the pri_map parameter to 0x11110000 to map priority 0 to 3 to CoS 0, and map priority 4 to 7 to CoS 1. The optional parameter qs_per_cos is used to specify how many queues will share the same CoS. This parameter is evaluated by the driver as a up to 3 values of 8 bits each. Each byte sets the desired number of queues for that CoS. The total number of queues is limited by the HW limit. For example, set the qs_per_cos parameter to 0x10101 to create total of three queues, one per CoS. Another example, set the qs_per_cos parame- ter to 0x404 to create total of 8 queues, divided into only 2 CoS, 4 queues in each CoS. The optional parameter cos_min_rate is used to determine the weight of each CoS for Round-robin scheduling in transmission. This parameter is evaluated by the driver as a up to 3 values of 8 bits each. Each byte sets the desired weight for that CoS. The weight ranges from 0 to 100. For example, set the cos_min_rate parameter to 0x101 for fair transmis- sion rate between 2 CoS. Another example, set the cos_min_rate parame- ter to 0x30201 give to higher CoS the higher rate of transmission. To avoid using the fairness algorithm, omit setting the optional parameter cos_min_rate or set it to 0. Set the multi_mode parameter to 2 as shown below to differentiate the traffic according to the VLAN PRI value. insmod bnx2x.ko multi_mode=2 pri_map=0x11110000 qs_per_cos=0x404 or modprobe bnx2x multi_mode=2 pri_map=0x11110000 qs_per_cos=0x404 Set the multi_mode parameter to 4 as shown below to differentiate the traffic according to the IP DSCP value. insmod bnx2x.ko multi_mode=4 pri_map=0x22221100 qs_per_cos=0x10101 cos_min_rate=0x30201 or modprobe bnx2x multi_mode=4 pri_map=0x22221100 qs_per_cos=0x10101 cos_min_rate=0x30201 There are some more optional parameters that can be supplied as a com- mand line argument to the insmod or modprobe command. These optional parameters are mainly to be used for debug and may be used only by an expert user. The debug optional parameter poll can be used for timer based polling. Set the poll parameter to the timer polling interval on all NetXtreme II NICs in the system. The debug optional parameter mrrs can be used to override the MRRS (Maximum Read Request Size) value of the HW. Set the mrrs parameter to the desired value (0..3) for on all NetXtreme II NICs in the system. The debug optional parameter debug can be used to set the default msglevel on all NetXtreme II NICs in the system. Use ethtool -s to set the msglevel for a specific NetXtreme II NIC. DEFAULT SETTINGS Speed : Autonegotiation with all speeds advertised Flow control : Autonegotiation with rx and tx advertised MTU : 1500 (range 46 - 9000) Rx Ring size : 4078 (range 0 - 4078) Tx Ring size : 4078 (range (MAX_SKB_FRAGS+4) - 4078) MAX_SKB_FRAGS varies on different kernels and different archi- tectures. On a 2.6 kernel for x86, MAX_SKB_FRAGS is 18. Coalesce rx usecs : 25 (range 0 - 3000) Coalesce tx usecs : 50 (range 0 - 12288) MSI-X : Enabled (if supported by 2.6 kernel) TSO : Enabled WoL : Disabled AUTHOR Eliezer Tamir - eliezert@broadcom.com SEE ALSO ifconfig(8), insmod(8), modprobe.conf(5), ethtool(8). Broadcom Corporation 11/29/07 BNX2X(4)