BASP(4) BASP(4) NAME basp - Broadcom Advanced Server Program for Linux DESCRIPTION BASP is a kernel module designed for Linux 2.2.x and 2.4.x kernels that provides load-balancing, fault-tolerance, and VLAN features. These features are provided by creating teams that consist of multiple NIC interfaces. A team can consist of 1 to 8 NIC interfaces and each interface can be designated primary or hot-standby. All primary NIC inter­ faces in a team will participate in Load-balancing opera­ tions by sending and receiving a portion of the total traffic. Hot-standby interfaces will take over in the event that all primary interfaces have lost their links. VLANs can be added to a team to allow multiple VLANs with different VLAN IDs. A virtual device is created for each VLAN added. BASP supports Smart Load-balance (SLB) and Generic trunking. In SLB mode, all the NIC drivers must support Broadcom NIC Extension (NICE). In this release, several NIC drivers patched with NICE are included. SLB mode works with all Ethernet switches without configuring the switch ports to any special trunking mode. Only IP traffic will be load-balanced in both inbound and outbound direc­ tions. The Generic trunking mode does not require NICE and can work with any NIC, however, it requires the Ethernet switch to support the technology and be properly config­ ured. This mode is protocol-independent and all traffic should be load-balanced and fault-tolerant. LIMITATION BASP supports Red Hat 6.2, 7.0, 7.1 and 7.2 Beta. The following installation procedures work with these distri­ butions. BASP has also been tested on SuSE 7.1 & 7.2, Caldera 2.4 & 3.1, Turbo Linux 6.5 and Mandrake 8.0. Minor modification to the Makefile may be required if problems are experienced when compiling BASP on other i386 Linux distributions. BASP also supports Red Hat Linux 7.1 for IA-64 architec­ ture and has been tested on Redhat Linux 7.2 Beta for IA-64. INSTALL - TAR ARCHIVE BASP for Linux is shipped in mixed forms, where the plat­ form and kernel specific files are in source code, and the core file is in object form. Three packages are shipped in this release: two tar archives and two RPM package. "basplnx-{version}.i386.tgz" is the tar archive for i386 platform, and "basplnx-{version}.ia64.tgz" is the tar archive for IA-64 platform. To uncompress and expand the tar archive, run % tar xvfz basplnx-{version}.{arch}.tgz The installation process involves the following steps: (1) Configure the major device number for BASP configura­ tion. The default major number is 250 and it can be changed by modifying "Makefile". (2) To build kernel module, "basp.o", % make Note that the Make process will automatically build the correct module for different kernel options, e.g. symbol versioning and SMP support. There is NO need to define -DMODVERSIONS in the Make­ file. (3) To create device file and to copy files, % make install (4) To update the module reference, % depmod -a (5) To load the driver, % insmod basp Note-1) If when loading the BASP module, an error message regarding "invalid IO or IRQ parameters" are observed, you will need to configure the major device number for BASP configura­ tion by following step (1) above. (6) Follow the "Configuration and Startup for Other Linux Distribution" section to set up the teams. INSTALL - RPM PACKAGE (1) To install the RPM source package, run % rpm -i basplnx-{version}.src.{arch}.rpm (2) Change directory to the RPM path and build the binary driver for the kernel % cd /usr/src/redhat % rpm -bb SPECS/basplnx.spec Note that the RPM path is different for different Linux distributions. (3) Install the newly built package % rpm -i RPMS/i386/basplnx-{version}.{arch}.rpm The driver and other required files will be automati­ cally installed. (4) To load the driver, % insmod basp (5) Follow the Configuration section to set up the teams. CONFIGURATION - RED HAT DISTRIBUTION The BASP distribution includes an utility program and sev­ eral scripts for team configuration. Following steps for Red Hat Linux distributions only. Most of the steps are only required to be performed after the first time instal­ lation. Step 2 "Modify the configuration script" should be performed whenever there is any change to the team config­ uration. For users of other Linux distributions, follow instruc­ tions in the "baspcfg" section. The configuration process involves the following steps: (1) Copy a configuration script from "/etc/basp/samples" directory to "/etc/basp" directory. Note the configuration script must be prefixed with "team-". (2) Modify the configuration script to (a) change the team type (b) add/delete physical network interfaces (c) add/delete virtual network interfaces The syntax of the configuration script can be found below. Note that when configuring Teaming, at lease one Primary Adapter is required. (3) Run "netconf" to configure the IP address of the vir­ tual network interface. It is important to manually enter the interface name, i.e. sw0, when configuring the IP address. This is because the virtual network interface, i.e. sw0, is not shown in the phys­ ical interface list. Failure to enter the virtual interface name will result in missing "ifcfg-" script in /etc/sysconfig/network-scripts directory. This script is required to start the BASP team. (4) Manually start the team for the first time, % /etc/rc.d/init.d/basp start Note that this step is only required for the first time installation. The team configuration will be automatically started on subsequent reboots. Note that if not all the virtual network interfaces are configured with IP address, there will be an error message in starting the BASP team. When this happens, repeat step (3) to con­ figure IP address for all the virtual network interfaces. Note-1) Forming multiple teams is possible by copying the sample files into "/etc/basp/team-" and modifying this file as described in the sample file. Note-2) In order to create more that one virtual interface (VLAN) for each team, refer to the respective description section in the sample files. CONFIGURATION AND STARTUP FOR OTHER LINUX DISTRIBUTION "baspcfg" is a command line tool to configure the BASP teams, add/remove NICs and add/remove virtual devices. This tool, may be used to configure your own startup script files. Please read your distribution specific doc­ umentation for more information on your distributions's startup procedure. Following is the usage of this tool: baspcfg v2.0.5 - Broadcom Advanced Server Program Config­ uration Utility Copyright (c) 2000-2001 Broadcom Corpora­ tion. All rights reserved. usage: baspcfg com­ mands: addteam -- create a team delteam -- delete a team addva [macaddr] -- add a virtual adapter to a team delva -- del a virtual adapter from a team bind -- bind a physical adapter to a team unbind -- unbind a physical adapter from a team show [tid] -- display team con­ figurations where tid -- An unique ID for each team, starting from 0 type -- Team type: 0=SLB, 1=FEC/GEC tname -- ASCII string of the team vlan_id -- VLAN ID: from 1 to 4094, 0=untagged or no VLAN vname -- ASCII string of the virtual device macaddr -- MAC address (optional), e.g. 00:10:18:00:11:44 role -- Role of the physical device: 0=primary, 1=hot-standby device -- ASCII string of the physical device, e.g. eth0 Following sample startup script should be used to start the BASP after first time installation and configuration, or in the subsequent reboots. #!/bin/bash # load basp module insmod basp # create new team baspcfg addteam 0 0 team-one # bind physical interfaces / two primary one backup baspcfg bind 0 0 eth0 baspcfg bind 0 0 eth1 baspcfg bind 0 1 eth2 # create the virtual interface baspcfg addva 0 0 sw0 # bind ip address to virtual interface and initialize ifconfig sw0 192.168.0.1 up Note: Baspcfg command can only be executed in Super User mode only, configure with other login mode will yield "Permission Denied. Must be root to manage BASP" error message. When configuring Teaming, at lease one Primary Adapter is required. BASPCFG See _b_a_s_p_c_f_g_(_8_) for details. STARTUP SCRIPTS - RED HAT DISTRIBUTIONS (1) basp This script is intended to be installed in /etc/rc.d/init.d directory. After copying the script, run "chkconfig --add basp". This script will be executed at runlevel 2, 3, 4 and 5. When "basp" is run, it will search the /etc/basp directory to list all the files with "team-" prefix, and then it will invoke "baspteam" script to add or delete the teams. It is normal that each "team-*" file in /etc/basp represents 1 team. (2) baspteam This script is called by "basp" script to add or delete a team. To install, create "/etc/basp" direc­ tory and copy this script over. To manually add a team: % baspteam team-sample add To delete a team: % baspteam team-sample del Note that "team-sample" is the configuration script. (3) team-sample This script contains a SLB team configura­ tion with 3 NICs: eth0, eth1 and eth2. The team name is "TeamSample". All 3 NICs are primary. One virtual inter­ face is also created for this team and the name of the virtual interface is "sw0". "sw0" is the device that "ifconfig" should be run against to set up the IP address. VLANs are not enabled in this script. This script and "team-gec" are intended to be customized. Refer to the configuration scripts section for details. This script should be copied to /etc/basp directory and retain the "team-" prefix. (4) team-gec This configuration script creates a GEC team with 3 network interfaces, eth0, eth1 and eth2. The team name is "TeamGEC". All 3 NICs are primary. One virtual interface is added to the team with the name "sw0" and VLANs are not enabled. This script and "team-sample" are intended to be cus­ tomized. Refer to the configuration scripts section for details. This script should be copied to /etc/basp direc­ tory and retain the "team-" prefix. CONFIGURATION SCRIPTS - RED HAT DISTRIBUTIONS Both "team-sample" and "team-gec" are configuration scripts that follow the same syntax, as following TEAM_ID: this number uniquely identifies a team. TEAM_TYPE: 0 = SLB, 1 = Generic Trunking/GEC/FEC TEAM_NAME: ascii name of the team TEAM_PAx_NAME: ascii name of the physical interface x, where x can be 0 to 7. TEAM_PAx_ROLE: role of the physical interface x 0 = Primary, 1 = Hot-standby. This field must be 0 for Generic Trunking/GEC/FEC team. TEAM_VAx_NAME: ascii name of the virtual interface x, where x can be 0 to 63 TEAM_VAx_VLAN: 802.1Q VLAN ID of the virtual interface x. For untagged virtual interface, i.e. without VLAN enable, set it to 0. The valid VLAN ID can be 0 to 4094. Note: Teaming scripts are intended for Red Hat distribu­ tions ONLY, use with other Linux distribution will causes ERROR. BROADCOM NICE PATCHES Also included in this release are network device drivers patched with Broadcom NICE support. These drivers are originally taken from the Linux 2.2.16 kernel distribu­ tion. To installed patched drivers: (1) Copy the Broadcom NICE header file, "nicext.h", to the Linux kernel include directory, # cp /usr/src/nice-2.2.16/nicext.h /usr/src/linux/include/linux (2) Rename the original network device driver under the Linux kernel source tree, "/usr/src/linux/drivers/net". (3) Copy the patched drivers to the Linux kernel network driver source directory, i.e. "/usr/src/linux/drivers/net". (4) Follow the kernel rebuild instructions to configure kernel support for these drivers, i.e. # cd /usr/src/linux # make config (5) If the patched drivers are configured into the kernel, goto step (7). If the patched drivers are configured as modules, goto step (6). (6) In the case of supporting only the module version of these drivers, it is possible to simply run the following to compile patched drivers and to install them into the proper module directory: # make modules # make modules_install There is no need to compile the complete kernel. Goto step (8). (7) Rebuild the kernel to compile these patched drivers # make clean # make dep # make (8) Either reboot the system or unload/load the patched modules. Run configuration scripts to test the patch. UNINSTALL - RPM PACKAGE To uninstall RPM package, # rpm -e basplnx and to reboot the system, # reboot REMOVAL OF PHYSICAL INTERFACE IN GENERIC TRUNKING MODE In Generic Trunking mode, all the physical and virtual interfaces belonging to a team have the same MAC address. This MAC address is the same address as that of the first physical interface bounded to the team. In the case that this first physical interface is removed dynamically from the team using "baspcfg" tool and bounded to the protocol directly, this could lead to a duplicate MAC address prob­ lem on the network. Note that if the removed physical interface does not participate in any traffic, there will not be any problem. To properly remove a physical interface in a Generic Trunking team, follows the steps listed below: (1) Backup the original team configuration script # cp /etc/basp/team-gec /etc/basp/backup-gec Note-1) "team-gec" is the name of the configuration script. Note-2) "backup-gec" is the name of the backup script. The name of the backup script must NOT be prefixed with "team-". (2) Modify the team configuration script to remove the physical interface (3) Stop the running team # /etc/basp/baspif /etc/basp/backup-gec stop # /etc/basp/baspteam /etc/basp/backup-gec del (4) Re-start the team again # /etc/basp/baspteam /etc/basp/team-gec add # /etc/basp/baspif /etc/basp/team-gec start FILES /dev/basp /usr/bin/baspcfg /etc/rc.d/init.d/basp /etc/basp/baspteam /etc/basp/baspif /etc/basp/samples/team-sample /etc/basp/samples/team-gec /etc/basp/samples/team-vlan SEE ALSO baspcfg(8) AUTHOR Frankie Fan Broadcom Corporation. COPYRIGHT Copyright (c) 2000-2001 Broadcom Corporation. All rights reserved. April 27, 2001 1