The RHEL 5 documentation is good about bonding pairs of ports in various ways.The Oracle team where I work want to get more complex than that. They want an active bonded pair, with a third interface ready as a passive backup.Has anyone tried bonding at this level of complexity?I thought about trying to layer bonding configurations. Could I configure bondeth0 as an active pair combining eth0 and eth2, and then create a bondeth1 with bondeth0 as active and eth1 as passive?So,eth0 + eth2 = bondeth0, configured as an active pair.bondeth0 + eth1 = bondeth1, configured for active/passive.Has anyone ever tried that?EDIT: Here's a more comprehensive description. These are two hosts in an Oracle 11g RAC/RDBMS cluster. eth0 is the unrouted management connection between the two hosts. eth1 and eth3 go from each host to its corresponding switch. eth2 goes from the host to the opposite host's switch. This way, no single networking failure takes down a host. Here's a rough drawing: __________ eth1           ____________ | Host 1 |================| Switch 1 | |________| eth3           |__________|eth2 |   |                      |     |   |                      |     |    \-eth0--------eth0-   |     |                       \  |     |                       |  | eth2 ____________         eth3 __|_______ | Switch 2 |==============| Host 2 | |__________|         eth1 |________|I don't need to worry about eth0; it's working just fine. The RHEL Documentation covers the basics of bonding well. Many examples of bonding are available. I've even found a great description of the six bonding modes.What I need to know is how to combine two modes across three ports.
RHEL 5 complex port bonding
linux;ethernet;bonding
You cannot do this, as you cannot bond bonds.What you should do is configure three ports in a Mode 4 bond to each switch, and use the ad_select=count bonding option.Each switch will negotiate a separate Aggregator ID, and only one Aggregator can be used at a time, so all traffic to and from a given Linux system will go via one switch.This will give you an uneven bond, where one Aggregator has two cables, and one Aggregator has one cable, giving the concept of a primary and secondary switch. The primary Aggregator will always be selected, but if the primary goes down then the secondary kicks in automatically.You also have a requirement of dedicated Oracle RAC control traffic. The last port (eth3) will be used for this. (Note: I don't know if Oracle support using crossover for this, I'm just going off your requirements)Network DiagramNote that Po on the switch denotes PortChannel, or an LACP bond..-------------.    .-------------.|   Switch 1  |    |   Switch 2  ||-------------|    |-------------|| Po1  | Po2  |    | Po1  | Po2  |'-=--=-+-=--=-'    '-=--=-+-=--=-'  |  |   |  |        |      |     |  |   |  '-----,  |      |     |  |   '-----,  |  |      |     |  |         |  |  |      |     |  |   ,-----------'      |     |  |   |  ,-------------- | -,  |  |   |  |  |  |         |  |  |  |   |  |  |  '-----,   |  |  |  |   |  |  '-----,  |   |  |  |  |   |  |        |  |   |  | .=--=---=--=.      .=--=---=--=. |e0|e1|e2|e3|      |e0|e1|e2|e3| +--+--+--+--+      +--+--+--+--+ |   bond0   |      |   bond0   | |-----------|      |-----------| |  Host 1   |      |  Host 2   | '-----------'      '-----------'Or to think about it another way:Host 1 eth0 = bond0 ---- Switch1 Po1Host 1 eth1 = bond0 ---- Switch1 Po1Host 1 eth2 = bond0 ---- Switch2 Po1Host 1 eth3 ------------ Host 2 eth3Host 2 eth0 = bond0 ---- Switch1 Po2Host 2 eth1 = bond0 ---- Switch1 Po2Host 2 eth2 = bond0 ---- Switch2 Po2Host 2 eth3 ------------ Host 1 eth3Configure each bond with BONDING_OPTS=miimon=100 mode=4 ad_select=countIf all your external traffic goes through a gateway, add xmit_hash_policy=layer2+3 to the bonding options to load balance, and configure Layer 3 load balancing on the switch's PortChannel interface as well.