I have above scenario where server has bond0 interface connected to SW1 and SW2 relevant interface of server, last night somehow SW1 got rebooted and server lost eth0 connectivity but it didn't failover to eth1 also every after switch rebooted successfully network was down and this happened on multiple server not single. so finally i have restart network on all server to make it work. we have primary=eth0 set in bond0 configuration. ifcfg-bond0DEVICE=bond0BOOTPROTO=noneONBOOT=yesBONDING_OPTS=mode=1 arp_interval=1000 arp_ip_target=192.168.10.1 miimon=500 downdelay=1000 primary=eth0 primary_reselect=alwaysNETMASK=255.255.255.0IPADDR=192.168.10.20GATEWAY=192.168.10.1ifcfg-eth0DEVICE=eth0BOOTPROTO=noneONBOOT=yesMASTER=bond0SLAVE=yesifcfg-eth1DEVICE=eth1BOOTPROTO=noneONBOOT=yesMASTER=bond0SLAVE=yesThis is what i am seeing in logs when this happened[root@server1 ~]# cat /var/log/messages | grep ethJun 8 05:03:05 s_sys@server1.example.com kernel: : bnx2 0000:02:00.0: eth0: NIC Copper Link is DownJun 8 05:03:05 s_sys@server1.example.com kernel: : bonding: bond0: link status definitely down for interface eth0, disabling itJun 8 05:03:05 s_sys@server1.example.com kernel: : bonding: bond0: making interface eth1 the new active one.Jun 8 05:07:03 s_sys@server1.example.com kernel: : bnx2 0000:02:00.0: eth0: NIC Copper Link is Up, 1000 Mbps full duplexJun 8 05:07:03 s_sys@server1.example.com kernel: : bonding: bond0: link status definitely up for interface eth0.Jun 8 05:20:45 s_sys@server1.example.com kernel: : bonding: bond0: Removing slave eth0Jun 8 05:20:45 s_sys@server1.example.com kernel: : bonding: bond0: Warning: the permanent HWaddr of eth0 - 18:A9:05:3A:39:1D - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts.Jun 8 05:20:45 s_sys@server1.example.com kernel: : bonding: bond0: releasing backup interface eth0Jun 8 05:20:46 s_sys@server1.example.com kernel: : bonding: bond0: Removing slave eth1Jun 8 05:20:46 s_sys@server1.example.com kernel: : bonding: bond0: releasing active interface eth1Jun 8 05:20:46 s_sys@server1.example.com kernel: : bonding: bond0: Adding slave eth0.Jun 8 05:20:46 s_sys@server1.example.com kernel: : bnx2 0000:02:00.0: eth0: using MSIXJun 8 05:20:46 s_sys@server1.example.com kernel: : bonding: bond0: enslaving eth0 as a backup interface with a down link.Jun 8 05:20:46 s_sys@server1.example.com kernel: : bonding: bond0: Adding slave eth1.Jun 8 05:20:46 s_sys@server1.example.com kernel: : bnx2 0000:02:00.1: eth1: using MSIXJun 8 05:20:46 s_sys@server1.example.com kernel: : bonding: bond0: enslaving eth1 as a backup interface with a down link.Jun 8 05:20:49 s_sys@server1.example.com kernel: : bnx2 0000:02:00.0: eth0: NIC Copper Link is Up, 1000 Mbps full duplexJun 8 05:20:49 s_sys@server1.example.com kernel: : bonding: bond0: link status definitely up for interface eth0.Jun 8 05:20:49 s_sys@server1.example.com kernel: : bonding: bond0: making interface eth0 the new active one.Jun 8 05:20:49 s_sys@server1.example.com kernel: : bnx2 0000:02:00.1: eth1: NIC Copper Link is Up, 1000 Mbps full duplexJun 8 05:20:50 s_sys@server1.example.com kernel: : bonding: bond0: link status definitely up for interface eth1.
Linux bonding failover issue between two cisco switch
linux;bonding;nsswitch;failover