I'll share with you a common theme at this year's Dell Storage Forum.  Many presentations that focused on optimization and tuning all suggested the very same thing.  Keep your SAN storage network flat (no subnetting), simple, and make sure you have your switches stacked (instead of LAG'd), and configured to be rock solid.  
Hi Guys, first time posting here so be gentle.   I am having some issues with the performance of my backups and I think to some extent the iSCSI speed from the VM Host.   I'll give you the setup as it is and try and cover all configs to get plenty of details here for you.   First off, the Enviroment is based around vSphere 5 Essentials Plus Pack with ESXi 5.0 Hosts.  All devices have been loaded with the latest firmware, SAN, Servers, Switches. Servers = 2x Dell R710 with an aditional Intel Quad Port Nic for iSCSI. Backup server is a Dell 2950, Windows Server 2008 R2 Std, 16GB RAM, 8x 3.0 Ghz CPU and a Dual Port Intel Nic for iSCSI, Were using symantec Backup EXEC 2010 R3 with the VMware agent. Backup server is also running the vCenter Server software.  It backs up to a DAS MD1000 with 12x500GB 7200 SAS Drives over a SAS cable, cant remember if its 5 or 6.  The DAS can read and write at 200MB/s, tested the system using ATTO benchmark. The is also a Dell 2850 Server running VM hosts also connected to the iSCSI network using an Intel Quadport NIC. Switches = 2x Dell 6224 PowerConnect.  I have configured then as per the Dell Best practices, Seperate VLAN for iSCSI traffic, Jumbo frames are enabled all the way through from host to SAN, flow control is enabled too.  These switches are dedicated to iSCSI traffic only. SAN = Dell Equallogic PS4000XV Fully Loaded with SAS 10,000RPM Disks, 450GB.  The SAN has 2 controllers, each controller has 2 1GB Links and a 100MB management connector.  SAN is confidured in RAID 50.   I have cabled the iSCSI network for redundancy so its connected has follows. VM Host 1 = I have 4 iSCSI connections, Ports 1 & 2 go to Switch 1, ports 1 & 2.                                                                   Ports 3 & 4 go to switch 2, ports 3 & 4 VM Host 2 = I have 4 iSCSI connections, Ports 1 & 2 go to Switch 2, ports 1 & 2.                                                                   Ports 3 & 4 go to Switch 1, ports 3 & 4 VM Host 3 = I have 4 iSCSI connections, Ports 1 & 2 go to Switch 1, ports 5 & 6.                                                                   Ports 3 & 4 go to Switch 2, ports 5 & 6 Backup Server = I have 2 iSCSI Connections, Port 1 goes to Switch 1 Port 23                                                                           Port 2 goes to Switch 2 Port 23 SAN =  The 2x Controllers are Active / Passive so, Eth0 Act & Pas go to Port 20 & 21 on Switch 1                                                                                  Eth1 Act & Pas go to Port 20 & 21 on Switch 2   I have setup the VM hosts to use round robin all the way and have reduced the que depth to 3 per data store as recommended by Dell, this is to help get more load going over all the links faster insted of it only working for large files. I have disabled Delayed Ack on the VM hosts and the 2008 backup server. If I benchmark an iSCSI connected test volume formatted to NTFS from the backup server is get 240MB's read and about 220MB's write so its clearly maxing out the iSCSI link speed when checked against a network monitor on the NICs handeling iSCSI. MPIO is installed on the backup server.   Now, my problem is this.  I see in the web interface for the 6224 switches that port 1 on the switch is over 4x the ammount of data throughput on it VS the othe 3 ports comming from the VM hosts.  I have configured each VM hosts iSCSI so each port has its own VM kernel and its own single nic linked to it and then from there each NIC VM kernel is bound to the VMHBA. When I perform a backup, I only get a throughput of 2000MB/Min - 2500MB/Min when Backing up over SAN Transport.  I can see both Nics are being used but they only ever do about 20MB/s each.  I get around 47MB/s when using NBD on a single NIC.  If I copy a file from my laptop (has an SSD) to the backup server via 1GB network to the DAS I max out the copy of the file at over 105MB/s when copying a 30GB file. If I upload the same file to one of the Datastores through VMware I cant copy the file any faster than 40 - 47MB/s. Since the slowest factor in this copy process to the SAN should my network link I should be maxing that out at around 100MB/s.  The vCenter Server is running on the same server as the backup software but its worth noting that the vCenter server was once running as VM on the Hosts and the backup speed was still slow then.  I have also copied the sam 30GB file direct to the datastore while connected directly to the ESXi host rather than via the vCenter server and the speed was the same. So this eliminated the issue being with the backup / vCenter server.  The final part of this is, the subnetting.  I have used the IP address range of 40.0.0.1 - 200 for these devices. SAN has 40.0.0.10 for the discovery portal, SAN nics are 40.0.0.2 & 3 VM host example is One IP for each iSCSI Nic port, so its 40.0.0.11, 21, 31, 41 on Subnet mask of 255.255.0.0 Should I be using a different subnet for each nic port and if so, could someone give an example of a way that would be better?   I am very interested in the subnetting as there seems to be people with different views and I want to know if I have done something wrong here to the point that its causing me these issues.   On this link below there is a dell foru  post that says something about different subnets causing data to flow only down the first nic it finds on that subnet or somthing like this. I dont understand the relivence of it though I am sorry to say, see link below. [URL]     If anyone can help here this would be great.   Kind Regards   SyLvEsTeR20007      
Help regarding Subnets and performance