P Socket: optimizing a communication library for a PCIe based intra rack interconnect
Data centers require efficient, low cost, flexible interconnects to manage the rapidly growing internal traffic generated by an increasingly diverse set of applications. To meet these requirements, data center networks are increasingly employing alternatives such as RapidIO, Freedom, and PCIe, which require fewer physical devices and/or have simpler protocols than more traditional interconnects. These networks offer raw high performance communication capabilities, but simply using them for conventional TCP/IP based communication fails to realize the potential performance of the physical network. Here we analyze causes for this performance loss for the TCP/IP protocol over one such fabric, PCIe, and we explore a hardware/software solution that mitigates overheads and exploits PCIe's advanced features. The result is P Socket, an efficient library that enables legacy socket applications to run without modification. Our experiments show that P Socket achieves an end to end latency of 1.2ms and effective bandwidth of up to 2.87GB/s (out of a theoretical peak of 3.05GB/s)
Conf. Computing Frontiers