Pulse shape optimization has been suggested and developed for wireless and optical communication systems. However, conventionally, pulses are optimized according to metrics derived from a pulse's characteristics and each pulse shape was optimized for a specific objective such as, for example, tolerance to jitter, reduction of peak-to-average power ratio, energy maximization in high frequencies, improved tolerance to impulse truncation, etc. In fiber optic networks, pulse shaping is performed through filters at a transmitter and a corresponding receiver. Conventionally, optimization of the pulse shaping is done either theoretically (based on calculations) or based on an offline optimization procedure to minimize Intersymbol Interference (ISI) at the sampling phase of a receiver. Disadvantageously, these approaches do not take into account system impairments, such as finite quantification, filtering effects, the interaction between submodules, etc. Accordingly, conventional optimization approaches do not result in an optimal filter shape for an optical modem under different distortion conditions. Furthermore, conventional optimization approaches do not consider the fact that DWDM channels experience both intra- and inter-channel distortions. Therefore, obtained optimized pulse shapes, with conventional optimization approaches, do not result in a global optimal solution. Also, conventional optimization approaches have not been implemented as part of a working, multi-channel, end-to-end optical communication system.