Design of Ad-Hoc Algorithms for Performance Optimization of High-Order Modulation Formats in Coherent Optical Transmission Systems

Coherent optical detection has drawn a considerable interest in the past few years. This is mainly due to its high tolerance towards linear and non-linear fiber impairments and improved spectral efficiency, enabled by the use of polarization multiplexing (PM) and multilevel modulation formats. Using digital signal processing, coherent optical detection has made it possible to increase the spectral efficiency (SE) of optical systems well beyond the 1 b/s/Hz achievable in intensity modulated direct detection (IMDD) systems. Combined with polarization multiplexing (PM) and multilevel M-ary quadrature amplitude modulation (QAM) formats, coherent optical detection is considered to be the best candidate for future high capacity 100 and 400 Gbps wavelength-division multiplexing (WDM) systems. A critical part of coherent optical communication systems is the phase sensitive coherent receiver whose performance is limited by the phase noise that exists on the recovered data samples. Major source of phase noise is the finite linewidth of both transmitter (Tx) laser and receiver (Rx) local oscillator. Amplified Spontaneous Emission (ASE) creating nonlinear phase noise that interacts with the nonlinear Kerr effect, can also add in the phase noise of recovered data signal. This phase noise causes distortion and hence random rotation of the received constellation points. As a consequence, design of efficient carrier phase estimation (CPE) algorithms has become very important, especially while implementing high order modulation formats. This thesis presents several novel CPE algorithms for phase noise detection and compensation of high order QAM formats. Some algorithms make a phase estimation by increasing the number of symbols that took part in CPE while others use novel multistage architectures. The algorithms are based on a classic feed forward Viterbi&Viterbi (V&V) scheme. Performance of the algorithms in terms of complexity and linewidth times symbol duration product with other standard CPE algorithms is also given. It was observed that some of the proposed schemes give even better performance than the best available algorithms present in the literature. Also in optical communication systems frequency offset compensation (FOC) and carrier phase estimation (CPE) techniques require a prior knowledge of the modulation format but owing to the flexible transceivers, it is no longer guaranteed that the signals arriving at the receiver side would have the same, known in advance, modulation format (MF). The receiver thus needs to have some 'blind' or 'flexible' algorithms to adapt to these changes. Modulation format identification (MFI) is of high interest for the next generation fiber-optic networks as it could grant more autonomy and flexibility to the network. Elastic optical networks (EON) and cognitive optical networks (CON), with rate-adaptive transceivers supporting multiple modulation formats, have recently drawn a considerable interest as future optical networks. Although MFI for wireless systems has been thoroughly investigated, not much work has been done for the recognition/identification of modulation formats in fiber-optic networks. For MFI, a digital coherent receiver should be able to identify the MF of the incoming signals to ensure proper demodulation. Since FOC and CPE techniques require a prior knowledge of the modulation format, this makes blind MFI on-the-fly more difficult. As a consequence, we either need to develop FOC or CPE algorithms that are MF oblivious or develop some schemes that do MFI before entering the FOC or CPE blocks in the DSP receiver. This thesis also gives a simple and novel MFI scheme based on the evaluation of the peak-to-average-power ratio (PAPR) of the received data samples. To the best of our knowledge the proposed scheme is the simplest among all the schemes present in the literature. At a particular optical signal to noise ratio (OSNR) value, different modulation formats have distinct PAPR which can be used as defining parameter for their correct identification. Simulation and experimental results demonstrate successful identification of four commonly used modulation formats. Furthermore, the propose technique can also be extended to other lower or higher order formats. In order to practically generate these QPSK and QAM formats, the most commonly used modulator is the Dual Parallel Mach-Zender (DPMZ) modulator. DPMZ modulators are also used in a wide variety of radio-over-fiber (ROF) links. A stable biasing condition of these modulators is very important for high gain and low noise figure (NF) of these links. Being technologically mature and due to the fact that it has a linear Pockels effect, LiNBO3 is commonly used as a manufacturing material for MZ modulators. However, a change in temperature, radio-frequency (RF) heating or aging, can result in a drift of operating bias of LiNBO3 modulators. To cope with it, a close-loop control technique has to be developed that will follow this drift of the operating bias and will keep the DPMZ modulators under their optimal condition. This thesis also presents a close loop control technique for the automatic bias control of both the inner and outer MZ's of the DPMZ structure. This technique requires the generation of three separate low amplitude pilot tones at three different frequencies. These pilot tones are then clamped with the dc biasing voltage, at biasing ports of the MZ's. The pilot tones should have low amplitude so that they will not interfere with the RF signal data of the modulators. By carefully monitoring the beating between these tones, a close-loop control technique can be developed that will automatically monitor and follow the drift of the dc biasing voltage from its optimum and will enable the modulators to exhibit long term stability. In summary, this thesis makes important contributions by designing ad-hoc algorithms for performance optimization of high-order modulation formats in coherent optical transmission systems.