Simulating End-to-End Optical Fiber Transmission,  Complete Journey of a Photon Using Matlab

Author : Waqas Javaid

Abstract

This article presents a comprehensive MATLAB simulation of a 40 Gbps coherent optical fiber communication system using QPSK modulation over 100 km of standard single-mode fiber. The simulation models the complete signal chain from QPSK signal generation through fiber propagation using the split-step Fourier method, incorporating key physical impairments including chromatic dispersion, fiber nonlinearities, and attenuation [1]. Laser phase noise and amplified spontaneous emission noise from EDFAs are realistically simulated to emulate practical system conditions. A coherent receiver with digital signal processing performs carrier phase recovery and timing synchronization to mitigate transmission impairments [2]. The simulation visualizes signal evolution through eight detailed plots showing constellation diagrams, eye patterns, noise distributions, and spectral changes. System performance is quantified through bit error rate calculations and Q-factor analysis, demonstrating the resilience of coherent detection against accumulated impairments [3]. This work provides an educational framework for understanding complex optical transmission physics and offers a practical tool for evaluating modulation formats under realistic channel conditions. The modular MATLAB implementation allows researchers to extend the model for studying advanced modulation formats, longer transmission distances, or alternative compensation techniques [4]. Ultimately, this simulation bridges theoretical concepts with practical system design considerations in modern high-speed optical communications.

1. Introduction

The relentless growth of global data traffic, fueled by bandwidth-intensive applications from cloud computing to streaming services, has cemented optical fiber technology as the backbone of modern telecommunication networks.

To meet escalating demands, transmission systems have evolved towards higher bit rates and advanced coherent modulation formats, moving beyond the capabilities of simple on-off keying. The complexity of these systems, governed by intricate interactions between linear effects like chromatic dispersion and attenuation, and nonlinear phenomena such as self-phase modulation, necessitates sophisticated tools for design and analysis [5]. While theoretical models provide fundamental understanding, they often simplify or isolate these intertwined physical effects, creating a gap between textbook principles and practical system engineering. This article bridges that gap by presenting a detailed, end-to-end simulation of a 40 Gbps coherent optical communication link using quadrature phase-shift keying (QPSK) modulation, implemented entirely in MATLAB [6]. The simulation meticulously models the signal’s journey from digital bit generation, through laser modulation and 100 km of fiber propagation using the industry-standard split-step Fourier method, to coherent reception and digital signal processing for impairment mitigation [7]. By incorporating realistic channel impairments including laser phase noise, dispersion, fiber loss compensated by noisy optical amplifiers, and timing jitter the work provides a virtual laboratory for observing phenomena like constellation rotation, spectral broadening, and eye diagram closure [8]. The visual and quantitative outputs, including bit error rate and Q-factor calculations, offer critical insights into system limitations and the effectiveness of digital compensation algorithms [9]. This approach not only serves as a powerful educational instrument for understanding the physics of lightwave communication but also establishes a flexible framework for prototyping and evaluating next-generation transmission schemes under configurable and repeatable conditions.

1.1 The Need for Advanced Simulation in Optical Networks

Modern society’s dependence on instantaneous data exchange has pushed optical fiber networks to their physical limits, requiring a shift from simple intensity modulation to complex coherent schemes that encode information in both the amplitude and phase of light [10]. Designing these high-speed systems demands a deep understanding of how optical signals interact with the fiber’s material properties over hundreds of kilometers, a process fraught with signal-degrading effects that are difficult to predict analytically. Traditional laboratory prototyping for such systems is prohibitively expensive and time-consuming, requiring specialized equipment like ultra-pure lasers, low-noise amplifiers, and high-speed coherent receivers. Consequently, numerical simulation has become an indispensable tool, allowing engineers to virtually test system architectures, quantify performance margins, and troubleshoot potential failure modes before physical deployment [11]. This article addresses that critical need by constructing a comprehensive, physics-based simulation model that demystifies the complete signal chain in a coherent optical link, serving as both a practical design aid and an educational platform for understanding lightwave communication principles in a controlled digital environment.

1.2 From Theory to Practice, Modeling the Complete Transmission Chain

Moving from isolated theoretical concepts to a functioning system model requires integrating discrete physical phenomena into a cohesive digital pipeline. Our simulation begins at the logical layer with the generation of a pseudo-random bit sequence, which is then mapped to a quadrature phase-shift keying (QPSK) constellation a format chosen for its spectral efficiency and resilience. The digital symbols are pulse-shaped and then imprinted onto an optical carrier, a process that introduces the first practical impairment: finite laser linewidth, which manifests as random phase noise [12]. The core of the model lies in simulating the fiber channel itself using the split-step Fourier method (SSFM), a numerical technique that alternately applies linear effects (dispersion, attenuation) in the frequency domain and nonlinear effects (Kerr effect) in the time domain. This accurate propagation model shows how the initial clean pulse broadens, distorts, and interacts with itself, providing a visual narrative of signal degradation [13]. The transmitted signal is then amplified by an Erbium-Doped Fiber Amplifier (EDFA), which introduces the critical trade-off of gain versus amplified spontaneous emission (ASE) noise, a dominant noise source in long-haul links.

1.3 The Role of Coherent Detection and Digital Signal Processing

After traversing the noisy, dispersive channel, the optical signal must be accurately recovered. This is achieved through coherent detection, where the incoming signal is mixed with a local oscillator laser, converting phase and amplitude information into the electrical domain on both in-phase and quadrature components. Our simulation models this optical “homodyning” process, followed by analog-to-digital conversion and subsequent digital signal processing (DSP). The DSP stages are crucial for mitigation; they include matched filtering, timing synchronization via an early-late gate algorithm to find the optimal sampling instant, and carrier phase recovery using a Viterbi-Viterbi algorithm to undo random phase rotations [14]. These steps collectively transform a distorted, noisy received waveform back into a discernible constellation of symbols. By implementing this full receiver chain, the simulation highlights the indispensable role of DSP in modern coherent systems, demonstrating how algorithms can electronically compensate for physical imperfections that were once corrected with complex optical hardware.

1.4 Objectives and Outcomes of the Simulation Framework

The primary objective of this work is to create a transparent, modular, and didactic simulation framework that quantifies system performance under realistic impairment scenarios. Key outcomes include the generation of standard diagnostic plots such as eye diagrams, signal spectra, and constellation diagrams at various stages of the link, allowing for direct visual assessment of signal quality degradation and recovery. Furthermore, the framework provides quantitative performance metrics, most importantly the Bit Error Rate (BER) and the Q-factor, calculated by comparing the original transmitted bit sequence with the final decoded sequence [15]. By adjusting parameters like launch power, fiber length, or laser linewidth, users can explore their impact on system performance, observing the fundamental trade-offs between nonlinearity, noise, and dispersion [16]. Ultimately, this simulation serves as a virtual testbed, bridging abstract communication theory with the practical challenges of optical system engineering and providing a foundational tool for research and education in advanced optical communications.

1.5 Deconstructing Key Physical Impairments and Their Simulation

The fidelity of any optical communication simulation hinges on its accurate modeling of physical impairments. Our framework meticulously deconstructs these effects. Chromatic dispersion, arising from the wavelength-dependent propagation speed in glass, is simulated in the frequency domain, causing pulse broadening that leads to inter-symbol interference [17]. The fiber’s intrinsic attenuation, modeled as an exponential loss, dictates the need for periodic amplification. Crucially, we simulate the Kerr nonlinearity, where the refractive index of the fiber depends on the signal’s own intensity, causing effects like self-phase modulation that broaden the signal spectrum. The split-step Fourier method elegantly handles these combined linear and nonlinear effects by segmenting the fiber into small steps [18]. Separately, we model active components: the transmitter laser’s finite linewidth introduces a stochastic Wiener phase noise process, while the EDFA’s gain is accompanied by additive white Gaussian noise, simulating ASE. By isolating and then recombining these impairments, the simulation provides unparalleled insight into which effect dominates under specific power or distance regimes, a critical analysis for real-world system design.

1.6 Implementation of the Coherent Receiver and Synchronization Algorithms

The coherent receiver is the cornerstone that enables the compensation of transmission impairments in the digital domain. Our simulation implements a standard dual-polarization (single polarization for simplicity) coherent receiver front-end, where the received optical field is mixed with a local oscillator laser and photodetected to produce I and Q electrical components. The subsequent digital signal processing chain is implemented stepwise. First, resampling and matched filtering maximize the signal-to-noise ratio. Then, a feedback-based timing recovery loop, using a Mueller-and-Müller or early-late gate error detector, corrects for sampling clock offsets and recovers the optimal symbol timing. Following this, a feed-forward carrier phase estimation block, employing the Viterbi-Viterbi algorithm for QPSK, estimates and removes the accumulated phase noise from the laser linewidth and nonlinear phase shift [19]. These algorithmic blocks are not just functional modules; their parameters (loop bandwidth, block size) are tunable, allowing users to experiment with the trade-off between tracking speed and estimation noise, mirroring practical DSP design choices.

1.7 Performance Evaluation: Metrics, Visualization, and Analysis

The ultimate test of the simulation is its ability to quantify system performance. We implement standard industry metrics: the error vector magnitude (EVM) is calculated per symbol to assess modulation accuracy, the signal-to-noise ratio (SNR) is estimated from the received constellation, and the final bit error rate (BER) is computed via direct error counting after decision. More importantly, the simulation generates rich visualizations for intuitive analysis [20]. The evolution of the signal constellation from transmitter to receiver tells a clear story of noise addition, phase scattering, and DSP cleanup. The eye diagram, constructed by overlaying symbol-rate slices of the analog waveform, visually reveals the timing jitter, noise margin, and intersymbol interference. Spectral plots show the broadening due to nonlinearity. By sweeping a key parameter like launch power, the simulation can automatically generate a performance curve (e.g., BER vs. Power), vividly illustrating the optimal launch power where the penalties from nonlinearity and signal-to-noise ratio balance a fundamental concept in optical system design.

1.8 Educational and Research Applications of the Simulation Platform

This MATLAB-based simulation platform is designed as a multipurpose tool with significant educational and research utility. In an academic setting, it serves as a virtual laboratory, allowing students to visualize abstract concepts like dispersion compensation or phase noise by interactively changing parameters and observing the immediate effect on signals and BER. For researchers and engineers, it acts as a rapid prototyping sandbox. The modular code structure allows for easy modification: one can replace the QPSK modulator with a 16-QAM block, implement a different nonlinearity model, or test a novel carrier phase recovery algorithm [21]. It enables cost-effective “what-if” analysis for system feasibility studies, such as estimating the maximum reach for a given format or assessing the benefit of digital backpropagation. By providing a complete, working model from bits to BER, this framework lowers the barrier to entry for optical system simulation and fosters a deeper, hands-on understanding of the field’s core engineering challenges.

2. Problem Statement

The design and optimization of high-speed, long-haul coherent optical fiber communication systems are fundamentally constrained by the complex interplay of multiple physical impairments, including chromatic dispersion, fiber nonlinearities, laser phase noise, and amplified spontaneous emission noise. Analytically predicting the combined impact of these effects on system performance particularly for advanced modulation formats like QPSK is intractable, leading to a reliance on oversimplified models that fail to capture real-world signal degradation. Experimental validation, while essential, is prohibitively expensive and inflexible, requiring specialized hardware and offering limited visibility into intermediate signal states. Consequently, there exists a critical gap between theoretical understanding and practical system engineering, necessitating a comprehensive, accessible, and accurate numerical simulation platform. This platform must model the complete transmission chain, from digital signal generation through fiber propagation and coherent detection to digital signal processing, while providing quantitative performance metrics and visual diagnostics. The absence of such an integrated, educational tool hinders efficient design iteration, limits deep intuitive learning of physical layer phenomena, and complicates the evaluation of new modulation schemes or compensation algorithms in a risk-free virtual environment.

You can download the Project files here: Download files now. (You must be logged in).

3. Mathematical Approach

The simulation employs a rigorous mathematical framework grounded in the nonlinear Schrödinger equation (NLSE) to model optical signal propagation, solved numerically via the symmetrized split-step Fourier method which alternates between applying linear operators in the frequency domain and nonlinear operators in the time domain. Signal generation and recovery are governed by digital communication theory, using raised-cosine pulse shaping and matched filtering to satisfy the Nyquist criterion, while carrier modulation/demodulation follows complex exponential mixing. Laser phase noise is modeled as a Wiener process, and amplified spontaneous emission (ASE) noise as additive white Gaussian noise with power derived from the amplifier’s noise figure. Coherent detection and digital signal processing algorithms, including early-late gate timing recovery and Viterbi-Viterbi phase estimation, are implemented using statistical signal processing and maximum likelihood principles. Finally, system performance is quantified by computing the bit error rate through direct symbol-by-symbol comparison and by deriving the Q-factor from the estimated signal-to-noise ratio, linking stochastic noise models to practical error probability. The complete system is mathematically captured by three fundamental equations governing propagation, detection, and performance:

Signal Propagation (Nonlinear Schrödinger Equation with Solution):

Coherent Reception and Phase Recovery (Unified Receiver Model):

System Performance (BER and Q-Factor Relationship):

The mathematical framework is built around three core principles. First, the propagation of light through the fiber is described by an equation that accounts for three simultaneous effects: the steady weakening of the signal due to material absorption, the smearing of pulses over time because different colors of light travel at different speeds, and a complex nonlinear effect where the light’s own intensity changes the fiber’s properties, further distorting the signal. This equation is solved using a clever numerical technique that breaks the fiber into short segments, alternately calculating the linear pulse broadening in the realm of frequencies and the intensity-dependent distortion in the realm of time.

Second, the process of receiving the signal is mathematically combined into a coherent operation. The incoming distorted light wave is mixed with a pure local laser reference, which converts both its amplitude and phase information into measurable electrical currents. A subsequent digital algorithm then processes a block of these measurements; by raising the complex values to the fourth power and averaging, it cancels out the transmitted data and isolates the random phase noise, allowing for its precise estimation and correction. Finally, the system’s success is quantified by two interconnected metrics. The raw performance is measured by directly counting how many data bits were incorrectly decoded after the entire transmission and processing chain. This measured error rate is then linked to a theoretical quality factor through a standard mathematical function; this factor represents the effective signal-to-noise ratio margin in the system, providing a single powerful number that predicts how the system would perform under different noise conditions and allows for comparison against fundamental physical limits.

4. Methodology

The methodology for this optical communication simulation is architected as an end-to-end digital twin of a physical coherent transmission link, implemented systematically in MATLAB. The process begins with the generation of a pseudo-random binary sequence, which is mapped to a Quadrature Phase-Shift Keying (QPSK) constellation, normalizing the symbol energy for a fair power comparison. These digital symbols are then upsampled and shaped using a root-raised-cosine filter to limit the transmitted bandwidth and prevent inter-symbol interference in a linear channel. The shaped waveform modulates an optical carrier, simulated by multiplication with a complex exponential at the laser frequency, to which a stochastic Wiener process is added to model the critical impairment of finite laser linewidth and phase noise [22]. The heart of the simulation is the propagation of this optical field through 100 km of single-mode fiber, modeled by solving the nonlinear Schrödinger equation using the split-step Fourier method; this algorithm divides the fiber into short segments, alternately applying linear attenuation and dispersion in the frequency domain and the intensity-dependent nonlinear phase shift in the time domain [23].

Table 1: Fiber and EDFA Parameters

Fiber propagation, which includes exponential signal loss, the attenuated signal is amplified by an Erbium-Doped Fiber Amplifier (EDFA) model that applies a fixed gain and adds broadband amplified spontaneous emission noise, a dominant noise source. The receiver stage employs coherent homodyne detection, mixing the noisy signal with a local oscillator laser, yielding in-phase and quadrature electrical components that are sampled. These sampled signals undergo digital signal processing: first, matched filtering is applied to maximize the signal-to-noise ratio, followed by timing synchronization using an early-late gate algorithm to find the optimal symbol sampling instant. Carrier phase recovery is then performed using a Viterbi-Viterbi algorithm, which removes residual phase offsets by averaging over blocks of symbols. The final step involves making symbol-by-symbol decisions based on the recovered constellation quadrants, demapping the symbols back to a bit stream [24]. System performance is rigorously evaluated by comparing the original and recovered bit sequences to compute the bit error rate, while supplementary metrics like the signal-to-noise ratio, Q-factor, and error vector magnitude are calculated to provide a multi-faceted performance assessment. The entire pipeline is instrumented to generate diagnostic visualizations including signal waveforms, eye diagrams, constellation plots, and spectral evolutions at key stages, creating a comprehensive, transparent, and reproducible virtual laboratory for optical system analysis.

5. Design Matlab Simulation and Analysis

This comprehensive MATLAB simulation constructs a virtual 40 Gbps coherent optical fiber communication system to analyze signal integrity across a 100 km link.

Table 2: Simulation Parameters

The process begins by generating a random binary data stream and mapping it to a Quadrature Phase-Shift Keying (QPSK) constellation, where every two bits are represented by one of four phase states on the complex plane. The digital symbols are then upsampled and shaped using a root-raised-cosine filter to limit bandwidth before being imprinted onto an optical carrier at 193.1 THz. Real-world transmitter imperfection is introduced by modeling laser phase noise as a cumulative random walk, simulating the finite linewidth of practical lasers. The core of the simulation uses the split-step Fourier method to solve the nonlinear Schrödinger equation, mathematically propagating the optical field through the fiber by alternately calculating linear effects like chromatic dispersion and attenuation in the frequency domain, and the nonlinear Kerr effect in the time domain. After propagation, the attenuated signal is amplified by a simulated Erbium-Doped Fiber Amplifier, which adds the critical impairment of broadband amplified spontaneous emission noise. Reception is performed via coherent detection, where the incoming signal is mixed with a local oscillator laser, converting the optical field into in-phase and quadrature electrical components that are digitized. Subsequent digital signal processing applies matched filtering, recovers the optimal sampling instant using an early-late gate timing algorithm, and removes residual carrier phase offsets with a Viterbi-Viterbi estimator. The clean constellation points are then demapped back to bits, and the final system performance is quantified by calculating the bit error rate through direct comparison with the original transmitted sequence. Throughout this pipeline, the simulation generates eight diagnostic plots including signal waveforms, eye diagrams, constellation evolutions, and spectral analyses providing a complete visual narrative of signal degradation and recovery, and offering a powerful, integrated tool for understanding and designing high-speed optical links.

You can download the Project files here: Download files now. (You must be logged in).

This figure presents the starting point of the digital transmission chain. The left panel shows the in-phase (blue) and quadrature (red) components of the time-domain baseband signal after pulse shaping with a root-raised-cosine filter. The smooth transitions between amplitude levels demonstrate the effect of bandwidth-limiting filtering, which is crucial for minimizing inter-symbol interference in bandlimited channels. The right panel displays the ideal QPSK constellation diagram, where the four complex symbols—each representing two bits are plotted at their normalized positions: (1+j)/√2, (1-j)/√2, (-1+j)/√2, and (-1-j)/√2. This clean, symmetric plot establishes the reference for comparison against corrupted constellations observed later in the system. The perfect alignment and absence of spreading or rotation confirm proper signal generation before any physical impairments are introduced. Together, these two visualizations provide baseline metrics for signal amplitude, phase accuracy, and symbol integrity before the optical modulation and transmission stages.

This figure visualizes a fundamental transmitter impairment: the random phase fluctuation caused by the finite spectral linewidth of the laser source. The plot shows the unwrapped phase in radians versus time in nanoseconds, generated by integrating a Gaussian random walk whose step variance is proportional to the laser linewidth (100 kHz) and sampling period. The characteristic “random walk” trajectory demonstrates how phase accumulates stochastically rather than deterministically, causing the optical carrier’s phase to drift unpredictably over time. This noise mechanism is particularly detrimental in coherent systems because it directly corrupts the phase-encoded information in QPSK signals. The continuous, unbounded nature of the unwrapped phase highlights why carrier phase recovery algorithms are essential at the receiver. By quantifying this impairment, the simulation captures a key limitation in practical laser sources that theoretical models often assume to be ideal, providing insight into the required performance of digital phase estimation algorithms.

This dual-panel figure quantifies the transformative impact of the optical fiber channel. The left plot compares the instantaneous optical power versus time for the transmitted (blue) and received (red) signals. The received signal shows noticeable pulse broadening and amplitude variation due to the combined effects of chromatic dispersion, which smears pulses, and nonlinearity, which redistributes energy within the pulse shape. The right panel displays the corresponding normalized power spectra on a logarithmic (dB) scale versus frequency offset. The transmitted spectrum (blue) maintains the well-defined shape of the filtered QPSK signal, while the received spectrum (red) exhibits noticeable spectral broadening and asymmetry a hallmark of fiber nonlinearity (self-phase modulation). This side-by-side comparison visually demonstrates the two primary signal degradation mechanisms: temporal broadening from dispersion and spectral distortion from nonlinear effects, providing intuitive understanding of why dispersion compensation and nonlinearity management are critical in high-speed systems.

This figure characterizes the statistical nature of the dominant noise source in amplified optical systems: amplified spontaneous emission from the EDFA. The gray bars represent the simulated probability density of the detected ASE noise power, obtained by histogramming the squared magnitude of the added complex noise samples. The red curve shows the theoretical exponential probability density function predicted for the power of complex circular Gaussian noise. The close match between simulation and theory validates the proper implementation of ASE noise generation, where the noise power is determined by the amplifier’s gain, noise figure, and the optical bandwidth set by the sampling rate. This visualization is crucial because ASE noise fundamentally limits the system’s signal-to-noise ratio and thus its maximum reach. By confirming the noise statistics, the figure provides confidence that subsequent BER calculations accurately reflect real-world performance limitations imposed by optical amplification.

This three-panel figure illustrates the critical signal conditioning steps in the coherent receiver front-end. The first two panels show the in-phase (I) and quadrature (Q) electrical signals versus time, both before (blue) and after (red) anti-aliasing low-pass filtering. The filtering removes out-of-band noise and high-frequency mixing products while preserving the signal bandwidth, resulting in cleaner waveforms with reduced high-frequency fluctuations. The third panel displays the resulting received constellation a cloud of points on the complex plane representing individual sampled symbols after carrier mixing and filtering. The significant spreading, rotation, and distortion visible in this constellation compared to Figure 1’s ideal plot directly illustrates the cumulative impact of all channel impairments: phase noise, dispersion, nonlinearity, and ASE noise. This constellation provides the raw input for subsequent digital signal processing algorithms that must recover timing, correct phase, and make symbol decisions.

You can download the Project files here: Download files now. (You must be logged in).

The upper panel presents a manually constructed eye diagram by overlaying multiple traces of the filtered I-channel (blue) and Q-channel (red) signals over two symbol periods (unit intervals). The “eye” opening the vertical separation between signal levels at the optimal sampling instant visually represents the margin against noise and intersymbol interference; a wide, clearly defined eye indicates good signal quality. The mean traces (thick lines) show the average signal path, highlighting systematic distortion. The lower panel tracks the evolution of the timing error signal generated by the early-late gate synchronization algorithm, which computes the power difference between early and late sampling points to detect timing offset. The convergence of this error signal toward zero demonstrates the algorithm’s effectiveness in finding the optimal sampling instant, a critical step for minimizing decision errors in the presence of clock recovery imperfections.

This figure demonstrates the operation and effectiveness of the Viterbi-Viterbi carrier phase estimation algorithm. The left panel shows the unwrapped phase estimate in radians versus block number, where each estimate is obtained by averaging the fourth-power of symbols within a block then dividing by four. The relatively smooth evolution reflects the algorithm’s ability to track the slowly varying phase noise imparted by the transmitter laser and nonlinear phase shifts. The right panel compares the constellation after phase recovery (blue dots) against the original transmitted symbols (red circles). The significant reduction in angular spreading compared to Figure 5’s rotated constellation confirms that the algorithm successfully removes most of the random phase rotation, clustering the received symbols around the four ideal decision regions. Some residual spreading remains due to additive noise and imperfect estimation, but the recovered quadrants are clearly distinguishable, enabling reliable symbol decisions.

This final figure provides comprehensive system performance assessment through two complementary metrics. The left panel shows the cumulative squared error (sum of error vector magnitude) versus symbol index, offering insight into how errors accumulate throughout the transmission. A steep, linear increase indicates consistent error generation, while plateaus suggest periods of correct reception. The right panel presents the core system performance result: simulated BER (blue circle) plotted against the theoretical QPSK BER curve (red line) as a function of signal-to-noise ratio. The position of the simulated point relative to the theoretical curve quantifies the total “system penalty” incurred from all impairments—dispersion, nonlinearity, phase noise, and imperfect recovery algorithms. A point close to the theoretical line indicates efficient impairment mitigation, while significant deviation reveals unresolved limitations.

6. Results and Discussion

The simulation results demonstrate that the 40 Gbps QPSK coherent system successfully transmits data over 100 km with a final calculated bit error rate (BER) on the order of 1e-4 to 1e-6, depending on specific noise realizations, indicating performance within acceptable thresholds for modern optical links [25]. The key visual result is the successful closure of the eye diagram and the recovery of a well-defined QPSK constellation after digital signal processing, confirming the effectiveness of the implemented timing and phase recovery algorithms against accumulated impairments. Quantitative analysis reveals a system penalty of approximately 2-3 dB when comparing the required SNR for the simulated BER against the theoretical QPSK limit, attributed primarily to residual phase noise, imperfect filtering, and nonlinear phase distortion. The split-step Fourier propagation vividly illustrated the competing effects of dispersion, which broadened pulses in time, and nonlinearity, which broadened the signal spectrum, with the nonlinear length (L_nl) and dispersion length (L_d) providing critical insight into which impairment dominated for the given launch power. A significant discussion point is the optimal launch power trade-off: too low a power suffers from poor signal-to-noise ratio after amplification, while too high a power induces excessive nonlinear penalty, a fundamental design constraint captured by the simulation [26]. The successful operation of the Viterbi-Viterbi phase recovery, shown by the unwrapped phase estimate converging and stabilizing, highlights the algorithm’s necessity in compensating for the laser’s 100 kHz linewidth. Furthermore, the early-late gate timing recovery effectively minimized intersymbol interference, as evidenced by the clear eye opening. The close match between the simulated ASE noise distribution and the theoretical exponential curve validates the accuracy of the noise modeling. However, the residual spreading in the final constellation points toward limitations in the recovery algorithms and the inescapable degradation from fiber nonlinearities that are not fully reversible [27]. These results collectively validate the simulation framework as a predictive tool, demonstrating that coherent detection combined with digital signal processing can overcome severe channel distortions, enabling high-speed long-haul transmission [28]. The modular nature of the code allows for future investigation into advanced modulation formats, longer distances, or more sophisticated digital backpropagation techniques for nonlinearity compensation.

7. Conclusion

This comprehensive MATLAB simulation successfully demonstrates the feasibility and performance of a 40 Gbps QPSK coherent optical transmission system over 100 km of standard single-mode fiber, providing a virtual laboratory that bridges theoretical principles with practical engineering challenges [29]. The work validates that coherent detection combined with digital signal processing algorithms specifically split-step Fourier propagation modeling, early-late gate timing recovery, and Viterbi-Viterbi phase estimation can effectively mitigate key impairments including chromatic dispersion, laser phase noise, and amplified spontaneous emission noise [30]. The visual and quantitative results, from generated eye diagrams to calculated bit error rates, offer critical insights into signal degradation mechanisms and the trade-offs between launch power, nonlinearity, and signal-to-noise ratio. Ultimately, this simulation framework serves as a versatile, educational, and cost-effective platform for designing, analyzing, and optimizing next-generation high-speed optical communication systems, reducing reliance on expensive physical prototyping while deepening understanding of lightwave communication physics.

8. References

[1] G. P. Agrawal, Fiber-Optic Communication Systems, 4th ed., Wiley, 2010.

[2] G. P. Agrawal, Nonlinear Fiber Optics, 5th ed., Academic Press, 2013.

[3] E. Ip and J. M. Kahn, “Compensation of dispersion and nonlinear impairments using digital backpropagation,” Journal of Lightwave Technology, vol. 26, no. 20, pp. 3416–3425, 2008.

[4] J. G. Proakis and M. Salehi, Digital Communications, 5th ed., McGraw-Hill, 2008.

[5] A. Mecozzi, C. Antonelli, and M. Shtaif, “Nonlinear Shannon limit in pseudolinear coherent systems,” Journal of Lightwave Technology, vol. 30, no. 12, pp. 2011–2024, 2012.

[6] S. J. Savory, “Digital coherent optical receivers: Algorithms and subsystems,” IEEE Journal of Selected Topics in Quantum Electronics, vol. 16, no. 5, pp. 1164–1179, 2010.

[7] D. Marcuse, C. R. Menyuk, and P. K. A. Wai, “Application of the Manakov-PMD equation to studies of signal propagation in optical fibers with randomly varying birefringence,” Journal of Lightwave Technology, vol. 15, no. 9, pp. 1735–1746, 1997.

[8] P. J. Winzer and R. J. Essiambre, “Advanced modulation formats for high-capacity optical transport networks,” Journal of Lightwave Technology, vol. 24, no. 12, pp. 4711–4728, 2006.

[9] I. Kaminow, T. Li, and A. E. Willner, Optical Fiber Telecommunications, Vol. V, Academic Press, 2008.

[10] R. Noé, “Phase noise tolerant synchronous QPSK/BPSK baseband-type intradyne receiver concept with feedforward carrier recovery,” Journal of Lightwave Technology, vol. 23, no. 2, pp. 802–808, 2005.

[11] A. J. Viterbi and A. M. Viterbi, “Nonlinear estimation of PSK-modulated carrier phase with application to burst digital transmission,” IEEE Transactions on Information Theory, vol. 29, no. 4, pp. 543–551, 1983.

[12] M. Secondini, E. Forestieri, and G. Prati, “Achievable information rate in nonlinear WDM fiber-optic systems with arbitrary modulation formats,” Journal of Lightwave Technology, vol. 31, no. 23, pp. 3839–3852, 2013.

[13] X. Liu et al., “Digital signal processing techniques enabling flexible and efficient optical transport networks,” IEEE Signal Processing Magazine, vol. 31, no. 2, pp. 16–24, 2014.

[14] R. Schmogrow et al., “Error vector magnitude as a performance measure for advanced modulation formats,” IEEE Photonics Technology Letters, vol. 24, no. 1, pp. 61–63, 2012.

[15] K. Kikuchi, “Fundamentals of coherent optical fiber communications,” Journal of Lightwave Technology, vol. 34, no. 1, pp. 157–179, 2016.

[16] J. M. Kahn and K. P. Ho, “Spectral efficiency limits and modulation/detection techniques for DWDM systems,” IEEE Journal of Selected Topics in Quantum Electronics, vol. 10, no. 2, pp. 259–272, 2004.

[17] A. Yariv and P. Yeh, Photonics: Optical Electronics in Modern Communications, 6th ed., Oxford University Press, 2007.

[18] H. Nyquist, “Certain topics in telegraph transmission theory,” Transactions of the AIEE, vol. 47, pp. 617–644, 1928.

[19] C. R. S. Fludger et al., “Coherent equalization and POLMUX-RZ-DQPSK for robust 100-GbE transmission,” Journal of Lightwave Technology, vol. 26, no. 1, pp. 64–72, 2008.

[20] A. Papoulis and S. U. Pillai, Probability, Random Variables and Stochastic Processes, 4th ed., McGraw-Hill, 2002.

[21] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor graphs and the sum-product algorithm,” IEEE Transactions on Information Theory, vol. 47, no. 2, pp. 498–519, 2001.

[22] J. C. Palais, Fiber Optic Communications, 5th ed., Pearson, 2005.

[23] E. Desurvire, Erbium-Doped Fiber Amplifiers: Principles and Applications, Wiley, 1994.

[24] P. Poggiolini et al., “The GN-model of fiber non-linear propagation and its applications,” Journal of Lightwave Technology, vol. 32, no. 4, pp. 694–721, 2014.

[25] A. H. Gnauck and P. J. Winzer, “Optical phase-shift-keyed transmission,” Journal of Lightwave Technology, vol. 23, no. 1, pp. 115–130, 2005.

[26] M. Seimetz, High-Order Modulation for Optical Fiber Transmission, Springer, 2009.

[27] S. Haykin, Communication Systems, 5th ed., Wiley, 2009.

[28] R. C. Jones, “A new calculus for the treatment of optical systems,” Journal of the Optical Society of America, vol. 31, no. 7, pp. 488–493, 1941.

[29] J. Li, E. Tipsuwannakul, and S. J. Savory, “Digital coherent receiver algorithms for phase noise mitigation,” IEEE Photonics Technology Letters, vol. 22, no. 12, pp. 872–874, 2010.

[30] ITU-T Recommendation G.652, “Characteristics of a single-mode optical fibre and cable,” International Telecommunication Union, Geneva.

You can download the Project files here: Download files now. (You must be logged in).