Design and Evaluation of a Variational Multimodal Image Fusion Algorithm in MATLAB

Author : Waqas Javaid

Abstract:

Multimodal image fusion aims to integrate complementary information from multiple imaging sources into a single high-quality representation. This study presents a variational fusion framework based on total-variation (TV) regularization optimized via the Split-Bregman method to achieve effective edge preservation and noise suppression. According to Rudin et al. (1992), nonlinear total variation based noise removal algorithms can be effective in image processing [1]. The proposed approach fuses luminance components in the YCbCr color domain to retain both structural details and visual consistency. Gopi et al. (2013) proposed an MR image reconstruction method based on iterative split Bregman algorithm and nonlocal total variation [2]. A weighted initialization followed by iterative TV minimization produces spatially coherent fused images while controlling smoothing through a tunable regularization parameter. The algorithm is implemented in MATLAB and evaluated using benchmark images. Objective quality assessment is performed using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). Edge comparison and difference maps further validate detail retention from both modalities. Convergence behavior is analyzed through residual curves demonstrating stable optimization performance. Experimental results indicate improved structural preservation compared to initial weighted fusion. Yuan et al. (2015) developed efficient convex optimization approaches to variational image fusion [3]. The proposed framework provides a reliable and computationally efficient solution for multimodal image fusion applications.

1. Introduction:

Multimodal image fusion has emerged as a critical technique for combining complementary information from multiple imaging sources into a single, informative visual representation. In many scientific, medical, and remote sensing applications, individual modalities capture different aspects of the same scene, such as structural detail, spectral content, or functional information, and no single modality is sufficient to describe the complete scene effectively.

Image fusion aims to integrate these diverse data sources while preserving essential edges, textures, and contrast characteristics. Li and Zeng (2016) presented a variational image fusion method with first and second-order gradient information [4]. Traditional fusion approaches based on simple averaging or transform-domain methods often result in reduced contrast, loss of fine details, or the introduction of artifacts. Variational methods have gained significant attention due to their strong theoretical foundation and flexibility in modeling image priors. Bungert et al. (2017) introduced a blind image fusion method for hyperspectral imaging with the directional total variation [5]. Among these, total-variation regularization is particularly effective for preserving sharp edges while suppressing noise and minor artifacts. The Split-Bregman method offers an efficient optimization framework for solving TV-based energy minimization problems with fast convergence and low computational cost. This study proposes a robust multimodal fusion framework that leverages TV regularization optimized via the Split-Bregman algorithm. The approach fuses the luminance component in the YCbCr color space to maintain both spatial detail and visual fidelity in color images. Weighted initialization provides an effective starting estimate that accelerates convergence. Objective performance is quantified using PSNR and SSIM metrics, complemented by edge maps and difference analysis for visual evaluation. Convergence behavior is systematically analyzed to confirm algorithm stability. Simões et al. (2014) proposed a hyperspectral image superresolution method using an edge-preserving convex formulation [6].  MATLAB-based simulations demonstrate the effectiveness of the proposed approach across standard benchmark images.

The results indicate superior preservation of edges and structural features compared to conventional weighted fusion techniques. Overall, the proposed method provides a reliable fusion solution suitable for diverse multimodal imaging applications.

1.1 Background and Motivation:

Multimodal image fusion integrates information from multiple imaging sources to generate a single, enhanced output image that is more informative than individual inputs. In applications such as medical imaging, surveillance, and remote sensing, each modality captures unique features such as anatomical structure, texture detail, or spectral characteristics. Direct analysis of multiple images can be inefficient and cognitively demanding for human observers and automated systems. Therefore, fusion algorithms aim to combine these complementary features into a single coherent representation. Papafitsoros and Schönlieb (2012) presented a combined first and second order variational approach for image reconstruction [7]. Conventional pixel-level fusion techniques using averaging or simple weighting often lead to reduced contrast and blurred boundaries. Transform-domain approaches such as wavelet fusion improve frequency representation but can introduce artifacts and fail to preserve local discontinuities. Consequently, spatial-domain techniques grounded in variational models have gained popularity. Chen et al. (2014) developed a simultaneous image registration and fusion method in a unified framework [8]. These approaches enable explicit regularization of image properties such as smoothness and edge preservation, making them better suited for real-world fusion problems.

1.2 Variational TV Fusion and Optimization Approach:

Total-variation (TV) regularization is a well-established method for preserving edges while reducing noise, making it particularly effective for image fusion tasks.

Table 1: Fusion Results and Metrics.

The TV framework formulates fusion as an optimization problem that balances fidelity to input modalities with smoothness constraints on the fused image. However, directly solving TV-regularized minimization problems can be computationally expensive and numerically unstable. The Split-Bregman method provides an efficient optimization framework by decomposing the problem into simpler subproblems that are solved iteratively. Zhu et al. (2010) proposed duality-based algorithms for total-variation-regularized image restoration [9]. This strategy accelerates convergence, reduces memory requirements, and ensures numerical stability. The method alternates between image update steps and constraint enforcement through Bregman variables, allowing large optimization problems to be solved efficiently. Compared to classical gradient descent or primal-dual solvers, Split-Bregman exhibits faster convergence and simpler implementation. Its effectiveness makes it especially attractive for real-time and large-scale image fusion applications.

1.3 Objectives and Contributions of this Study:

This study presents a robust multimodal image fusion framework based on variational TV minimization optimized via the Split-Bregman algorithm. The fusion process focuses on combining luminance components in the YCbCr color space to preserve chromatic consistency while enhancing spatial features. An adaptive weighted fusion initialization is utilized to provide a stable starting solution and improve convergence speed. Comprehensive performance evaluation is conducted using objective metrics including Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). Zhu et al. (2010) proposed duality-based algorithms for total-variation-regularized image restoration [9]. Visual analysis through edge detection, difference maps, intensity profiles, and convergence curves further validates the effectiveness of the proposed approach. The entire methodology is implemented in MATLAB using benchmark grayscale and color test images. Experimental results demonstrate improved structural preservation and contrast enhancement compared to traditional weighted fusion methods. Nonlocal total variation based image deblurring using Split Bregman method and fixed point iteration” was presented in Applied Mechanics and Materials (2013) [10].  The proposed framework offers a computationally efficient and scalable solution suitable for a wide range of multimodal imaging applications.

1.4 Experimental Design and Evaluation Strategy:

To validate the proposed fusion framework, a comprehensive experimental setup is designed using benchmark grayscale and color images of varying texture and contrast complexities. Input images are preprocessed through contrast normalization and spatial resizing to ensure uniformity across modalities. Color images are decomposed into YCbCr space, and only the luminance component is fused while chrominance channels are combined using weighted averaging to maintain color fidelity. Wang et al. (2012) presented a total variant image deblurring method based on split Bregman method [11]. Qualitative visualization techniques including edge maps, difference maps, and pixel intensity profiles are employed to assess detail preservation and artifact suppression. Quantitative evaluations are conducted using PSNR and SSIM metrics to measure fidelity and perceptual structural similarity. Convergence performance is examined using residual error curves obtained from the Split-Bregman iterations. Huang et al. (2020) reviewed multimodal medical image fusion techniques [12]. Parameter sensitivity analysis is carried out by varying the TV regularization weight. This multifaceted evaluation strategy ensures robust validation of algorithm effectiveness.

1.5 Computational Implementation and Optimization:

The entire fusion algorithm is implemented in MATLAB to ensure reproducibility and ease of experimentation. Efficient matrix operations and built-in image processing functions are exploited to reduce computational overhead. Srikanth et al. (2024) proposed a medical image fusion method using multi-scale guided filtering [13]. The Split-Bregman solver is structured to allow flexible tuning of key parameters such as regularization strength, stopping tolerance, and iteration limits. Automated result saving and batch parameter sweep functionality are included to streamline experimentation. Memory-efficient data handling techniques are utilized to process high-resolution images effectively. Iterative convergence monitoring enables early stopping when the residual change drops below the defined threshold. Additional diagnostic tools, including histogram analysis and saliency map visualization, are incorporated to explore feature contribution across modalities. Chen and Wunderli (2002) discussed adaptive total variation for image restoration in BV space [14]. This optimized computational design delivers reliable performance suitable for both academic research and practical deployment.

1.6 Significance and Practical Implications:

The proposed variational fusion framework offers meaningful contributions to image processing practice and research. By integrating TV regularization with efficient Split-Bregman optimization, the method achieves superior edge preservation without introducing excessive noise suppression artifacts. Its adaptability to both grayscale and color image fusion enhances application flexibility across diverse imaging domains. Chen and Wunderli (2002) discussed adaptive total variation for image restoration in BV space [15]. The combination of objective quality metrics with rich visual diagnostics ensures interpretation consistency and clarity for both algorithm developers and end-users. The framework can be readily adapted to emerging modalities such as hyperspectral, thermal-infrared, and medical imaging systems. Furthermore, the modular MATLAB structure allows extension toward real-time or GPU-accelerated implementations. Overall, the methodological rigor and practical value establish this work as a reliable reference for future advancements in multimodal image fusion research.

2. Problem Statement:

Multimodal image fusion requires combining information from different imaging modalities into a single image that retains the most meaningful features of each source. However, many existing fusion approaches struggle to balance noise suppression with edge and texture preservation, often producing blurred boundaries or loss of fine structural details. Simple averaging and transform-based methods may introduce artifacts or fail to maintain spatial consistency. Optimization-based fusion models, although more effective, frequently suffer from high computational cost or slow convergence. Additionally, improper handling of color information can lead to spectral distortion in fused outputs. Parameter sensitivity further complicates achieving consistent fusion quality across different datasets. Objective evaluation of fusion performance remains challenging when perceptual quality and structural fidelity must both be considered. Efficient implementation is also critical for practical deployment in real-time or large-scale applications. Therefore, an effective fusion method must integrate robust optimization, fast convergence, accurate feature preservation, and reliable performance assessment. This study addresses these challenges by formulating multimodal fusion using total-variation regularization optimized with the Split-Bregman method.

3. Mathematical Approach:

The multimodal image fusion problem is formulated as a variational optimization task in which two input images (A) and (B) are combined to estimate a single fused image (U). A weighted data-fidelity term ensures that the fused image remains consistent with both modalities, while total variation (TV) regularization promotes spatial smoothness and preserves sharp edges. The energy functional is defined as the sum of squared differences between the fused image and each input image, weighted by modality parameters (mu_1) and (mu_2), along with a TV penalty scaled by regularization parameter (lambda). The isotropic TV operator is expressed through the spatial gradient magnitude, penalizing unnecessary intensity oscillations while maintaining significant discontinuities such as object boundaries. To overcome the non-differentiability of the TV term, an auxiliary variable is introduced, separating the gradient operator from the regularization. This converts the constrained minimization problem into an augmented Lagrangian form solvable via the Split-Bregman method. The resulting iterative scheme alternates between solving a quadratic subproblem for the image update, performing a soft-thresholding operation to update the auxiliary gradient variables, and updating the Bregman multipliers to enforce consistency constraints. The image update equation reduces to a linear system integrating data fidelity and Laplacian smoothing terms. Gradient shrinkage implements anisotropic denoising by attenuating small gradient components while preserving significant structures. The Bregman update ensures convergence toward constraint satisfaction. Convergence is measured by norm-based residual reduction across iterations. The weighted initialization provides fast early convergence toward optimal solutions. Color images are processed by transforming into the YCbCr space, solving the variational model on the luminance channel only, and recombining weighted chrominance components. This mathematical formulation provides a stable, edge-aware fusion process yielding visually coherent and quantitatively superior fused images.

4. Methodology:

The proposed multimodal fusion methodology begins with preprocessing of the source images to ensure spatial alignment and intensity normalization.

Table 2: Model Parameters.

Color images are provided, the RGB data are transformed into the YCbCr color space, and only the luminance channel is selected for optimization to preserve structural detail while maintaining color consistency. A weighted averaging scheme is applied to generate an initial fused image, which serves as the starting point for variational optimization. Chen and Wunderli (2002) discussed adaptive total variation for image restoration in BV space [16]. The fusion task is formulated using a total-variation regularization framework that balances fidelity to both source images against spatial smoothness constraints. Meyer (2001) studied oscillating patterns in image processing and nonlinear evolution equations [17]. The Split-Bregman algorithm is then employed to solve the resulting optimization problem efficiently. At each iteration, a quadratic image update step is computed, followed by gradient shrinkage through soft-thresholding to enforce TV regularization. Bregman multipliers are updated to maintain constraint consistency between image gradients and auxiliary variables. Iterations continue until the relative residual falls below a predefined tolerance or the maximum iteration count is reached. Osher (2005) presented Mumford-Shah and total variation based variational methods for imaging [18]. The regularization parameter is tuned to control oversmoothing versus detail retention. After convergence, fused luminance is recombined with averaged chrominance channels to reconstruct the final color image. Performance evaluation includes objective measures such as PSNR and SSIM, alongside visual analysis using edge maps, intensity profiles, and difference images. Convergence behavior is studied via residual curves captured across iterations. Piella (2003) reviewed image fusion quality models [19].  All algorithms are implemented in MATLAB, with batch processing and result-saving routines for reproducibility. This comprehensive methodology ensures robust fusion performance across different input modalities.

5. Design Matlab Simulation and Analysis:

The simulation implements a variational total variation (TV) based multimodal image fusion using the Split-Bregman optimization method. It starts by loading two input images, which may be grayscale or color, and resizes them to a common dimension for alignment. The images are preprocessed via slight contrast enhancement to normalize intensities, ensuring consistent fusion behavior. For color images, the RGB channels are converted to YCbCr, and the fusion is applied only to the luminance (Y) channel, preserving chrominance with a weighted average. An initial weighted fusion is computed using the predefined contribution parameters (mu_1) and (mu_2), serving as the starting point for the iterative optimization. The core fusion algorithm minimizes an energy functional consisting of data fidelity terms for both inputs and a TV regularization term to maintain edges and structures. The Split-Bregman method is used to solve this non-differentiable optimization efficiently, performing iterative updates for the fused image, auxiliary gradient variables, and Bregman multipliers until convergence. During the simulation, edge maps and difference images are calculated to visually evaluate structural preservation and discrepancies with the original images. Pixel intensity profiles along the central row provide further insight into how features from both inputs contribute to the fusion. Fang et al. (2013) proposed a variational method for multisource remote-sensing image fusion [20]. The script also computes quantitative metrics, including PSNR and SSIM, and optionally generates SSIM maps to assess similarity with each input. Saliency or weight maps are derived from gradient magnitudes to indicate local contributions of each source image. The simulation supports parameter sweeps for (lambda), allowing exploration of regularization effects on smoothness and detail retention. Multiple figures are generated, including input images, initial fusion, final fused image, edge comparisons, difference maps, intensity profiles, convergence curves, histograms, and weight maps. All results are saved to a dedicated output folder for reproducibility and further analysis. The MATLAB implementation is modular, allowing easy adaptation to different datasets or color modalities. Timing and iteration information is displayed, providing performance insights. The script handles both grayscale and color inputs seamlessly, with visualization routines ensuring clarity of fusion results. Overall, the simulation demonstrates a robust, edge-preserving, and visually consistent approach for combining complementary information from multiple imaging modalities.

Figure 3 displays the first input image, representing the first modality (e.g., MRI, visible light, or any grayscale/color image). The image has been preprocessed with slight contrast adjustment to enhance visibility. If the original image is color, it is shown in full RGB; otherwise, the grayscale image is displayed. This figure serves as a reference for comparing structural and intensity information preserved in the fusion process. Visual inspection allows identification of prominent edges, textures, and regions of interest. The intensity distribution reflects the inherent features of the first modality. It provides a baseline for evaluating differences between the fused output and the original. Edge structures observed here are later compared in the edge map figure. This figure is crucial for understanding which features from modality–1 contribute to the fused result. It also demonstrates the effectiveness of preprocessing normalization applied to the image. Overall, it establishes the context for subsequent fusion and analysis steps.

You can download the Project files here: Download files now. (You must be logged in).

Figure 4 shows the second input image, representing a complementary imaging modality. Similar to Figure 1, it is either grayscale or color and has been contrast-normalized. This image provides features that may be absent or weak in the first modality. Observing edges, textures, and intensity variations allows users to anticipate which regions might dominate in the fused image. Differences between this modality and the first are critical for the fusion algorithm to preserve unique structural information. The visualization highlights the complementary nature of multimodal imaging. Comparing Figures 1 and 2 reveals areas where details might be enhanced by fusion. This figure also forms the basis for computing difference maps and gradient-based saliency maps. It helps in assessing the effectiveness of the fusion method in preserving information from both modalities. The content here is later quantitatively evaluated using PSNR and SSIM against the fused output.

Figure 5 presents the initial fusion obtained by a simple weighted average of the two input images, using the parameters (mu_1) and (mu_2). It serves as a starting point for the Split-Bregman optimization algorithm. This initial fusion preserves approximate intensities from both modalities but lacks edge preservation and structural refinement. Visual inspection reveals blurring or blending of features where the modalities differ significantly. The figure highlights regions where one modality dominates based on the weighting factors. It is important for demonstrating the improvement achieved after iterative TV-based fusion. The initial fusion also provides a reference for comparing difference maps before and after optimization. In color images, only the luminance channel is fused here, ensuring chroma consistency. It illustrates the importance of variational optimization for preserving sharp details. Overall, this figure gives a baseline for evaluating the convergence of the fusion process.

Figure 6 shows the final fused output after performing the variational TV-based optimization. This image integrates structural and intensity information from both modalities while preserving sharp edges and fine details. The TV regularization ensures smooth regions remain uniform without introducing artifacts. Differences between the fused image and the initial weighted fusion can be observed, particularly around edges and high-contrast regions. For color images, the fusion is applied to the luminance channel, while chroma channels are combined via weighted averaging. The figure demonstrates effective blending of complementary features from both input images. Visual comparison with Figures 1 and 2 allows evaluation of structural retention and contrast enhancement. This image serves as the main output for quantitative assessment using PSNR and SSIM. The overall quality indicates the effectiveness of the Split-Bregman iterative approach. It is the most critical figure for demonstrating the success of the proposed fusion methodology.

Figure 7 displays edge maps derived using the Canny operator for both input images and the fused result. This visualization emphasizes how well the fused image retains structural features from the original modalities. Edges from Modality–1 and Modality–2 can be directly compared with those in the fused image. The fused edge map demonstrates the preservation of prominent boundaries while suppressing irrelevant or noisy edges. Differences in edge density reveal which features from each modality are retained. This figure helps assess the ability of TV regularization to preserve critical structural information. The visualization highlights areas of overlap and unique features from each modality. Edge comparison is important for medical imaging and remote sensing, where structure is crucial. Overall, this figure provides a qualitative assessment of fusion performance. It complements quantitative metrics like SSIM that measure structural similarity.

Figure 8 shows the absolute differences between the fused image and each original input, indicating how much information has changed or been retained. Bright regions indicate significant differences, while darker regions suggest similarity. These maps help identify which features from each modality dominate in the fused result. They also reveal potential artifacts or areas where information might be lost. The difference maps are useful for both qualitative evaluation and algorithm debugging. They provide insight into the spatial distribution of fusion contributions from the two inputs. Observing these maps allows tuning of weighting parameters (mu_1) and (mu_2) for better balance. This figure complements edge maps by highlighting intensity changes rather than just boundaries. Overall, it provides a detailed assessment of fusion fidelity and structural preservation.

You can download the Project files here: Download files now. (You must be logged in).

 Figure 9 plots the intensity variation along the middle row of the images, comparing both inputs with the fused result. The plot illustrates how the fused image integrates features from each modality. Peaks and valleys correspond to edges and structural changes. Differences between the curves indicate areas dominated by one modality over the other. The fused profile typically smooths minor differences while retaining major peaks from both inputs. This analysis allows visual verification of edge preservation and intensity blending. It provides a quantitative sense of how TV fusion affects pixel-level intensities. The mid-row profile is representative but can be extended to other rows for further analysis. It helps in understanding the spatial contribution of each source image. Overall, this figure validates the effectiveness of the fusion algorithm at the pixel level.

Figure 10 illustrates the convergence behavior of the iterative TV fusion algorithm. The relative change in the fused image is plotted for each iteration, showing how quickly the solution stabilizes. A decreasing curve indicates successful minimization of the energy functional. The number of iterations required to reach the tolerance threshold provides insight into algorithm efficiency. This figure helps assess stability and performance of the Split-Bregman method. Sudden plateaus or spikes may indicate numerical issues or the need for parameter adjustment. Observing the convergence curve ensures that the fused image corresponds to a near-optimal solution. Faster convergence reduces computation time, which is important for large images. This figure complements the quantitative metrics by validating the iterative process. It demonstrates the robustness and reliability of the variational TV fusion approach.

Figure 11 visualizes the SSIM map between the fused image and the first input modality. Brighter regions correspond to areas with higher structural similarity, while darker regions indicate lower similarity. This figure highlights how well the fusion process preserves local structures from Modality–1. It complements quantitative SSIM metrics by showing spatial distribution of similarity rather than just a global value. Differences in intensity reflect the contribution of the second modality in the fused output. Areas with low similarity may indicate where unique features from Modality–2 have been integrated. This visualization is crucial in medical or remote sensing applications, where retaining structural details is important. Observing the SSIM map alongside edge maps and difference maps provides a comprehensive understanding of fusion quality. It also helps identify regions that may require parameter tuning. Overall, the figure validates the effectiveness of the TV-based fusion in preserving essential image structures.

Figure 12 displays the histograms of pixel intensities for both input images and the fused result. This allows visual inspection of how intensity distributions are combined in the fusion process. Peaks indicate dominant intensity levels, while the spread reflects contrast and dynamic range. Comparing histograms helps evaluate whether the fused image maintains brightness and contrast from both modalities. A balanced histogram indicates that no modality overly dominates the fused result. This figure also highlights potential artifacts, such as intensity clipping or loss of details in certain ranges. Observing histograms helps in tuning preprocessing, weighting parameters, or regularization strength. It provides a quantitative and visual complement to edge maps and SSIM metrics. Overall, this figure demonstrates the fusion method’s ability to integrate intensity information while preserving global image characteristics.

You can download the Project files here: Download files now. (You must be logged in).

Figure 13 visualizes the saliency or weight map derived from local gradient magnitudes and differences between the input images. Brighter regions indicate higher contribution from Modality–1, while darker regions correspond to higher influence of Modality–2. This figure helps understand the spatial distribution of information in the fused image. Regions with strong edges from a particular modality are typically weighted higher, preserving important structural features. The weight map provides insight into the fusion algorithm’s decision-making process on a per-pixel basis. It complements edge maps and difference images by highlighting how the algorithm balances contributions from each input. This visualization is useful for interpreting fusion results in applications where modality-specific features are critical. It also aids in evaluating the effectiveness of gradient-based weighting strategies. Overall, the figure illustrates the dynamic nature of the fusion and the local adaptation of the algorithm.

6. Results and Discussion:

The proposed variational TV-based multimodal image fusion successfully integrates complementary information from two distinct input images, producing a fused result that preserves structural and intensity features from both modalities. Visual inspection of the fused images (Figure 4) reveals enhanced clarity and sharpness compared to the initial weighted fusion (Figure 3), demonstrating the efficacy of the Split-Bregman iterative optimization. Edge preservation is evident from the comparison of Canny edge maps (Figure 5), where significant structural boundaries from both modalities are retained in the fused output. Difference maps (Figure 6) highlight regions dominated by each modality, confirming balanced contributions from both inputs. Pixel intensity profiles along the mid-row (Figure 7) indicate that the fused image maintains essential peaks and valleys from both sources, ensuring faithful representation of high-contrast structures. Convergence curves (Figure 8) confirm stable and efficient optimization, with relative changes diminishing to near-zero values within the specified tolerance. 19. Chan et al. (2015) discussed recent developments in TV-based algorithms for image processing [21]. You et al. (2017) presented a multimodal image fusion method using total generalized variation [22]. SSIM maps (Figure 9) demonstrate high structural similarity with each input, confirming that the algorithm effectively preserves local features. Histograms (Figure 10) illustrate balanced intensity distributions, indicating that global contrast and dynamic range are maintained without introducing artifacts. Saliency maps (Figure 11) provide insight into the spatial weighting of inputs, highlighting regions where one modality dominates due to stronger gradients. Quantitative metrics, including PSNR and SSIM, further confirm improved fusion quality, with the fused image showing higher structural fidelity than individual inputs. Geman and Geman (1984) discussed stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images [23]. The algorithm performs consistently for grayscale and color images, with optional fusion of the luminance channel preserving chromatic consistency.

Table 3: Edge and Difference Map Statistics.

Parameter sensitivity analysis indicates that the regularization parameter (lambda) controls edge sharpness and smoothness, allowing trade-offs between noise suppression and detail preservation. Weighted combination of chroma channels ensures color fidelity in RGB fusion. The method demonstrates robustness against input misalignments when images are pre-registered. Computational efficiency is adequate for moderate image sizes, with the Split-Bregman approach converging rapidly. Overall, the fusion process achieves a visually and quantitatively superior output. Figueiredo and Nowak (2003) proposed an EM algorithm for wavelet-based image restoration [24]. The algorithm effectively integrates complementary information, reduces redundancy, and enhances image interpretability. These results indicate that TV-based fusion is suitable for applications in medical imaging, remote sensing, and surveillance. The findings validate the utility of gradient-based saliency weighting in adaptive fusion. In conclusion, the proposed method provides a comprehensive framework for multimodal image fusion, balancing structural preservation, contrast enhancement, and computational efficiency.

7. Conclusion:

In this study, a variational Total Variation (TV) based multimodal image fusion method using the Split-Bregman approach was successfully implemented. The algorithm effectively integrates complementary features from multiple input modalities while preserving important structural details and intensity information. Fused images demonstrate improved clarity, edge preservation, and contrast compared to individual inputs. Quantitative metrics, including PSNR and SSIM, confirm superior structural fidelity and visual quality. Saliency and weight maps provide insight into the adaptive contribution of each modality across the image. Li (2012) compared multi-focus image fusion techniques [25]. The method supports both grayscale and color images, with optional fusion of the luminance channel ensuring color consistency. Parameter tuning allows control over edge sharpness and smoothness, enabling flexibility for different applications. Convergence analysis shows the approach is computationally efficient and stable. Overall, the proposed TV-based fusion framework provides a robust, reliable, and visually effective solution for multimodal image integration. These results highlight its potential utility in medical imaging, remote sensing, and other fields requiring enhanced image interpretability.

8. References:

[1] L. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D: Nonlinear Phenomena, vol. 60, no. 1–4, pp. 259–268, 1992.

[2] V. P. Gopi, P. Palanisamy, K. A. Wahid, and P. Babyn, “MR Image Reconstruction Based on Iterative Split Bregman Algorithm and Nonlocal Total Variation,” Computational and Mathematical Methods in Medicine, vol. 2013, Article ID 985819, 2013

[3] J. Yuan, B. Miles, G. Garvin, X.-C. Tai, and A. Fenster, “Efficient convex optimization approaches to variational image fusion,” Numerical Mathematics: Theory, Methods and Applications, 2015.

[4] F. Li and T. Zeng, “Variational image fusion with first and second-order gradient information,” Journal of Computational Mathematics, vol. 34, no. 2, pp. 200–222, 2016.

[5] L. Bungert, D. A. Coomes, M. J. Ehrhardt, J. Rasch, R. Reisenhofer, and C.-B. Schönlieb, “Blind image fusion for hyperspectral imaging with the directional total variation,” arXiv preprint arXiv:1710.05705, 2017.

[6] M. Simões, J. M. Bioucas-Dias, L. B. Almeida, and J. Chanussot, “Hyperspectral image superresolution: An edge‑preserving convex formulation,” arXiv preprint arXiv:1403.8098, 2014.

[7] “A combined first and second order variational approach for image reconstruction,” K. Papafitsoros and C.-B. Schönlieb, arXiv preprint arXiv:1202.6341, 2012.

[8] C. Chen, Y. Li, W. Liu, and J. Huang, “SIRF: Simultaneous image registration and fusion in a unified framework,” arXiv preprint arXiv:1411.5065, 2014.

[9] M. Zhu, S. J. Wright, and T. F. Chan, “Duality‑based algorithms for total‑variation‑regularized image restoration,” Computational Optimization and Applications, vol. 47, pp. 377–400, 2010.

[10] “Nonlocal total variation based image deblurring using Split Bregman method and fixed point iteration,” Applied Mechanics and Materials, vol. 333–335, pp. 875–882, 2013.

[11] “Total Variant Image Deblurring Based on Split Bregman Method,” J. Wang, K. Lü, N. He, and Q. Wang, Acta Electronica Sinica, vol. 40, no. 8, pp. 1503–1508, 2012.

[12] B. Huang et al., “A Review of Multimodal Medical Image Fusion Techniques,” BMC Medical Imaging, vol. 20, Article number 118, 2020.

[13] M. V. Srikanth, A. S. Suneel Kumar, B. Nagasirisha, and T. V. Venkata Lakshmi, “Medical Image Fusion using Multi‑Scale Guided Filtering,” i‑manager’s Journal on Pattern Recognition, vol. 11, no. 1, pp. 13–23, 2024.

[14] (Recent) “Multimodal medical image fusion combining saliency perception and generative adversarial network,” Scientific Reports, 2025.

[15] (Recent) “Multimodal medical image fusion based on interval gradients and convolutional neural networks,” BMC Medical Imaging, 2024.

[16] Y. Chen and T. Wunderli, “Adaptive total variation for image restoration in BV space,” Journal of Mathematical Analysis and Applications, vol. 272, no. 1, 2002.

[17] Y. Meyer, Oscillating Patterns in Image Processing and Nonlinear Evolution Equations, University Lecture Series, AMS, 2001.

[18] (Foundational) S. Osher, “Mumford–Shah and total variation based variational methods for imaging,” in Variational Methods in Image Segmentation, 2005.

[19] (Survey) J. Piella, “A review of image fusion quality models,” Proceedings of the Nordic Signal Processing Symposium (NORSIG), 2003.

[20] (Multisource fusion) F. Fang, M. Bertalmio, V. Caselles et al., “A variational method for multisource remote-sensing image fusion,” International Journal of Remote Sensing, 2013.

[21] T. Chan, S. Esedoglu, M. Nikolova, and F. López, “Recent developments in TV-based algorithms for image processing,” in Handbook of Mathematical Methods in Imaging, Springer, 2015.

[22] (Higher-order fusion) J. You, D. Tao, X. Gao et al., “Multimodal image fusion using total generalized variation,” Signal Processing: Image Communication, 2017.

[23] S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, no. 6, pp. 721–741, 1984.

[24] M. A. T. Figueiredo and R. D. Nowak, “An EM algorithm for wavelet-based image restoration,” IEEE Trans. Image Processing, vol. 12, no. 8, pp. 906–916, 2003.

[25] (Benchmarking & comparison) Z. Li, “A comparative study of multi-focus image fusion techniques,” International Journal of Computer Applications, vol. 50, no. 12, pp. 1–9, 2012.

You can download the Project files here: Download files now. (You must be logged in).

Keywords: Multimodal image fusion, Variational total variation, Split-Bregman optimization, Edge preservation, Structural similarity index, Peak signal-to-noise ratio, MATLAB simulation, YCbCr color fusion, Luminance channel fusion, Image regularization, Convergence analysis, Gradient-based weighting, Difference mapping, Quantitative performance evaluation, Computational imaging.

Do you need help with Design and Evaluation of a Variational Multimodal Image Fusion Algorithm in MATLAB? Don’t hesitate to contact our Tutors to receive professional and reliable guidance.