Abstract
This work revisits the hyperspectral super-resolution (HSR) problem, i.e., fusing a pair of spatially co-registered hyperspectral (HSI) and multispectral (MSI) images to recover a super-resolution image (SRI) that enhances the spatial resolution of the HSI. Coupled tensor decomposition (CTD)-based methods have gained traction in this domain, offering recoverability guarantees under various assumptions. Existing models such as canonical polyadic decomposition (CPD) and Tucker decomposition provide strong expressive power but lack physical interpretability. The block-term decomposition model with rank-$(L_r, L_r, 1)$ terms (the LL1 model) yields interpretable factors under the linear mixture model (LMM) of spectral images, but LMM assumptions are often violated in practice -- primarily due to nonlinear effects such as endmember variability (EV). To address this, we propose modeling spectral images using a more flexible block-term tensor decomposition with rank-$(L_r, M_r, N_r)$ terms (the LMN model). This modeling choice retains interpretability, subsumes CPD, Tucker, and LL1 as special cases, and robustly accounts for non-ideal effects such as EV, offering a balanced tradeoff between expressiveness and interpretability for HSR. Importantly, under the LMN model for HSI and MSI, recoverability of the SRI can still be established under proper conditions -- providing strong theoretical support. Extensive experiments on synthetic and real datasets further validate the effectiveness and robustness of the proposed method compared with existing CTD-based approaches.
Summary
This paper addresses the hyperspectral super-resolution (HSR) problem, which involves fusing spatially co-registered hyperspectral (HSI) and multispectral (MSI) images to generate a super-resolution image (SRI) with enhanced spatial resolution compared to the HSI. The authors argue that existing coupled tensor decomposition (CTD)-based methods, while offering recoverability guarantees, often struggle with the trade-off between physical interpretability and expressiveness, particularly when dealing with endmember variability (EV). To overcome this limitation, they propose a new CTD model based on block-term decomposition with rank-(L_r, M_r, N_r) terms (the LMN model). The LMN model aims to retain interpretability while being more flexible in capturing non-ideal effects like EV. The authors demonstrate that the LMN model subsumes existing models like CPD, Tucker, and LL1 as special cases. Importantly, they provide a theoretical analysis establishing the recoverability of the SRI under the LMN model, even in the presence of EV across spectral pixels, which is a significant improvement over existing methods that primarily consider EV between HSI and MSI. Furthermore, they designed regularization based on the physical interpretations of the LMN latent factors and proposed a first-order inexact block coordinate descent algorithm to solve the resulting constrained CTD problem. Extensive experiments on both synthetic and real datasets validate the effectiveness and robustness of the proposed method.
Key Insights
- •Novel LMN Model: The paper introduces the LMN model for representing spectral images, which balances expressiveness and interpretability better than existing CPD, Tucker, and LL1 models, particularly in the presence of endmember variability (EV). It models the spectral signature variations using a low multilinear rank Tucker tensor.
- •Recoverability Guarantee: The authors provide theoretical proof that the SRI can be uniquely recovered under the LMN model with reasonable conditions, specifically when I_H*J_H >= LMR, I_M >= LR, J_M >= MR, and LM >= N >= max{⌈L/M⌉+⌈M/L⌉,3} for all r ∈ [R].
- •New Identifiability Result: The paper presents a new essential uniqueness condition for the LMN decomposition (Theorem 3.2), which is crucial for proving the SRI recoverability and is tailored to the HSR setting, overcoming limitations of existing LMN decomposition conditions. This condition only requires that *R* (number of endmembers) and *L,M,N* are sufficiently small.
- •Regularization Design: The authors leverage the physical interpretation of the LMN latent factors to design model-based constraints and regularization terms, specifically spatial and spectral smoothness regularization, to improve HSR performance.
- •Algorithm: A first-order inexact block coordinate descent algorithm (CLIMB) is proposed to solve the constrained CTD problem with the LMN model. Convergence to stationary points is guaranteed under proper choices of step size and extrapolation parameters, with an iteration complexity of O(1/T).
- •Numerical Validation: Experiments using real hyperspectral images (Urban and Pavia University datasets) demonstrate that the LMN model achieves smaller fitting errors (up to 50% lower NRE than LL1) compared to CPD, Tucker, and LL1 models with similar amounts of parameters.
Practical Implications
- •Improved Hyperspectral Super-Resolution: The proposed LMN-based CTD method (CLIMB) offers a more robust and accurate solution for HSR, particularly in scenarios with significant endmember variability, leading to better image quality and analysis capabilities.
- •Real-world Applications: This research benefits applications relying on high-resolution hyperspectral imagery, such as precision agriculture, environmental monitoring, mineral exploration, and urban planning, where accurate spectral information is crucial for decision-making.
- •Practitioners/Engineers: Practitioners can implement the CLIMB algorithm using the provided guidelines and apply it to fuse HSI and MSI data, leveraging the physical interpretations of the LMN model for regularization and improved performance.
- •Future Research: Future research can explore incorporating other regularization terms or constraints that reflect prior information of the LMN components, such as sparsity, nonlocal self-similarity, and nonnegativity, to further enhance the performance and robustness of the HSR method. The developed essential uniqueness condition for LMN decomposition could also be of broader interest beyond HSR.