Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation / Edition 1

Hardcover (Print)
Used and New from Other Sellers
Used and New from Other Sellers
from $115.67
Usually ships in 1-2 business days
(Save 29%)
Other sellers (Hardcover)
  • All (7) from $115.67   
  • New (5) from $121.59   
  • Used (2) from $115.67   


This book provides a broad survey of models and efficient algorithms for Nonnegative Matrix Factorization (NMF). This includes NMF’s various extensions and modifications, especially Nonnegative Tensor Factorizations (NTF) and Nonnegative Tucker Decompositions (NTD). NMF/NTF and their extensions are increasingly used as tools in signal and image processing, and data analysis, having garnered interest due to their capability to provide new insights and relevant information about the complex latent relationships in experimental data sets. It is suggested that NMF can provide meaningful components with physical interpretations; for example, in bioinformatics, NMF and its extensions have been successfully applied to gene expression, sequence analysis, the functional characterization of genes, clustering and text mining. As such, the authors focus on the algorithms that are most useful in practice, looking at the fastest, most robust, and suitable for large-scale models.

Key features:

  • Acts as a single source reference guide to NMF, collating information that is widely dispersed in current literature, including the authors’ own recently developed techniques in the subject area.
  • Uses generalized cost functions such as Bregman, Alpha and Beta divergences, to present practical implementations of several types of robust algorithms, in particular Multiplicative, Alternating Least Squares, Projected Gradient and Quasi Newton algorithms.
  • Provides a comparative analysis of the different methods in order to identify approximation error and complexity.
  • Includes pseudo codes and optimized MATLAB source codes for almost all algorithms presented in the book.

The increasing interest in nonnegative matrix and tensor factorizations, as well as decompositions and sparse representation of data, will ensure that this book is essential reading for engineers, scientists, researchers, industry practitioners and graduate students across signal and image processing; neuroscience; data mining and data analysis; computer science; bioinformatics; speech processing; biomedical engineering; and multimedia.

Read More Show Less

Editorial Reviews

From the Publisher
"[A] focus on the algorithms that are most useful in practice and aim to derive and implement, in MATLAB, efficient and simple iterative algorithms that work with real-world data." (Book News, December 2009)
Read More Show Less

Product Details

  • ISBN-13: 9780470746660
  • Publisher: Wiley
  • Publication date: 11/16/2009
  • Edition number: 1
  • Pages: 500
  • Product dimensions: 6.70 (w) x 9.90 (h) x 1.20 (d)

Table of Contents



Glossary of Symbols and Abbreviations.

1 Introduction – Problem Statements and Models.

1.1 Blind Source Separation and Linear Generalized ComponentAnalysis.

1.2 Matrix Factorization Models with Nonnegativity and SparsityConstraints.

1.2.1 Why Nonnegativity and Sparsity Constraints?

1.2.2 Basic NMF Model.

1.2.3 Symmetric NMF.

1.2.4 Semi-Orthogonal NMF.

1.2.5 Semi-NMF and Nonnegative Factorization of ArbitraryMatrix.

1.2.6 Three-factor NMF.

1.2.7 NMF with Offset (Affine NMF).

1.2.8 Multi-layer NMF.

1.2.9 Simultaneous NMF.

1.2.10 Projective and Convex NMF.

1.2.11 Kernel NMF.

1.2.12 Convolutive NMF.

1.2.13 Overlapping NMF.

1.3 Basic Approaches to Estimate Parameters of Standard NMF.

1.3.1 Large-scale NMF.

1.3.2 Non-uniqueness of NMF and Techniques to Alleviate theAmbiguity Problem.

1.3.3 Initialization of NMF.

1.3.4 Stopping Criteria.

1.4 Tensor Properties and Basis of Tensor Algebra.

1.4.1 Tensors (Multi-way Arrays) – Preliminaries.

1.4.2 Subarrays, Tubes and Slices.

1.4.3 Unfolding – Matricization.

1.4.4 Vectorization.

1.4.5 Outer, Kronecker, Khatri-Rao and Hadamard Products.

1.4.6 Mode-n Multiplication of Tensor by Matrix andTensor by Vector, Contracted Tensor Product.

1.4.7 Special Forms of Tensors.

1.5 Tensor Decompositions and Factorizations.

1.5.1 Why Multi-way Array Decompositions and Factorizations?

1.5.2 PARAFAC and Nonnegative Tensor Factorization.

1.5.3 NTF1 Model.

1.5.4 NTF2 Model.

1.5.5 Individual Differences in Scaling (INDSCAL) and ImplicitSlice Canonical Decomposition Model (IMCAND).

1.5.6 Shifted PARAFAC and Convolutive NTF.

1.5.7 Nonnegative Tucker Decompositions.

1.5.8 Block Component Decompositions.

1.5.9 Block-Oriented Decompositions.

1.5.10 PARATUCK2 and DEDICOM Models.

1.5.11 Hierarchical Tensor Decomposition.

1.6 Discussion and Conclusions.

2 Similarity Measures and Generalized Divergences.

2.1 Error-induced Distance and Robust Regression Techniques.

2.2 Robust Estimation.

2.3 Csiszár Divergences.

2.4 Bregman Divergence.

2.4.1 Bregman Matrix Divergences.

2.5 Alpha-Divergences.

2.5.1 Asymmetric Alpha-Divergences.

2.5.2 Symmetric Alpha-Divergences.

2.6 Beta-Divergences.

2.7 Gamma-Divergences.

2.8 Divergences Derived from Tsallis and Rényi Entropy.

2.8.1 Concluding Remarks.

3 Multiplicative Iterative Algorithms for NMF with SparsityConstraints.

3.1 Extended ISRA and EMML Algorithms: Regularization andSparsity.

3.1.1 Multiplicative NMF Algorithms Based on the SquaredEuclidean Distance.

3.1.2 Multiplicative NMF Algorithms Based on Kullback-LeiblerI-Divergence.

3.2 Multiplicative Algorithms Based on Alpha-Divergence.

3.2.1 Multiplicative Alpha NMF Algorithm.

3.2.2 Generalized Multiplicative Alpha NMF Algorithms.

3.3 Alternating SMART: Simultaneous Multiplicative AlgebraicReconstruction Technique.

3.3.1 Alpha SMART Algorithm.

3.3.2 Generalized SMART Algorithms.

3.4 Multiplicative NMF Algorithms Based on Beta-Divergence.

3.4.1 Multiplicative Beta NMF Algorithm.

3.4.2 Multiplicative Algorithm Based on the Itakura-SaitoDistance.

3.4.3 Generalized Multiplicative Beta Algorithm for NMF.

3.5 Algorithms for Semi-orthogonal NMF and OrthogonalThree-Factor NMF.

3.6 Multiplicative Algorithms for Affine NMF.

3.7 Multiplicative Algorithms for Convolutive NMF.

3.7.1 Multiplicative Algorithm for Convolutive NMF Based onAlpha-Divergence.

3.7.2 Multiplicative Algorithm for Convolutive NMF Based onBeta-Divergence.

3.7.3 Efficient Implementation of CNMF Algorithm.

3.8 Simulation Examples for Standard NMF.

3.9 Examples for Affine NMF.

3.10 Music Analysis and Decomposition Using Convolutive NMF.

3.11 Discussion and Conclusions.

4 Alternating Least Squares and Related Algorithms for NMFand SCA Problems.

4.1 Standard ALS Algorithm.

4.1.1 Multiple Linear Regression – Vectorized Version ofALS Update Formulas.

4.1.2 Weighted ALS.

4.2 Methods for Improving Performance and Convergence Speed ofALS Algorithms.

4.2.1 ALS Algorithm for Very Large-scale NMF.

4.2.2 ALS Algorithm with Line-Search.

4.2.3 Acceleration of ALS Algorithm via SimpleRegularization.

4.3 ALS Algorithm with Flexible and Generalized RegularizationTerms.

4.3.1 ALS with Tikhonov Type Regularization Terms.

4.3.2 ALS Algorithms with Sparsity Control andDecorrelation.

4.4 Combined Generalized Regularized ALS Algorithms.

4.5 Wang-Hancewicz Modified ALS Algorithm.

4.6 Implementation of Regularized ALS Algorithms for NMF.

4.7 HALS Algorithm and its Extensions.

4.7.1 Projected Gradient Local Hierarchical Alternating LeastSquares (HALS) Algorithm.

4.7.2 Extensions and Implementations of the HALS Algorithm.

4.7.3 Fast HALS NMF Algorithm for Large-scale Problems.

4.7.4 HALS NMF Algorithm with Sparsity, Smoothness andUncorrelatedness Constraints.

4.7.5 HALS Algorithm for Sparse Component Analysis and FlexibleComponent Analysis.

4.7.6 Simplified HALS Algorithm for Distributed and Multi-taskCompressed Sensing.

4.7.7 Generalized HALS-CS Algorithm.

4.7.8 Generalized HALS Algorithms Using Alpha-Divergence.

4.7.9 Generalized HALS Algorithms Using Beta-Divergence.

4.8 Simulation Results.

4.8.1 Underdetermined Blind Source Separation Examples.

4.8.2 NMF with Sparseness, Orthogonality and SmoothnessConstraints.

4.8.3 Simulations for Large-scale NMF.

4.8.4 Illustrative Examples for Compressed Sensing.

4.9 Discussion and Conclusions.

5 Projected Gradient Algorithms.

5.1 Oblique Projected Landweber (OPL) Method.

5.2 Lin’s Projected Gradient (LPG) Algorithm with ArmijoRule.

5.3 Barzilai-Borwein Gradient Projection for SparseReconstruction (GPSR-BB).

5.4 Projected Sequential Subspace Optimization (PSESOP).

5.5 Interior Point Gradient (IPG) Algorithm.

5.6 Interior Point Newton (IPN) Algorithm.

5.7 Regularized Minimal Residual Norm Steepest Descent Algorithm(RMRNSD).

5.8 Sequential Coordinate-Wise Algorithm (SCWA).

5.9 Simulations.

5.10 Discussions.

6 Quasi-Newton Algorithms for Nonnegative MatrixFactorization.

6.1 Projected Quasi-Newton Optimization.

6.1.1 Projected Quasi-Newton for Frobenius Norm.

6.1.2 Projected Quasi-Newton for Alpha-Divergence.

6.1.3 Projected Quasi-Newton for Beta-Divergence.

6.1.4 Practical Implementation.

6.2 Gradient Projection Conjugate Gradient.

6.3 FNMA algorithm.

6.4 NMF with Quadratic Programming.

6.4.1 Nonlinear Programming.

6.4.2 Quadratic Programming.

6.4.3 Trust-region Subproblem.

6.4.4 Updates for A.

6.5 Hybrid Updates.

6.6 Numerical Results.

6.7 Discussions.

7 Multi-Way Array (Tensor) Factorizations andDecompositions.

7.1 Learning Rules for the Extended Three-way NTF1 Problem.

7.1.1 Basic Approaches for the Extended NTF1 Model.

7.1.2 ALS Algorithms for NTF1.

7.1.3 Multiplicative Alpha and Beta Algorithms for the NTF1Model.

7.1.4 Multi-layer NTF1 Strategy.

7.2 Algorithms for Three-way Standard and Super SymmetricNonnegative Tensor Factorization.

7.2.1 Multiplicative NTF Algorithms Based on Alpha- andBeta-Divergences.

7.2.2 Simple Alternative Approaches for NTF and SSNTF.

7.3 Nonnegative Tensor Factorizations for Higher-OrderArrays.

7.3.1 Alpha NTF Algorithm.

7.3.2 Beta NTF Algorithm.

7.3.3 Fast HALS NTF Algorithm Using Squared EuclideanDistance.

7.3.4 Generalized HALS NTF Algorithms Using Alpha- andBeta-Divergences.

7.3.5 Tensor Factorization with Additional Constraints.

7.4 Algorithms for Nonnegative and Semi-Nonnegative TuckerDecompositions.

7.4.1 Higher Order SVD (HOSVD) and Higher Order OrthogonalIteration (HOOI) Algorithms.

7.4.2 ALS Algorithm for Nonnegative Tucker Decomposition.

7.4.3 HOSVD, HOOI and ALS Algorithms as Initialization Tools forNonnegative Tensor Decomposition.

7.4.4 Multiplicative Alpha Algorithms for Nonnegative TuckerDecomposition.

7.4.5 Beta NTD Algorithm.

7.4.6 Local ALS Algorithms for Nonnegative TUCKERDecompositions.

7.4.7 Semi-Nonnegative Tucker Decomposition.

7.5 Nonnegative Block-Oriented Decomposition.

7.5.1 Multiplicative Algorithms for NBOD.

7.6 Multi-level Nonnegative Tensor Decomposition - High AccuracyCompression and Approximation.

7.7 Simulations and Illustrative Examples.

7.7.1 Experiments for Nonnegative Tensor Factorizations.

7.7.2 Experiments for Nonnegative Tucker Decomposition.

7.7.3 Experiments for Nonnegative Block-OrientedDecomposition.

7.7.4 Multi-Way Analysis of High Density Array EEG –Classification of Event Related Potentials.

7.7.5 Application of Tensor Decompositions in Brain ComputerInterface – Classification of Motor Imagery Tasks.

7.7.6 Image and Video Applications.

7.8 Discussion and Conclusions.

8 Selected Applications.

8.1 Clustering.

8.1.1 Semi-Binary NMF.

8.1.2 NMF vs. Spectral Clustering.

8.1.3 Clustering with Convex NMF.

8.1.4 Application of NMF to Text Mining.

8.1.5 Email Surveillance.

8.2 Classification.

8.2.1 Musical Instrument Classification.

8.2.2 Image Classification.

8.3 Spectroscopy.

8.3.1 Raman Spectroscopy.

8.3.2 Fluorescence Spectroscopy.

8.3.3 Hyperspectral Imaging.

8.3.4 Chemical Shift Imaging.

8.4 Application of NMF for Analyzing Microarray Data.

8.4.1 Gene Expression Classification.

8.4.2 Analysis of Time Course Microarray Data.



Read More Show Less

Customer Reviews

Be the first to write a review
( 0 )
Rating Distribution

5 Star


4 Star


3 Star


2 Star


1 Star


Your Rating:

Your Name: Create a Pen Name or

Barnes & Review Rules

Our reader reviews allow you to share your comments on titles you liked, or didn't, with others. By submitting an online review, you are representing to Barnes & that all information contained in your review is original and accurate in all respects, and that the submission of such content by you and the posting of such content by Barnes & does not and will not violate the rights of any third party. Please follow the rules below to help ensure that your review can be posted.

Reviews by Our Customers Under the Age of 13

We highly value and respect everyone's opinion concerning the titles we offer. However, we cannot allow persons under the age of 13 to have accounts at or to post customer reviews. Please see our Terms of Use for more details.

What to exclude from your review:

Please do not write about reviews, commentary, or information posted on the product page. If you see any errors in the information on the product page, please send us an email.

Reviews should not contain any of the following:

  • - HTML tags, profanity, obscenities, vulgarities, or comments that defame anyone
  • - Time-sensitive information such as tour dates, signings, lectures, etc.
  • - Single-word reviews. Other people will read your review to discover why you liked or didn't like the title. Be descriptive.
  • - Comments focusing on the author or that may ruin the ending for others
  • - Phone numbers, addresses, URLs
  • - Pricing and availability information or alternative ordering information
  • - Advertisements or commercial solicitation


  • - By submitting a review, you grant to Barnes & and its sublicensees the royalty-free, perpetual, irrevocable right and license to use the review in accordance with the Barnes & Terms of Use.
  • - Barnes & reserves the right not to post any review -- particularly those that do not follow the terms and conditions of these Rules. Barnes & also reserves the right to remove any review at any time without notice.
  • - See Terms of Use for other conditions and disclaimers.
Search for Products You'd Like to Recommend

Recommend other products that relate to your review. Just search for them below and share!

Create a Pen Name

Your Pen Name is your unique identity on It will appear on the reviews you write and other website activities. Your Pen Name cannot be edited, changed or deleted once submitted.

Your Pen Name can be any combination of alphanumeric characters (plus - and _), and must be at least two characters long.

Continue Anonymously

    If you find inappropriate content, please report it to Barnes & Noble
    Why is this product inappropriate?
    Comments (optional)