In the applied math world, there has been a recent surge of interest in tensor decompositions (see Kolda and Bader [2009] for a review). Tensor decompositions seek to find simple or low-dimensional representations of tensor-valued data to uncover meaningful patterns. In Gerard and Hoff [2016], we develop reformulations of common matrix decompositions as constrained minimization problems to create tensor analogues to the LQ and Polar decompositions, as well as a novel form the singular value decomposition (SVD).

The LQ decomposition of \(X \in \mathbb{R}^{p\times n}\) may be written as \(X = \ell LQI_n\), where \(\ell > 0\), \(L\) is \(p\) by \(p\) lower triangular with positive diagonal elements and unit determinant, \(I_n\) is the \(n\) by \(n\) identity matrix, and \(Q\) has orthonormal rows. Our higher-order LQ decomposition (HOLQ) takes the form \(\ell(L_1,\ldots,L_K,I_n)\cdot Q\) where \(\ell > 0\), each \(L_k\) is \(p_k\) by \(p_k\) lower triangular with positive diagonal elements and unit determinant, and \(Q \in \mathbb{R}^{p_1\times\cdots\times p_K \times n}\) has certain orthogonality properties which generalize the orthonormal rows property in the LQ decomposition. One application that we found for the HOLQ was in likelihood estimation and testing for the multilinear normal model: The MLE of each \(\Sigma_k\) in the multilinear normal model is \(L_kL_k^T\) in the HOLQ, and the form of likelihood ratio test statistics in the multilinear normal model may be represented as the ratio of the scale parameters, \(\ell\), from two sub-decompositions of the HOLQ that we called "HOLQ juniors". I like this connection to likelihood inference because it allows for a dual interpretation for the components of the HOLQ: The applied math interpretation of accounting for the heterogeneity in a tensor in terms of a constrained least-squares optimization, and the statistical interpretation of finding estimators in a parametric model.

For our generalization of the SVD, recall that the SVD may be written as \(X = \ell UDV^T\), where \(\ell > 0\), \(D\) is diagonal with unit determinant, and \(U\) and \(V\) both have orthonormal columns. Our higher-order version of the SVD takes the form \(X = \ell(U_1,\ldots,U_K)\cdot[(D_1,\ldots,D_K)\cdot V]\) where \(\ell > 0\), each \(U_k\) is \(p_k\) by \(p_k\) orthogonal, each \(D_k\) is diagonal with unit determinant, and \(V \in \mathbb{R}^{p_1\times\cdots\times p_K}\) also has certain orthogonality properties which generalize the orthonormal properties of the SVD. Unlike other tensor SVD's, this version separates the "singular values" from the "core tensor", \(V\), which allows for a more interpretable tensor SVD. I believe this tensor SVD might have applications to optimal mean or covariance estimation for tensor-valued data.


References

Gerard, D., & Hoff, P. (2016). A higher-order LQ decomposition for separable covariance models. Linear Algebra and its Applications, 505, p. 57–84. doi:10.1016/j.laa.2016.04.033 | arXiv:1410.1094

Kolda, T. G., & Bader, B. W. (2009). Tensor decompositions and applications. SIAM review, 51(3), p. 455-500. doi:10.1137/07070111X