QR Decomposition with Householder Reflections

The more common approach to QR decomposition is employing Householder reflections rather than utilizing Gram-Schmidt. In practice, the Gram-Schmidt procedure is not recommended as it can lead to cancellation that causes inaccuracy of the computation of [latex]q_j[/latex], which may result in a non-orthogonal [latex]Q[/latex] matrix. Householder reflections are another method...

QR Decomposition with the Gram-Schmidt Algorithm

QR decomposition is another technique for decomposing a matrix into a form that is easier to work with in further applications. The QR decomposition technique decomposes a square or rectangular matrix, which we will denote as [latex]A[/latex], into two components, [latex]Q[/latex], and [latex]R[/latex]. [latex display="true"] A = QR [/latex] Where [latex]Q[/latex] is...

Image Compression with Singular Value Decomposition

As mentioned in a previous post, image compression with singular value decomposition is a frequently occurring application of the method. The image is treated as a matrix of pixels with corresponding color values and is decomposed into smaller ranks that retain only the essential information that comprises the image. In...

Singular Value Decomposition in R

Following from a previous post on the Cholesky decomposition of a matrix, I wanted to explore another often used decomposition method known as Singular Value Decomposition, also called SVD. SVD underpins many statistical and real-world applications principal component analysis, image compression, noise reduction of an image, and even climate studies....

How to Calculate Eigenvalues and Eigenvectors Manually and with R

Eigenvalues and eigenvectors prominently appear in many statistical and other computational fields that require transformations of linear systems or are interested in the evolution of systems from an initial point. Some examples of the applications of eigenvalues and eigenvectors include Google’s PageRank algorithm, image compression, compounding interest, Markov Chains, and...

The Matrix Trace in R and Some Properties of the Trace

Although comparatively straightforward in nature, the matrix trace has many properties related to other matrix operations and often appears in statistical methods such as maximum likelihood estimation of the covariance matrix of a multivariate normal distribution due to its usefulness in computing the derivatives of quadratic forms. One such property...

Cholesky Decomposition of a Positive-Definite Matrix

Cholesky decomposition, also known as Cholesky factorization, is a method of decomposing a positive-definite matrix. A positive-definite matrix is defined as a symmetric matrix where for all possible vectors [latex]x[/latex], [latex]x'Ax > 0[/latex]. Cholesky decomposition and other decomposition methods are important as it is not often feasible to perform matrix...

How to Calculate the Inverse Matrix for 2×2 and 3×3 Matrices

Inverses of Numbers and Matrices The inverse of a number is its reciprocal. For example, the inverse of 8 is [latex]\frac{1}{8}[/latex], the inverse of 20 is [latex]\frac{1}{20}[/latex] and so on. Therefore, a number multiplied by its inverse will always equal 1. An inverse of a number is denoted with a [latex]-1[/latex]...