QR Decomposition with Householder Reflections

The more common approach to QR decomposition is employing Householder reflections rather than utilizing Gram-Schmidt. In practice, the Gram-Schmidt procedure is not recommended as it can lead to cancellation that causes inaccuracy of the computation of [latex]q_j[/latex], which may result in a non-orthogonal [latex]Q[/latex] matrix. Householder reflections are another method...

QR Decomposition with the Gram-Schmidt Algorithm

QR decomposition is another technique for decomposing a matrix into a form that is easier to work with in further applications. The QR decomposition technique decomposes a square or rectangular matrix, which we will denote as [latex]A[/latex], into two components, [latex]Q[/latex], and [latex]R[/latex]. [latex display="true"] A = QR [/latex] Where [latex]Q[/latex] is...

Hierarchical Clustering Nearest Neighbors Algorithm in R

Hierarchical clustering is a widely used and popular tool in statistics and data mining for grouping data into ‘clusters’ that exposes similarities or dissimilarities in the data. There are many approaches to hierarchical clustering as it is not possible to investigate all clustering possibilities. One set of approaches to hierarchical...

Iterated Principal Factor Method of Factor Analysis with R

The iterated principal factor method is an extension of the principal factor method that seeks improved estimates of the communality. As seen in the previous post on the principal factor method, initial estimates of [latex]R - \hat{\Psi}[/latex] or [latex]S - \hat{\Psi}[/latex] are found to obtain [latex]\hat{\Lambda}[/latex] from which the factors...

Factor Analysis with the Principal Factor Method and R

As discussed in a previous post on the principal component method of factor analysis, the [latex]\hat{\Psi}[/latex] term in the estimated covariance matrix [latex]S[/latex], [latex]S = \hat{\Lambda} \hat{\Lambda}' + \hat{\Psi}[/latex], was excluded and we proceeded directly to factoring [latex]S[/latex] and [latex]R[/latex]. The principal factor method of factor analysis (also called the...

Factor Analysis with the Principal Component Method Part Two

In the first post on factor analysis, we examined computing the estimated covariance matrix [latex]S[/latex] of the rootstock data and proceeded to find two factors that fit most of the variance of the data using the principal component method. However, the variables in the data are not on the same...

Factor Analysis Introduction with the Principal Component Method and R

Factor analysis is a controversial technique that represents the variables of a dataset [latex]y_1, y_2, \cdots, y_p[/latex] as linearly related to random, unobservable variables called factors, denoted [latex]f_1, f_2, \cdots, f_m[/latex] where [latex]m \lt p[/latex]. The factors are representative of ‘latent variables’ underlying the original variables. The existence of the...

Image Compression with Principal Component Analysis

Image compression with principal component analysis is a frequently occurring application of the dimension reduction technique. Recall from a previous post that employed singular value decomposition to compress an image, that an image is a matrix of pixels represented by RGB color values. Thus, principal component analysis can be used...

Principal Component Analysis

Often, it is not helpful or informative to only look at all the variables in a dataset for correlations or covariances. A preferable approach is to derive new variables from the original variables that preserve most of the information given by their variances. Principal component analysis is a widely used...

Quadratic Discriminant Analysis of Several Groups

Quadratic discriminant analysis for classification is a modification of linear discriminant analysis that does not assume equal covariance matrices amongst the groups [latex](\Sigma_1, \Sigma_2, \cdots, \Sigma_k)[/latex]. Similar to LDA for several groups, quadratic discriminant analysis of several groups classification seeks to find the group that maximizes the quadratic classification function...