Combined Linear Congruential Generators with R

Combined linear congruential generators, as the name implies, are a type of PRNG (pseudorandom number generator) that combine two or more LCGs (linear congruential generators). The combination of two or...

Multiplicative Congruential Generators in R

Multiplicative congruential generators, also known as Lehmer random number generators, is a type of linear congruential generator for generating pseudorandom numbers in [latex]U(0, 1)[/latex]. The multiplicative congruential generator, often abbreviated...

Linear Congruential Generator in R

A Linear congruential generator (LCG) is a class of pseudorandom number generator (PRNG) algorithms used for generating sequences of random-like numbers. The generation of random numbers plays a large role...

Simpson’s Rule for Approximating Definite Integrals in R

Simpson’s rule is another closed Newton-Cotes formula for approximating integrals over an interval with equally spaced nodes. Unlike the trapezoidal rule, which employs straight lines to approximate a definite integral,...

The Trapezoidal Rule of Numerical Integration in R

The Trapezoidal Rule is another of Closed Newton-Cotes formulas for approximating the definite integral of a function. The trapezoidal rule is so named due to the area approximated under the...

Numerical Differentiation with Finite Differences in R

Numerical differentiation is a method of approximating the derivative of a function [latex]f[/latex] at particular value [latex]x[/latex]. Often, particularly in physics and engineering, a function may be too complicated to...

Neville’s Method of Polynomial Interpolation

Neville’s method evaluates a polynomial that passes through a given set of [latex]x[/latex] and [latex]y[/latex] points for a particular [latex]x[/latex] value using the Newton polynomial form. Neville’s method is similar...

Lagrangian Polynomial Interpolation with R

Polynomial interpolation is the method of determining a polynomial that fits a set of given points. There are several approaches to polynomial interpolation, of which one of the most well...

Algebra of Sets in R

The set operations, union and intersection, the relative complement [latex]-[/latex] and the inclusion relation (subsets) [latex]\subseteq[/latex] are known as the algebra of sets. The algebra of sets can be used...

Set Operations Unions and Intersections in R

The set operations of unions and intersections should ring a bell for those who’ve worked with relational databases and Venn Diagrams. The ‘union’ of two of sets [latex]A[/latex] and [latex]B[/latex]...

Introduction to Set Theory and Sets with R

Sets define a ‘collection’ of objects, or things typically referred to as ‘elements’ or ‘members.’ The concept of sets arises naturally when dealing with any collection of objects, whether it...

QR Decomposition with Householder Reflections

The more common approach to QR decomposition is employing Householder reflections rather than utilizing Gram-Schmidt. In practice, the Gram-Schmidt procedure is not recommended as it can lead to cancellation that...

QR Decomposition with the Gram-Schmidt Algorithm

QR decomposition is another technique for decomposing a matrix into a form that is easier to work with in further applications. The QR decomposition technique decomposes a square or rectangular...

Factor Analysis with the Principal Factor Method and R

As discussed in a previous post on the principal component method of factor analysis, the [latex]\hat{\Psi}[/latex] term in the estimated covariance matrix [latex]S[/latex], [latex]S = \hat{\Lambda} \hat{\Lambda}' + \hat{\Psi}[/latex], was...

Image Compression with Principal Component Analysis

Image compression with principal component analysis is a frequently occurring application of the dimension reduction technique. Recall from a previous post that employed singular value decomposition to compress an image,...

Principal Component Analysis

Often, it is not helpful or informative to only look at all the variables in a dataset for correlations or covariances. A preferable approach is to derive new variables from...

Quadratic Discriminant Analysis of Several Groups

Quadratic discriminant analysis for classification is a modification of linear discriminant analysis that does not assume equal covariance matrices amongst the groups [latex](\Sigma_1, \Sigma_2, \cdots, \Sigma_k)[/latex]. Similar to LDA for...

LDA for Classification into Several Groups

Similar to the two-group linear discriminant analysis for classification case, LDA for classification into several groups seeks to find the mean vector that the new observation [latex]y[/latex] is closest to...

Quadratic Discriminant Analysis of Two Groups

As mentioned in the post on classification with linear discriminant analysis, LDA assumes the groups in question have equal covariance matrices [latex](\Sigma_1 = \Sigma_2 = \cdots = \Sigma_k)[/latex]. Therefore, often...

Classification with Linear Discriminant Analysis

Classification with linear discriminant analysis is a common approach to predicting class membership of observations. A previous post explored the descriptive aspect of linear discriminant analysis with data collected on...

Discriminant Analysis of Several Groups

Discriminant analysis is also applicable in the case of more than two groups. In the first post on discriminant analysis, there was only one linear discriminant function as the number...

MANOVA Test Statistics with R

Multiple tests of significance can be employed when performing MANOVA. The most well known and widely used MANOVA test statistics are Wilk’s [latex]\Lambda[/latex], Pillai, Lawley-Hotelling, and Roy’s test. Unlike ANOVA...

Multiple Analysis of Variance (MANOVA)

MANOVA, or Multiple Analysis of Variance, is an extension of Analysis of Variance (ANOVA) to several dependent variables. The approach to MANOVA is similar to ANOVA in many regards and...

Discriminant Analysis for Group Separation in R

The term ‘discriminant analysis’ is often used interchangeably to represent two different objectives. These objectives of discriminant analysis are: Description of group separation. Linear combinations of variables, known as discriminant functions,...

Image Compression with Singular Value Decomposition

As mentioned in a previous post, image compression with singular value decomposition is a frequently occurring application of the method. The image is treated as a matrix of pixels with...

Singular Value Decomposition in R

Following from a previous post on the Cholesky decomposition of a matrix, I wanted to explore another often used decomposition method known as Singular Value Decomposition, also called SVD. SVD...

The Matrix Trace in R and Some Properties of the Trace

Although comparatively straightforward in nature, the matrix trace has many properties related to other matrix operations and often appears in statistical methods such as maximum likelihood estimation of the covariance...

Cholesky Decomposition of a Positive-Definite Matrix

Cholesky decomposition, also known as Cholesky factorization, is a method of decomposing a positive-definite matrix. A positive-definite matrix is defined as a symmetric matrix where for all possible vectors [latex]x[/latex],...

The Bisection Method of Root-Finding with R

The bisection method is another approach to finding the root of a continuous function [latex]f(x)[/latex] on an interval [latex][a, b][/latex]. The method takes advantage of a corollary of the intermediate...

The Secant Method Root-Finding Algorithm in R

The secant method for finding roots of nonlinear equations is a common and popular variation of the Newton-Raphson method that has been used for several millennia before the invention of...

The Newton-Raphson Root-Finding Algorithm in R

The Newton-Raphson method is an approach for finding the roots of nonlinear equations and is one of the most common root-finding algorithms due to its relative simplicity and speed. The...

Linear Regression through the Origin

The linear regression models examined so far have always included a constant that represents the point the regression line crosses the y-axis, called the intercept. However, there are some cases...

Linear Regression Confidence and Prediction Intervals

In a previous example, linear regression was examined through the simple regression setting, i.e., one independent variable. Fitting a linear model allows one to answer questions such as: What is the...

Simple Linear Regression Models with R

Linear regression is a widely used technique to model the association between a dependent variable and one or more independent variables. In the Simple Linear Regression setting, which is what...

Games-Howell Test for Post-Hoc Analysis

The Games-Howell post-hoc test is another nonparametric approach to compare combinations of groups or treatments. Although rather similar to Tukey’s test in its formulation, the Games-Howell test does not assume...

Spearman’s Rank Correlation Coefficient

In a previous example, linear correlation was examined with Pearson’s [latex]r[/latex]. The cars dataset that was examined exhibited a strong linear relationship, and thus Pearson’s correlation was a good candidate...

Post-Hoc Analysis with Tukey’s Test

In a previous example, ANOVA (Analysis of Variance) was performed to test a hypothesis concerning more than two groups. Although ANOVA is a powerful and useful parametric approach to analyzing...

More Post-Hoc Testing with Kruskal-Wallis

The Kruskal-Wallis test extends the Mann-Whitney-Wilcoxon Rank Sum test for more than two groups. The test is nonparametric similar to the Mann-Whitney test and as such does not assume the...

ANOVA for Comparing More than Two Groups

ANOVA, or Analysis of Variance, is a commonly used approach to testing a hypothesis when dealing with two or more groups. One-way ANOVA, which is what will be explored in...

Measuring Cabbages with Mann-Whitney

In previous examples, hypothesis testing with two independent samples drawn from normally distributed populations was explored. Often, however, data is not normally distributed, which causes the t-test to output incorrect...

Estimating Professor Salaries with Confidence Intervals

Introduction Estimating with confidence intervals is another form of hypothesis testing that is often preferred over standard hypothesis testing such as what was explored in the previous post. A primary reason...

Predicting Extramarital Affairs with Decision Trees and R

In this example, we'll build classification decision trees to analyze if a particular individual will commit an affair on their partner based on demographics and other data. Getting Started Start by loading...

ARIMA Forecasting with Excel and R

Hello! Today I am going to walk you through an introduction to the ARIMA model and its components, as well as a brief explanation of the Box-Jenkins method of how...

Linear Regression with R Example

Linear regression models find relationships between a dependent variable, often designated y, and one or more dependent variables often denoted x. Linear regression has two primary functions and has a...