public class OLSMultipleLinearRegression extends AbstractMultipleLinearRegression
Implements ordinary least squares (OLS) to estimate the parameters of a multiple linear regression model.
The regression coefficients, b
, satisfy the normal equations:
X^{T} X b = X^{T} y
To solve the normal equations, this implementation uses QR decomposition
of the X
matrix. (See QRDecomposition
for details on the
decomposition algorithm.) The X
matrix, also known as the design matrix,
has rows corresponding to sample observations and columns corresponding to independent
variables. When the model is estimated using an intercept term (i.e. when
isNoIntercept
is false as it is by default), the X
matrix includes an initial column identically equal to 1. We solve the normal equations
as follows:
X^{T}X b = X^{T} y
(QR)^{T} (QR) b = (QR)^{T}y
R^{T} (Q^{T}Q) R b = R^{T} Q^{T} y
R^{T} R b = R^{T} Q^{T} y
(R^{T})^{1} R^{T} R b = (R^{T})^{1} R^{T} Q^{T} y
R b = Q^{T} y
Given Q
and R
, the last equation is solved by backsubstitution.
Constructor and Description 

OLSMultipleLinearRegression()
Create an empty OLSMultipleLinearRegression instance.

OLSMultipleLinearRegression(double threshold)
Create an empty OLSMultipleLinearRegression instance, using the given
singularity threshold for the QR decomposition.

Modifier and Type  Method and Description 

double 
calculateAdjustedRSquared()
Returns the adjusted Rsquared statistic, defined by the formula

protected RealVector 
calculateBeta()
Calculates the regression coefficients using OLS.

protected RealMatrix 
calculateBetaVariance()
Calculates the variancecovariance matrix of the regression parameters.

RealMatrix 
calculateHat()
Compute the "hat" matrix.

double 
calculateResidualSumOfSquares()
Returns the sum of squared residuals.

double 
calculateRSquared()
Returns the RSquared statistic, defined by the formula

double 
calculateTotalSumOfSquares()
Returns the sum of squared deviations of Y from its mean.

void 
newSampleData(double[] y,
double[][] x)
Loads model x and y sample data, overriding any previous sample.

void 
newSampleData(double[] data,
int nobs,
int nvars)
Loads model x and y sample data from a flat input array, overriding any previous sample.

protected void 
newXSampleData(double[][] x)
Loads new x sample data, overriding any previous data.

calculateErrorVariance, calculateResiduals, calculateYVariance, estimateErrorVariance, estimateRegressandVariance, estimateRegressionParameters, estimateRegressionParametersStandardErrors, estimateRegressionParametersVariance, estimateRegressionStandardError, estimateResiduals, getX, getY, isNoIntercept, newYSampleData, setNoIntercept, validateCovarianceData, validateSampleData
public OLSMultipleLinearRegression()
public OLSMultipleLinearRegression(double threshold)
threshold
 the singularity thresholdpublic void newSampleData(double[] y, double[][] x) throws MathIllegalArgumentException
y
 the [n,1] array representing the y samplex
 the [n,k] array representing the x sampleMathIllegalArgumentException
 if the x and y array data are not
compatible for the regressionpublic void newSampleData(double[] data, int nobs, int nvars)
Loads model x and y sample data from a flat input array, overriding any previous sample.
Assumes that rows are concatenated with y values first in each row. For example, an input
data
array containing the sequence of values (1, 2, 3, 4, 5, 6, 7, 8, 9) with
nobs = 3
and nvars = 2
creates a regression dataset with two
independent variables, as below:
y x[0] x[1]  1 2 3 4 5 6 7 8 9
Note that there is no need to add an initial unitary column (column of 1's) when
specifying a model including an intercept term. If AbstractMultipleLinearRegression.isNoIntercept()
is true
,
the X matrix will be created without an initial column of "1"s; otherwise this column will
be added.
Throws IllegalArgumentException if any of the following preconditions fail:
data
cannot be nulldata.length = nobs * (nvars + 1)
nobs > nvars
This implementation computes and caches the QR decomposition of the X matrix.
 Overrides:
newSampleData
in class AbstractMultipleLinearRegression
 Parameters:
data
 input data array
nobs
 number of observations (rows)
nvars
 number of independent variables (columns, not counting y)

calculateHat
public RealMatrix calculateHat()
Compute the "hat" matrix.
The hat matrix is defined in terms of the design matrix X
by X(X^{T}X)^{1}X^{T}
The implementation here uses the QR decomposition to compute the
hat matrix as Q I_{p}Q^{T} where I_{p} is the
pdimensional identity matrix augmented by 0's. This computational
formula is from "The Hat Matrix in Regression and ANOVA",
David C. Hoaglin and Roy E. Welsch,
The American Statistician, Vol. 32, No. 1 (Feb., 1978), pp. 1722.
Data for the model must have been successfully loaded using one of
the newSampleData
methods before invoking this method; otherwise
a NullPointerException
will be thrown.
 Returns:
 the hat matrix
 Throws:
NullPointerException
 unless method newSampleData
has been
called beforehand.

calculateTotalSumOfSquares
public double calculateTotalSumOfSquares()
Returns the sum of squared deviations of Y from its mean.
If the model has no intercept term, 0
is used for the
mean of Y  i.e., what is returned is the sum of the squared Y values.
The value returned by this method is the SSTO value used in
the Rsquared
computation.
 Returns:
 SSTO  the total sum of squares
 Throws:
NullPointerException
 if the sample has not been set
 See Also:
AbstractMultipleLinearRegression.isNoIntercept()

calculateResidualSumOfSquares
public double calculateResidualSumOfSquares()
Returns the sum of squared residuals.
 Returns:
 residual sum of squares
 Throws:
MathIllegalArgumentException
 if the design matrix is singular
NullPointerException
 if the data for the model have not been loaded

calculateRSquared
public double calculateRSquared()
Returns the RSquared statistic, defined by the formula R^{2} = 1  SSR / SSTO
where SSR is the sum of squared residuals
and SSTO is the total sum of squares
If there is no variance in y, i.e., SSTO = 0, NaN is returned.
 Returns:
 Rsquare statistic
 Throws:
NullPointerException
 if the sample has not been set
MathIllegalArgumentException
 if the design matrix is singular

calculateAdjustedRSquared
public double calculateAdjustedRSquared()
Returns the adjusted Rsquared statistic, defined by the formula
R^{2}_{adj} = 1  [SSR (n  1)] / [SSTO (n  p)]
where SSR is the sum of squared residuals
,
SSTO is the total sum of squares
, n is the number
of observations and p is the number of parameters estimated (including the intercept).
If the regression is estimated without an intercept term, what is returned is
1  (1  calculateRSquared()
) * (n / (n  p))
If there is no variance in y, i.e., SSTO = 0, NaN is returned.
 Returns:
 adjusted RSquared statistic
 Throws:
NullPointerException
 if the sample has not been set
MathIllegalArgumentException
 if the design matrix is singular
 See Also:
AbstractMultipleLinearRegression.isNoIntercept()

newXSampleData
protected void newXSampleData(double[][] x)
Loads new x sample data, overriding any previous data.
The input x
array should have one row for each sample
observation, with columns corresponding to independent variables.
For example, if x = new double[][] {{1, 2}, {3, 4}, {5, 6}}
then setXSampleData(x)
results in a model with two independent
variables and 3 observations:
x[0] x[1]

1 2
3 4
5 6
Note that there is no need to add an initial unitary column (column of 1's) when
specifying a model including an intercept term.
This implementation computes and caches the QR decomposition of the X matrix
once it is successfully loaded.
 Overrides:
newXSampleData
in class AbstractMultipleLinearRegression
 Parameters:
x
 the rectangular array representing the x sample

calculateBeta
protected RealVector calculateBeta()
Calculates the regression coefficients using OLS.
Data for the model must have been successfully loaded using one of
the newSampleData
methods before invoking this method; otherwise
a NullPointerException
will be thrown.
 Specified by:
calculateBeta
in class AbstractMultipleLinearRegression
 Returns:
 beta
 Throws:
MathIllegalArgumentException
 if the design matrix is singular
NullPointerException
 if the data for the model have not been loaded

calculateBetaVariance
protected RealMatrix calculateBetaVariance()
Calculates the variancecovariance matrix of the regression parameters.
Var(b) = (X^{T}X)^{1}
Uses QR decomposition to reduce (X^{T}X)^{1}
to (R^{T}R)^{1}, with only the top p rows of
R included, where p = the length of the beta vector.
Data for the model must have been successfully loaded using one of
the newSampleData
methods before invoking this method; otherwise
a NullPointerException
will be thrown.
 Specified by:
calculateBetaVariance
in class AbstractMultipleLinearRegression
 Returns:
 The beta variancecovariance matrix
 Throws:
MathIllegalArgumentException
 if the design matrix is singular
NullPointerException
 if the data for the model have not been loaded
Copyright © 2016–2020 Hipparchus.org. All rights reserved.