Statistics/Mathematics 3340 - Regression Analysis, Fall, 2025
Review materials:
Please review this material yourself as needed.
Selinger, Matrix Theory and Linear Algebra. material on projections in Chapter 5 will be useful.
James et al., Introduction to
Statistical Learning. Chapter 3 has material on simple and
multiple linear regression. Chapter 6 has material on model selection,
and chapter 7 discusses polynomial regression and splines.
Info on installation of R:
Notes on the installation of R, Rstudio and Rmarkdown.
Rmd file corresponding to the previous pdf.
LECTURE NOTES (under construction)
Topic 1: Simple linear regression
notes from Stat2080, with a bit of added R code.
R code for simple linear regression. Illustrates the use of R as a calculator, with applicaton to simple linear regression.
Prof. Dowd's handwritten Stat2080 notes on multiple regression.
Topic 2: Linear algebra
Some basic matrix algebra. Review on your own..
Projections. Includes derivation of prediction equation using a squence of projections.
Topic 3: Intro to multiple regression in R
Using the "lm" command to carry out regression in R.
R commands to read data in a .csv file, and carry out a multiple regression using the "lm" function in R
Topic 4: Some useful regression models
Indicator variables, and their use in
analysis of variance models.
Types of linear regression models.
Polynomial regression.
Topic 5: Comparing models with the partial F test
Partial F test
example: carrying out the partial F test in R by comparing a full and a reduced model
Topic 6: Some basic residual analysis
Intro to residual analysis
Common issues with residual plots
Topic 7: Differentiating with respect to a vector
Formulas for differentiating with respect to a vector. Optional reading This gives a convenient method for deriving the least squares estimator in multiple regression.
(To deive the formulas, one needs some ideas from multivariable calculus, and a more advanced class
in multivariate statistical analysis such as Stat4350.)
Topic 8: Least squares estimation for the multiple regression model.
Multiple regression model, least squares estimation.
Derivation of the estimates of
intercept and slope for simple linear regression, using matrix calculations - independent review
Topic 9: Means and covariances, random vectors
Rules for expected values and variances of linear combinations of r.v.'s - independent review.
Random vectors: definitions, expectation and covariance, including linear combinations.
R code to check mean and covariance calculations on random vectors page - independent review.
Topic 10: Sampling distributions and confidence intervals
sampling distributions of y, betahat, predicted values, residuals, and confidence intervals..
R code for construction of simulateous CI and an elliptical confidence region.
Topic 11: F tests
Cochran's theorem and the overall F test of significance.
review of hypothesis testing in multiple linear regression
Several examples showing how to test hypotheses using the lm and anova commands.
example: carrying out the partial F test in R - cement data set
an example
example of constructing an added variable plot
Testing the General Linear Hypothesis (section 3.3.4).
Topic 12: Miscellaneous topics
A summary of some ideas on influence, leverage, standardized residuals, transformation and model selection.
ASSIGNMENTS
Assignment 1, due Wednesday, October 1, 11:59 PM
This is the pdf version of the assignment.
R markdown file for assignment 1.
Assignment 1 solutions
Assignment 2, due Thursday, October 16, 11:59 PM
This is the pdf version of the assignment.
R markdown file for assignment 2.
Assignment 2 solutions
Assignment 3, due Sundary, October 26, 11:59 PM
Assignment 3 solutions
Assignment 4, due Thursday, November 20, 11:59 PM
This is the pdf version of the assignment.
R markdown file for assignment 4.
Assignment 4 solutions.
Assignment 5, due Monday, Sunday, Dec 7, 11:59 PM
This is the pdf version of the assignment.
R markdown file for assignment 5.
Assignment 5 solutions
Assignment 6, due Wednesday, December 10, 11:59 PM
Assignment 6 solutions
Exam Info
midterm solutions.
midterm practice questions. (skip questions 4 and 5)
solutions to practice questions.
The final exam will be scheduled by the Registrar.
Practice final examination .
This is the exam from fall, 2015. Coverage of this years exam may
vary somewhat, and will include some questions requiring derivations.
solutions to practice final examination
Formula sheet for final exam Formula sheet which will be included with exams - first ywo pages for midterm exam.
Statistical Tables