Statistics/Mathematics 3340 - Regression Analysis, Fall, 2026

• Syllabus

• Review of confidence intervals
Ideas covered in Stat1060, Stat2060, Stat2080.

• Review of hypothesis testing
Ideas covered in Stat1060, Stat2060, Stat2080.

Reference for linear and matrix algebra:
• Selinger, Matrix Theory and Linear Algebra. material on projections in Chapter 5 will be useful.
• James et al., Introduction to Statistical Learning. Chapter 3 has material on simple and multiple linear regression. Chapter 6 has material on model selection, and chapter 7 discusses polynomial regression and splines.
  
  Info on installation of R:
• Notes on the installation of R, Rstudio and Rmarkdown.
  
• Rmd file corresponding to the previous pdf.
  


LECTURE NOTES (under construction)

Topic 1: Simple linear regression
• notes from Stat2080, with a bit of added R code.

Topic 2: Linear algebra
• Some basic matrix algebra. Review on your own..
  
• Projections. Includes derivation of prediction equation using a squence of projections.
  
  Topic 3: Intro to multiple regression
  
• Prof. Dowd's handwritten Stat2080 notes on multiple regression. (Review)
• The multiple regression model - fitting the model using the "lm" command to carry out regression in R.
• Partial F test
• example of carrying out the partial F test

Topic 4: Some useful regression models
• Indicator variables, and their use in analysis of variance models.
• Types of linear regression models.
• Polynomial regression.



Differentiating with respect to a vector
• Formulas for differentiating with respect to a vector. Optional reading This gives a convenient method for deriving the least squares estimator in multiple regression. (To deive the formulas, one needs some ideas from multivariable calculus, and a more advanced class in multivariate statistical analysis such as Stat4350.)

Topic: Least squares estimation for the multiple regression model.
• Multiple regression model, least squares estimation.
• Derivation of the estimates of intercept and slope for simple linear regression, using matrix calculations - Optional reading

Topic: Means and covariances, random vectors
• Rules for expected values and variances of linear combinations of r.v.'s
• Random vectors: definitions, expectation and covariance, including linear combinations.
• R code to check mean and covariance calculations on random vectors page - independent review.
• sampling distributions of y, betahat, predicted values, and residuals. Confidence intervals.
• Some facts about distributions.

Simultaneous confidence intervals, general regression significance test
• Simultaneous confidence intervals.
• Testing the General Linear Hypothesis.
• R code for construction of simulateous CI and an elliptical confidence region.

F tests
• Cochran's theorem and the overall F test of significance.
• review of hypothesis testing in multiple linear regression
• Several examples showing how to test hypotheses using the lm and anova commands.
• example: carrying out the partial F test in R - cement data set
• an example
• example of constructing an added variable plot


Topic 12: Diagnostics
Some basic residual analysis
• Intro to residual analysis
• Common issues with residual plots


• leverage
• Multicollinearity
• standardized residuals and case deletion statistics
e
• transformations
• some linearizing transformations


Topic 13:
• some sources of bias.

Topic: Some related models
• Logistic regression.
• Poisson regression.


ASSIGNMENTS

• Assignment 1, due Sunday, September 20, 11:59 PM
This is the pdf version of the assignment.
• R markdown file for assignment 1.
• assignment 1 solutions.



• Assignment 2, due Sunday, October 4, 11:59 PM
This is the pdf version of the assignment.
• R markdown file for assignment 2.
• assignment 2 solutions.



• Assignment 3, due Sunday, October 18, 11:59 PM
• assignment 3 solutions.


Exam Info

• The midterm exam will be on Friday, October 30, in class time.
• midterm practice questions.
• solutions to practice questions.



• Formula sheet for final exam