Statistics/Mathematics 3340 - Regression Analysis, Fall, 2026

• Syllabus

• Review of confidence intervals
Ideas covered in Stat1060, Stat2060, Stat2080.

• Review of hypothesis testing
Ideas covered in Stat1060, Stat2060, Stat2080.

Reference for linear and matrix algebra:
• Selinger, Matrix Theory and Linear Algebra. material on projections in Chapter 5 will be useful.
• James et al., Introduction to Statistical Learning. Chapter 3 has material on simple and multiple linear regression. Chapter 6 has material on model selection, and chapter 7 discusses polynomial regression and splines.
  
  Info on installation of R:
• Notes on the installation of R, Rstudio and Rmarkdown.
  
• Rmd file corresponding to the previous pdf.
  


LECTURE NOTES (under construction)

Topic 1: Simple linear regression
• notes from Stat2080, with a bit of added R code.
• R code for simple linear regression. Illustrates the use of R as a calculator, with applicaton to simple linear regression.

Topic 2: Intro to multiple regression

• Prof. Dowd's handwritten Stat2080 notes on multiple regression.
• Using the "lm" command to carry out regression in R.
• R commands to read data in a .csv file, and carry out a multiple regression using the "lm" function in R
  
• Partial F test
• example: carrying out the partial F test in R by comparing a full and a reduced model

Topic 3: Some useful regression models
• Indicator variables, and their use in analysis of variance models.
• Types of linear regression models.
• Polynomial regression.
• Spline regression.

Topic 4: Some basic residual analysis
• Intro to residual analysis
• Common issues with residual plots

Topic 5: Some related models
• Logistic regression.
• Poisson regression.

Topic 6: Linear algebra
• Some basic matrix algebra. Review on your own..
  
• Projections. Includes derivation of prediction equation using a squence of projections.
  
  Topic 7: Differentiating with respect to a vector
• Formulas for differentiating with respect to a vector. Optional reading This gives a convenient method for deriving the least squares estimator in multiple regression. (To deive the formulas, one needs some ideas from multivariable calculus, and a more advanced class in multivariate statistical analysis such as Stat4350.)

Topic 8: Least squares estimation for the multiple regression model.
• Multiple regression model, least squares estimation.
• Derivation of the estimates of intercept and slope for simple linear regression, using matrix calculations - independent review

Topic 9: Means and covariances, random vectors
• Rules for expected values and variances of linear combinations of r.v.'s - independent review.
• Random vectors: definitions, expectation and covariance, including linear combinations.
• R code to check mean and covariance calculations on random vectors page - independent review.

Topic 10: Sampling distributions and confidence intervals
• sampling distributions of y, betahat, predicted values, residuals, and confidence intervals..
• R code for construction of simulateous CI and an elliptical confidence region.

Topic 11: F tests
• Cochran's theorem and the overall F test of significance.
• review of hypothesis testing in multiple linear regression
• Several examples showing how to test hypotheses using the lm and anova commands.
• example: carrying out the partial F test in R - cement data set
• an example
• example of constructing an added variable plot


• Testing the General Linear Hypothesis (section 3.3.4).



Topic 12: Diagnostics
• leverage ( Chapter 6)
• Multicollinearity (Chapters 3&9)
• standardized residuals (Chapter 4) and case deletion statistics (Chapter 6)
• transformations (chapter 5)
• some linearizing transformations (table 5.4)





• Formula sheet for final exam




Statistical Tables
• T and F tables