Simple Linear regression - The relationship between bonelength and stature

 MTB > info
 
 COLUMN    NAME      COUNT
 C1        bone         10
 C2        stature      10
 
 1.  Make a scatterplot of the data.  Since we are trying to predict
     stature using the length of metacarpal bone I, the dependent
     variable(y) is stature and the independent variable(x) is bonelength.
 
 MTB > plot c2 c1;
 SUBC> xlabel 'bone length in mm';
 SUBC> ylabel 'stature in cm';
 SUBC> title 'The relationship between stature and bonelength'.
 
                 The relationship between stature and bonelength
          -
          -                                           *           *
 s        -
 t     180+
 a        -                                                   *
 t        -                   *
 u        -                                   *
 r        -                           *   *       *
 e     170+
          -
 i        -
 n        -           *
          -
 c     160+
 m        -   *
          -
          -
            ------+---------+---------+---------+---------+---------+
               40.0      42.5      45.0      47.5      50.0      52.5
     
                            bone length in mm

	There appears to be a positive linear association between
	stature and the length of metacarpal bone I.

 MTB > regress c2 on 1 in c1 c3 c4;  (c1=x, c2=y, c3 = ri, c4= y-hat)
 SUBC> residuals c5;(this subcommand calculates the ei and puts them in c5)
 SUBC> predict 45.  (this subcommand does predictions for x = 45)
 
 The regression equation is
 stature = 94.0 + 1.71 bone
 
 Predictor       Coef       Stdev    t-ratio        p
 Constant       93.99       14.62       6.43    0.000
 bone          1.7074      0.3159       5.40    0.000
 
 s = 3.983       R-sq = 78.5%     R-sq(adj) = 75.8%
 
 Analysis of Variance
 
 SOURCE       DF          SS          MS         F        p
 Regression    1      463.21      463.21     29.20    0.000
 Error         8      126.89       15.86
 Total         9      590.10
 
 Unusual Observations
 Obs.    bone   stature       Fit Stdev.Fit  Residual   St.Resid
   9     43.0    175.00    167.41      1.60      7.59      2.08R 
 
 R denotes an obs. with a large st. resid. ( Minitab flags any residuals
	which are more than two standard deviations away from the mean)

Note that R-squared is fairly large (.785). This tells us that 78.5% of
the variability in y is explained by x.

When we test whether the slope is zero or not(twosided test) the test
statistic is t = 5.40 .  t has 8 df and the p-value is less than .001.
We can conclude that the slope is not zero and is in fact positive.We could
do a one sided test since all our knowledge about growth suggests that if
there is an association between the two variables it would be positive.

Note that we can also do the two sided test using the ANOVA table where the
value of the test statistic is F = 29.20.  F has 1,8 df giving a p-value of
less than .001.  The two sided t test and the F test give the same results;
in linear regression where t has n-2 df and F has 1,n-2 df, the square root
of F = t.   In this case the square root of 29.2 = 5.4
 
This part of the output was generated using the predict subcommand.
In this case we wanted to do predictions at x = 45, so the subcommmand
is SUBC > predict 45.

      Fit  Stdev.Fit         95% C.I.         95% P.I.
   170.82       1.31   ( 167.81, 173.84)  ( 161.15, 180.49)   
 
 
 
 MTB > name c3 'ri'  The ri are the standardized residuals

 MTB > name c4 'y-hat'  The y-hat are the predicted or fitted values

 MTB > name c5 'ei'  The ei are the raw residuals
	 ei = yi -(yi-hat)= actual y  minus the predicted value of y


	Residual plots 
Any of the following plots give the same picture as you can see below.
	standardized residuals (ri) vs fitted values (y-hat)
	standardized residuals (ri) vs x
	raw residuals (ei) vs fitted values (y-hat)
	raw  residuals (ei) vs x

	If the assumptions are not violated we expect the plot to be
	a cloud or band of points centered on zero. 

	If there is curvature in the scatterplot this will be magnified
	in the residual plot.  If there is curvature in the residual plot,
	we may want to consider a different model i.e. one that allows for
 	curvature.

	The width of the band should be approximately the same for all
	values of x. If you see a plot where the band is very narrow at
 	small values of x and much wider at large values of x, you have
 	a problem with unequal variance ( heteroscedasticity, if you like
	big words).


	Any large residuals may represent outliers in the data and these 
	points should be investigated.  These outliersmay be due to errors
 	in data measurement or in data entry. 
	
		RESIDUAL PLOTS (ALL 4 GIVE THE SAME PICTURE)

 MTB > plot c3 c4
 
       2.4+
          -                      *
  ri      -
          -
          -                                          *
       1.2+
          -
          -
          -
          -
       0.0+                             *                       *
          -               *                *  *
          -
          -
          -                                       *         *
      -1.2+        *
          -
            ------+---------+---------+---------+---------+---------+y-hat   
              160.0     165.0     170.0     175.0     180.0     185.0
 
 MTB > plot c3 c1
 
       2.4+
          -                   *
  ri      -
          -
          -                                           *
       1.2+
          -
          -
          -
          -
       0.0+                           *                           *
          -           *                   *   *
          -
          -
          -                                       *           *
      -1.2+   *
          -
            ------+---------+---------+---------+---------+---------+bone    
               40.0      42.5      45.0      47.5      50.0      52.5
 

 MTB > plot c5 c4
 
       8.0+
          -                      *
  ei      -
          -                                          *
          -
       4.0+
          -
          -
          -
          -
       0.0+                             *                       *
          -               *                *
          -                                   *
          -
          -        *                                        *
      -4.0+                                       *
          -
            ------+---------+---------+---------+---------+---------+y-hat   
              160.0     165.0     170.0     175.0     180.0     185.0
 
 MTB > plot c5 c1
 
       8.0+
          -                   *
  ei      -
          -                                           *
          -
       4.0+
          -
          -
          -
          -
       0.0+                           *                           *
          -           *                   *
          -                                   *
          -
          -   *                                               *
      -4.0+                                       *
          -
            ------+---------+---------+---------+---------+---------+bone    
               40.0      42.5      45.0      47.5      50.0      52.5
 
	The residual plot is satisfactory.  There is one residual labelled
 	on the output as large( ri= 2.08).  This is really not a problem.
	From the empirical rule, we expect about 95% of the ri to be
 	between 2 and -2 (i.e. within 2 standard deviations of the mean).
	This residual is only slightly more than two standard deviations 
	away from the mean.

	Nscores plots and histograms give information about the normality
 	assumption. If the residuals are normally distributed, the nscores
	plot should be a straight line. Marked deviations from a straight 
	line indicate non normality. In reality we don't expect to see
	an absoutely straight line even if the residuals do come from
	a normal distribution.  This is particularly true when n is small.

 MTB > nscores c3 c6
 MTB > plot c3 c6
 
       2.4+
          -                                                       *
  ri      -
          -
          -                                              *
       1.2+
          -
          -
          -
          -
       0.0+                                   *    *
          -                       *   *   *
          -
          -
          -            *     *
      -1.2+   *
          -
            --------+---------+---------+---------+---------+--------C6      
                -1.20     -0.60      0.00      0.60      1.20
 
 MTB > hist c3
 
 Histogram of ri   N = 10
 
 Midpoint   Count
     -1.0       3  ***
     -0.5       2  **
      0.0       3  ***
      0.5       0
      1.0       0
      1.5       1  *
      2.0       1  *


	In this case the plots are not what we would like to see, but it
 	is hard to tell anything from a plot with only ten points unless
	the distribution is very skewed or there are large outliers in the
	data.
 
	Since the residual plots are acceptable, the test for the slope
	suggests that the slope is significantly greater than zero and
	R-squared is fairly large (.785), our model seems to be a 
	reasonable model to use to predict stature from the length
	of metacapel bone I. 


This part of the output was generated using the predict subcommand.
In this case we wanted to do predictions at x = 45, so the subcommmand
is SUBC > predict 45.


      Fit  Stdev.Fit         95% C.I.         95% P.I.
   170.82       1.31   ( 167.81, 173.84)  ( 161.15, 180.49)

The fit column gives the value of y-hat. In this case our point estimate
of the stature of an individual with metacarpel bone length I of 45 mm
is 170.82 cm

The column Stdev.Fit  gives us the standard deviation of the fit
( the s y-hat)

The column 95% confidence interval gives us the 95% confidence interval
for the mean stature of individuals in the population with a metacarpel
bone that is 45 mm long.

The column 95% prediction interval gives us the 95% prediction interval
for an individual in the population with a metacarpel bone I which is
45 mm long.  Suppose we are anthropologists investigating this population
and we find a metacapel bone I which is 45 mm long.  How tall was this 
person?

If we were only allowed to guess one number for the person's stature,
the fit of 170.82 would be our best guess.

The 95% prediction interval tells us that we are 95% confident that
the person was between 161.15 cm and 180.49 cm tall. ( 95% confident
means that our method is reliable 95% of the time.  That is, if we use
this method over and over again the true height is contained in the
interval 95% of the time.)

 


MTB > regress c2 on 1 in c1;

 SUBC> predict 39;
 SUBC> predict 42;
 SUBC> predict 45;
 SUBC> predict 46;
 SUBC> predict 48;
 SUBC> predict 51;
 SUBC> predict 56;
 SUBC> predict 100.
 
 The regression equation is
 stature = 94.0 + 1.71 bone
 
 Predictor       Coef       Stdev    t-ratio        p
 Constant       93.99       14.62       6.43    0.000
 bone          1.7074      0.3159       5.40    0.000
 
 s = 3.983       R-sq = 78.5%     R-sq(adj) = 75.8%
 
 Analysis of Variance
 
 SOURCE       DF          SS          MS         F        p
 Regression    1      463.21      463.21     29.20    0.000
 Error         8      126.89       15.86
 Total         9      590.10
 
 Unusual Observations
 Obs.    bone   stature       Fit Stdev.Fit  Residual   St.Resid
   9     43.0    175.00    167.41      1.60      7.59      2.08R 
 
 R denotes an obs. with a large st. resid.
 
      Fit  Stdev.Fit         95% C.I.         95% P.I.
   160.58       2.57   ( 154.64, 166.51)  ( 149.64, 171.51)   
 
   165.70       1.81   ( 161.53, 169.87)  ( 155.61, 175.79)   
 
   170.82       1.31   ( 167.81, 173.84)  ( 161.15, 180.49)   
 
   172.53       1.26   ( 169.62, 175.44)  ( 162.89, 182.16)   
 
   175.94       1.40   ( 172.73, 179.16)  ( 166.21, 185.68)   
 
   181.07       2.00   ( 176.46, 185.67)  ( 170.79, 191.34)   
 
   189.60       3.37   ( 181.83, 197.38)  ( 177.57, 201.64) X 
 
   264.73      17.08   ( 225.34, 304.11)  ( 224.28, 305.17) XX
 
 X  denotes a row with X values away from the center
 XX denotes a row with very extreme X values
 
Here I have done a lot of predictions.  Look at the values for Stdev.Fit.
You can see that at mean x (46.1), the standard deviation of y-hat is the
smallest.  As we predict at values further away from mean x, the standard
deviation of y-hat increases, so confidence intervals for expected y and
prediction intervals for new y are wider.  


