Multiple Linear Regression & Regression Coefficients Questions Response Below is the question.I have also attached the lecture notes.Below is the question.

Multiple Linear Regression & Regression Coefficients Questions Response Below is the question.I have also attached the lecture notes.Below is the question.I have also attached the lecture notes.Below is the question.I have also attached the lecture notes. What we have seen
¨
¨
¨
¨
! ! and ”
! ” according
We have derived the expression of ”
to the method of least squares (OLS).
! ! and # ”
!” ,
We have also derived the expression of # ”
! ! and $%& ”
!” .
and of $%& ”
! ! and ”
! ” in
We have also derived the distribution of ”
=
=
the case of normally-distributed errors.
We can use that to perform inference (hypothesis
testing and confidence intervals).
Old Faithful eruptions
tipi
Example: Data on
duration of eruptions of
the Old Faithful geyser
and interval of time
until the next eruption
for all 222 recorded
eruptions in a single
month.
We can fit a linear
regression. What can
we say about the
relationship between
interval and duration?
Do we have to wait
about 25 minutes for
every minute that the
eruption lasts?
Inferring upon E[Y|X=x]
¨
A common use of regression models is to estimate the
expected value of the outcome Y conditional on a
given value of X.
¨
For example, we may wish to know the estimated
waiting time between Old Faithful eruptions if we know
that the previous eruption lasted 3 minutes.
¨
In addition, we would like to know the precision
with
which we are estimating #[(|* = ,]
Uarfetyl
X- n ] )
Inferring upon E[Y|X=x]
¨
We can easily note that an estimate of E[Y|X=x]
is given by:
u
Tµx#Rotting (5%5)+52=1
pix
th
¨
ftp.I-xtvurbtnl
E-g-
What about /01 2 “|- = &$
Hwk 9]
4 $ ? [Asked to find this for
= /01 ”
R,
(i)
(ii)
Var fft
couff,,5)
=
O
I
§
i.
←
Hi-x-p
s e e
o n
(iii)
in
linear
regression t h e X ‘ s
V I I .
(iv)
I t G .e z . .
a r e
yil
pEnN(Po…)
I
if
considered
ftp.p,
E r n Nco, o r )
R-NCR,
posting
Canvas
1
Ai-x-p
constant
nN(mean, variance)
Inferring upon E[Y|X=x]
I n 1 2 0B
¨
In the homework, you are also asked to find
§ Distribution of E[Y|X=x]
–
Z ,Y
a r e
two
independent
n o r m a l e
r u
– 7 Y tZ-Norma,
§ Distribution of E[Y|X=x] when variance .# is not
known.
§ These results can be used to derive CI for E[Y|X=x].
A note
¨
You have seen how to make predictions for future
values of Y with the TAs last Wednesday.
¨
Note the difference: there we were predicting the
expected value of Y when X is equal to a nonobserved value (predicting out of sample).
¨
Here we trying to estimate the expected value of Y
given a value of X observed.
Moving onto multiple linear
regression
¨
¨
When working with multiple covariates, it is mathematically more
convenient to work with matrices.
A simple linear regression model can be more conveniently be
rewritten as:
it.2.. h e 4
s i
¥÷h÷h÷÷
±¥t
¨
Yi–Potfixit
‘
‘¥!..
4 ¥ ii.
X is called the design matrix. The i-th row corresponds to the
value of the covariate for the i-th observation.
y-xe.ee#i¥÷÷¥E¥÷¥m
talk:b,
4,11K¥!
xe.ci?.:EikIf:*:*’
2×1
4×2
desmisafrix
,
4×1
Expectation of a random vector
¨
Consider the random
vector
–
#!
LT
⋮
#”
na
With % ## = &# and ‘() ## , ## = +#$ .
!=
Then, the mean vector , is given by
*
I:*:L-4¥:L
Eth)
n x t
vec tor
Yu k . .
Emil
Varlyil
Yu
ru’s
Covariance matrix
square m a t r i x
¨
The covariance matrix / is defined as:
!=
Some results on random vectors
Let Z be a random vector defined as – + /! where
§ Y is a random vector with % ! = ,
§ c is a fixed vector
§ A is fixed matrix
Then: % 4 = % – + /! =
¨
Some results on random vectors
Let Z be a random vector defined as – + /! where
§ Y is a random vector with % ! = , and 567 ! = 8
§ c is a fixed vector
§ A is fixed matrix
Then: Var 4 = /8/%
¨
Another result on random vectors
I
Let Y be a !×1 random vector with $ % = ‘ and ()* % = +.
Let A be a !×! matrix, then
¨
Let X be a random vector of length n with variance 0, and let
% = ,5 and 6 = 75 where A and B are fixed matrices of
dimension 8×! and 9×!. Then, the covariance of Y and Z is:
A ¨
!
$ ,% = -*)./ ,0 + ‘ 0’
Feetot
where -*)./ , = sum of its diagonal elements = ∑%”#$ )”” .
–
–
y Va r a – s i
VarcAY) – A I A ‘ t
art’Ay, # y F –
.
→
ON
:;< %, 6 = ,07& ¥ x "Bx 530%20×1=6.61×1 k¥41 (' 8 ? ? Multiple linear regression ¨ ÷%¥# The multiple linear regression is defined as i n . . . O¢¥¥y, f) =" = >‘ + >$ ?$” + >( ?(” + ⋯ + >) ?)” + A”
# x p ¥ u e o t Y i
i n
Fn¥g%%,
where
§ ?$” , ?(” , ⋯ , ?)” are the (fixed) value of the predictors for
observation B
⇐
§ A” is a random error term such that
n a
o $ A” = 0
o ()* A” = 0
0 2
o A” and A* are independent
[¥g)
sad,§
m£
“Tine: regression
E(4=(?§
(
In matrix form..
¨
ysxp.rs
The multiple linear regression model can be written in
matrix form as
f i t s o f multiple l i n e a r
regression,
y
→
a”
I’
th
1 ?$$
⋮
= ⋮
1 ?$%
.
Xdeiign
matrix
→
↳
Denot
2nd,
e
>’ + >$ ?$$ + >( ?($ + ⋯ + >) ?)$ + A$
=$
⋮
%= ⋮ = I
=)
man, >’ + >$ ?$% + >( ?(% + ⋯ + >) ?)% + A%
it
+
XP
O>’
?($ ?)$
>$
⋮ … ⋮
F
⋮
?(% ?)%
>)
X
n×(p+,)
A$
A
+ (
⋮
A%
T
PCptDxl
= GH + I
←
←
man
→
Y-ftp.OX.to
Bi
exultEnedxchannakseinbyYeneuni
t
Interpretation of regression parameters
Since $ A” = 0, we have that
$ =” =BLIP,X i i t a x z i t . .
Thus, we are modeling the conditional mean of Y given
?$ , ?( , ⋯ , ?) .
¨
¨
¨
tppXpi”
>’ denotes the expected value of Y when ?$” = ?(” = ⋯ =
?)” = 0. To see this note that
$ =|? = 0 = >’ .
>* denotes the expected difference in Y that is associated
with a 1-unit change in ?* ,#holding
all other7covariate values
nconstant.
To see this note that
$ =|?$ , ⋯ , ?*+$ , ?* = K + 1, ?*,$ , ⋯ , ?) − $M=|?$ , ⋯ , ?*+$ , ?* =
K, ?*,$ , ⋯ , ?) N = >*
–
↳tpixitfzxzt.itfjyxj.it#xtO)tPjtiXjtH-.tPpXp))
(Bot@Xi
tBzXzt.etPj-iXjit@j.x
+
B i t ,X p , . . . ttPpXp)
Hoi
Y e Got@x,
(Y- pot f .
t
p ,-
0
R e j e c tH o
hip#a model
s
1
Vs
§,
a ‘
f,
from
X, t
f i xu p , X z t
‘model 1
is
s
model 2
the same
Ho:P,- 0
H,:p,-1-0
,
from
model
2
n o t
reject
Ho
t
Group
ID
Gender
Weight
Beers
BAC
1
Female
132
5
0.1
2
Female
128
2
0.03
3
Female
110
9
0.19
4
Male
192
8
0.12
5
Male
172
3
0.04
6
Female
250
7
0.095
7
Female
125
3
0.07
8
Male
175
5
0.06
9
Female
175
3
0.02
10
Male
275
5
0.05
11
Female
130
4
0.07
12
Male
168
6
0.1
13
Female
128
5
0.085
14
Male
246
7
0.09
15
Male
164
1
0.01
16
Male
175
4
0.05
We will continue to
use the OSU data on
BAC/Alcohol intake
seen in Lecture 15.
Research question:
relationship between
BAC, number of
beers consumed
accounting for
differences in
weight.
9
¥,
=
Got B . X i
#beers
t-f’s,#
weight
Relationship between BAC, beers and weight
¨
¨
¨
In a previous lecture, I discussed as an example of simple linear
regression the relationship between BAC and number of beers consumed.
Since we believe that weight might be also related to BAC, we think that
it might be more appropriate to consider the joint effect of beers and
weight on BAC
i
expected change i n B A C
For this we call:
when # o f beers i n c r e a s e
§ X1=Beers
&,
§
§
b y 1 , holding weight constant
X2=Weight
Y=BAC
We can then investigate the association between BAC and the two
predictors using multiple linear regression.
BAC
expected change i n
increase o f 1lbs,
w h e n weight
constant
keeping t h e # o f beers
Pz:
Y – (§’;).
YI-ly.z.y.my#=yftyz2ty5
Estimation via least squares
¨
As in the case of simple linear regression, we want to minimize
$
! = # $! − & $!
!”#
= * − +,
¨
(
%
= # $! − ‘& − ‘# (#! − ‘% (%! − ⋯ − ” (‘!
=
!”# #
* − +,
element
iCq- x p )
%
o fv e c t o r
Expanding Q we obtain:
Q e ( y – x a t (Y-XP)
=
#
$
(Yt-Ext)(Y-Xp)
yty-2§XTY
+ f t XT
Xp
(Yt#DT) (Y-Mone t h e
–
Fix
=
¥
1
yty-Gp-pextyt
ptxtxp-
transparent
uyf.iq#lxcpt”],
picptDxi
× :
#
nxCpH)
T
B)
=
pixy
Estimation via least squares
¨
To minimize the sum of squares with respect to ,, we note that:
-!
←
-‘&
-!
←
-!
= -‘#
-,
⋮
-!
-”
213×1=3
so that:
-!
=
-,
s e t
it
c
scyty-2ptxtytptxtxp)
÷
equal
t o
0
a n d
s o l v ei t
fo r P
2(¥E¥%§;p..
Estimation via least squares
“YtYIFFkE@EqqIEzgj2.X’ + 2 – 5 × 4
T
2§p=-2XtYt2pxTX
–
t o
solicit
forp
XTY t p t X X = 0
XTX
P-XTY
(xtqXY(x,Ix)p=(XX5’xty
I-P-#
multiply i t
RightI l e f t
1bK ¥
” *y
-§-(xtXT’×#
a s
Xfisuotank
Mean and variance of OLS estimates
¨
O is unbiased, that is $ H
O =H
H
Post
canvas
o
n
Mean and variance of OLS estimates
¨
If we assume constant variance of the errors then:
O =
()* H
–
Va r ( E x )
–
‘xyt.LI/cxzxT’
Var(AY)=
A seat
PEI
canvas
Estimation of !
!
In the simple linear regression we found that
%
(
%
∑
∑
QPP
/
pl
“#$ ”
“#$ =” − >’ − >$ ?”
fo,
(
P =
=
=
!−2
!−2
!−2
was an unbiased estimate of S ( .
¨ In the case of multiple linear regression, we have
¨
(
•
&:
fop
pH
∑%”#$
QPP
(
P =
=
!− 8+1
% − 5H ! % − 5H
=
!− 8+1
r
a
=” − >’ − >$ ?$” − ⋯ − >) ?)”
!− 8+1
t
¨
–
P ( is an unbiased estimate of S (
(
Inference on regression coefficients
¨
From the above arguments,
Ho:p;- 0
H i Bj-70
§
y1¥f¥-thepth
–
()* >T’ :;< >T’ , >T$
O =
()* H
:;< >T$ , >T’ ()* >T$
⋮
⋮
And
→
⋯
⋯
()*U
>T = 8
is an unbiased estimate of it.
:;< >T’ , >T)
:;< >T$ , >T)
⋮
2%5′
= S ((5&l 5″
g
Inference on regression coefficients
¨
Under the case of normal errors, as in a simple linear
regression, we have that
!! #”!
”
$
!!
%&’
”
~”(# )*+
For k=0, 1,…, p.
¨
We can use this result to perform inference on regression
coefficients, e.g. carry out hypothesis testing and construct
confidence intervals.

Purchase answer to see full
attachment