1、Introductory Econometrics for Finance Chris Brooks 20021Chapter 3A brief overview of the classical linear regression modelIntroductory Econometrics for Finance Chris Brooks 20022RegressionRegression is probably the single most important tool at the econometricians disposal.But what is regression ana

2、lysis?It is concerned with describing and evaluating the relationship between a given variable (usually called the dependent variable) and one or more other variables (usually known as the independent variable(s).Introductory Econometrics for Finance Chris Brooks 20023Some NotationDenote the depende

3、nt variable by y and the independent variable(s) by x1, x2, . , xk where there are k independent variables.Some alternative names for the y and x variables:yxdependent variableindependent variablesregressandregressorseffect variablecausal variables explained variableexplanatory variableNote that the

4、re can be many x variables but we will limit ourselves to the case where there is only one x variable to start with. In our set-up, there is only one y variable.Introductory Econometrics for Finance Chris Brooks 20024Regression is different from Correlation If we say y and x are correlated, it means

5、 that we are treating y and x in a completely symmetrical way.In regression, we treat the dependent variable (y) and the independent variable(s) (xs) very differently. The y variable is assumed to be random or “stochastic” in some way, i.e. to have a probability distribution. The x variables are, ho

6、wever, assumed to have fixed (“non-stochastic”) values in repeated samples. Introductory Econometrics for Finance Chris Brooks 20025Simple Regression For simplicity, say k=1. This is the situation where y depends on only one x variable. Examples of the kind of relationship that may be of interest in

7、clude:How asset returns vary with their level of market riskMeasuring the long-term relationship between stock prices and dividends.Constructing an optimal hedge ratioIntroductory Econometrics for Finance Chris Brooks 20026Simple Regression: An ExampleSuppose that we have the following data on the e

8、xcess returns on a fund managers portfolio (“fund XXX”) together with the excess returns on a market index: We have some intuition that the beta on this fund is positive, and we therefore want to find whether there appears to be a relationship between x and y given the data that we have. The first s

9、tage would be to form a scatter plot of the two variables.Year, tExcess return= rXXX,t rftExcess return on market index= rmt - rft117.813.7239.023.2312.8 6.9424.216.8517.212.3Introductory Econometrics for Finance Chris Brooks 20027Graph (Scatter Diagram)0510152025303540450510152025Excess return on m

10、arket portfolioExcess return on fund XXXIntroductory Econometrics for Finance Chris Brooks 20028Finding a Line of Best FitWe can use the general equation for a straight line, y=a+bx to get the line that best “fits” the data. However, this equation (y=a+bx) is completely deterministic. Is this realis

11、tic? No. So what we do is to add a random disturbance term, u into the equation.yt = + xt + utwhere t = 1,2,3,4,5Introductory Econometrics for Finance Chris Brooks 20029Why do we include a Disturbance term?The disturbance term can capture a number of features:- We always leave out some determinants

12、of yt- There may be errors in the measurement of yt that cannot be modelled.- Random outside influences on yt which we cannot model Introductory Econometrics for Finance Chris Brooks 200210Determining the Regression CoefficientsSo how do we determine what and are? Choose and so that the (vertical) d

13、istances from the data points to the fitted lines are minimised (so that the line fits the data as closely as possible): y xIntroductory Econometrics for Finance Chris Brooks 200211Ordinary Least SquaresThe most common method used to fit a line to the data is known as OLS (ordinary least squares).Wh

14、at we actually do is take each distance and square it (i.e. take the area of each of the squares in the diagram) and minimise the total sum of the squares (hence least squares).Tightening up the notation, letyt denote the actual data point t denote the fitted value from the regression line denote th

15、e residual, yt - ty ty tu Introductory Econometrics for Finance Chris Brooks 200212Actual and Fitted Value y ix x iy iy iu Introductory Econometrics for Finance Chris Brooks 200213How OLS WorksSo min. , or minimise . This is known as the residual sum of squares. But what was ? It was the difference

16、between the actual point and the line, yt - . So minimising is equivalent to minimising with respect to and . 2524232221uuuuuty tu 512ttu2ttyy2tuIntroductory Econometrics for Finance Chris Brooks 200214Deriving the OLS EstimatorBut , so let Want to minimise L with respect to (w.r.t.) and , so differ

17、entiate L w.r.t. and (1) (2)From (1), But and .ttxytttxyL0)(2ttttxyxL0)(200)(tttttxTyxy yTyt xTxttittttxyyyL22)()(Introductory Econometrics for Finance Chris Brooks 200215Deriving the OLS Estimator (contd)So we can write or (3)From (2), (4)From (3), (5)Substitute into (4) for from (5),0 xyttttxyx0)(

18、xyttttttttttttttxxTxyTyxxxxxyyxxxyyx000)(2220 xTTyTIntroductory Econometrics for Finance Chris Brooks 200216Deriving the OLS Estimator (contd)Rearranging for ,So overall we haveThis method of finding the optimum is known as ordinary least squares.tttyxxyTxxT)(22xyxTxyxTyxtttand22Introductory Econome

19、trics for Finance Chris Brooks 200217 What do We Use and For?In the CAPM example used above, plugging the 5 observations in to make up the formulae given above would lead to the estimates = -1.74 and = 1.64. We would write the fitted line as:Question: If an analyst tells you that she expects the mar

20、ket to yield a return 20% higher than the risk-free rate next year, what would you expect the return on fund XXX to be? Solution: We can say that the expected value of y = “-1.74 + 1.64 * value of x”, so plug x = 20 into the equation to get the expected value for y:06.312064. 174. 1iyttxy64. 174. 1I

21、ntroductory Econometrics for Finance Chris Brooks 200218Accuracy of Intercept EstimateCare needs to be exercised when considering the intercept estimate, particularly if there are no or few observations close to the y-axis: y 0 xIntroductory Econometrics for Finance Chris Brooks 200219The Population

22、 and the SampleThe population is the total collection of all objects or people to be studied, for example, Interested inPopulation of interestpredicting outcomethe entire electorateof an electionA sample is a selection of just some items from the population. A random sample is a sample in which each

23、 individual item in the population is equally likely to be drawn.Introductory Econometrics for Finance Chris Brooks 200220The DGP and the PRFThe population regression function (PRF) is a description of the model that is thought to be generating the actual data and the true relationship between the v

24、ariables (i.e. the true values of and ).The PRF is The SRF is and we also know that .We use the SRF to infer likely values of the PRF.We also want to know how “good” our estimates of and are.ttxytttuxytttyyuIntroductory Econometrics for Finance Chris Brooks 200221LinearityIn order to use OLS, we nee

25、d a model which is linear in the parameters ( and ). It does not necessarily have to be linear in the variables (y and x). Linear in the parameters means that the parameters are not multiplied together, divided, squared or cubed etc.Some models can be transformed to linear ones by a suitable substit

26、ution or manipulation, e.g. the exponential regression modelThen let yt=ln Yt and xt=ln XttttuxytttuttuXYeXeYtlnlnIntroductory Econometrics for Finance Chris Brooks 200222Linear and Non-linear ModelsThis is known as the exponential regression model. Here, the coefficients can be interpreted as elast

27、icities.Similarly, if theory suggests that y and x should be inversely related:then the regression can be estimated using OLS by substituting But some models are intrinsically non-linear, e.g.tttuxyttxz1tttuxyIntroductory Econometrics for Finance Chris Brooks 200223Estimator or Estimate?Estimators a

28、re the formulae used to calculate the coefficientsEstimates are the actual numerical values for the coefficients. Introductory Econometrics for Finance Chris Brooks 200224 The Assumptions Underlying the Classical Linear Regression Model (CLRM)The model which we have used is known as the classical li

29、near regression model. We observe data for xt, but since yt also depends on ut, we must be specific about how the ut are generated. We usually make the following set of assumptions about the uts (the unobservable error terms):Technical NotationInterpretation1. E(ut) = 0The errors have zero mean2. Va

30、r (ut) = 2The variance of the errors is constant and finiteover all values of xt3. Cov (ui,uj)=0The errors are statistically independent of one another4. Cov (ut,xt)=0No relationship between the error andcorresponding x variateIntroductory Econometrics for Finance Chris Brooks 200225The Assumptions

31、Underlying the CLRM AgainAn alternative assumption to 4., which is slightly stronger, is that the xts are non-stochastic or fixed in repeated samples.A fifth assumption is required if we want to make inferences about the population parameters (the actual and ) from the sample parameters ( and )Addit

32、ional Assumption 5. ut is normally distributedIntroductory Econometrics for Finance Chris Brooks 200226Properties of the OLS EstimatorIf assumptions 1. through 4. hold, then the estimators and determined by OLS are known as Best Linear Unbiased Estimators (BLUE). What does the acronym stand for?“Est

33、imator” - is an estimator of the true value of .“Linear”- is a linear estimator“Unbiased”- On average, the actual value of the and s will be equal to the true values.“Best”- means that the OLS estimator has minimum variance among the class of linear unbiased estimators. The Gauss-Markov theorem prov

34、es that the OLS estimator is best.Introductory Econometrics for Finance Chris Brooks 200227Consistency/Unbiasedness/EfficiencyConsistentThe least squares estimators and are consistent. That is, the estimates will converge to their true values as the sample size increases to infinity. Need the assump

35、tions E(xtut)=0 and Var(ut)=2 0.5 rather than 0.5or we could have hadH0 : = 0.5H1 : critical F-value.Introductory Econometrics for Finance Chris Brooks 2002102Determining the Number of Restrictions in an F-testExamples :H0: hypothesisNo. of restrictions, m1 + 2 = 212 = 1 and 3 = -122 = 0, 3 = 0 and

36、4 = 03If the model is yt = 1 + 2x2t + 3x3t + 4x4t + ut,then the null hypothesisH0: 2 = 0, and 3 = 0 and 4 = 0 is tested by the regression F-statistic. It tests the null hypothesis that all of the coefficients except the intercept coefficient are zero.Note the form of the alternative hypothesis for a

37、ll tests when more than one restriction is involved: H1: 2 0, or 3 0 or 4 0Introductory Econometrics for Finance Chris Brooks 2002103What we Cannot Test with Either an F or a t-testWe cannot test using this framework hypotheses which are not linear or which are multiplicative, e.g.H0: 2 3 = 2 or H0: 2 2 = 1 cannot be tested.Introductory Econometrics for Finance Chris Brooks 2002104The Relationship between the t and the F-DistributionsAny hypothesis which could be tested with a t-test could have been tested using an F-test, but not the other way around.