EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS Vol. 3, No. 3, 2010, 347-369 ISSN 1307-5543 – www.ejpam.com SPECIAL ISSUE ON GRANGER ECONOMETRICS AND STATISTICAL MODELING DEDICATED TO THE MEMORY OF PROF. SIR CLIVE W.J. GRANGER A Modern Approach to Teaching Econometrics David F. Hendry∗ and Bent Nielsen Economics Department, Oxford University Abstract. We explain the computer-based approach to the teaching of econometrics used in Oxford from the elementary to the advanced. The aims are to enable students to critically evaluate published applied studies and to undertake empirical research in economics. The unified theoretical framework is that of likelihood, using likelihood-ratio tests for inference and evaluation, and focusing on develop- ing well-specified empirical models of interesting economic issues. A sequence of increasingly realistic models is developed from independent, identically distributed binary data through to selecting cointe- grated equations in the face of structural breaks–in a one-year course. 2000 Mathematics Subject Classifications: 62E01, 62E07 Key Words and Phrases: Clive Granger; Teaching Econometrics; model evaluation; likelihood; struc- tural breaks Preface by David Hendry Although my first interchanges with Clive Granger involved disagreements over modeling non-stationary economic time series, that stimulus led to his famous formulation of the con- cept of cointegration, and associated developments in econometric modeling (see [20, 34], and http://nobelprize.org/nobel_prizes/economics/laureates/2003/granger-lecture.pdf , based on [13], and [7] ). Clive was already well known both for his ideas on causality [12], appraised in [26], and distinguished from exogeneity in [8]), and for re-emphasizing the dangers in applying static regression models to integrated data (in [16], following pio- neering research by [38]). From my first visit to the University of California at San Diego in 1975, where Clive had moved in 1974, our friendship blossomed, built around a common ∗Corresponding author. Email address: david.hendry@nuffield.ox.ac.uk (D. Hendry) bent.nielsen@nuffield.ox.ac.uk (B. Nielsen) http://www.ejpam.com 347 c⃝ 2010 EJPAM All rights reserved. D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 348 desire to improve the quality of econometric model building, especially by a better match to the empirical evidence: Clive’s contributions to doing so have been one of the most successful research programmes in econometrics, and are a lasting contribution (for formal Obituaries, see [29], and [28]). My own approach focused on implementing modeling methods, and led to our only joint publication [14], discussing automatic modeling. Clive also kept the theory of economic forecasting under the spotlight when it was not in fashion [see 15, 17], another interest we had in common (including our accidentally adopting the same title in [3]). In addition to his astonishing creativity and innovative ideas, Clive was a master of written and presentational clarity, so we also shared a desire to communicate both with students (Clive supervised a large number of successful doctoral students) and colleagues on a world-wide basis. The paper which follows develops modeling ideas in the teaching domain, where ma- jor changes in how we explain and teach econometrics to the next generation could further enhance the quality with which econometrics is applied to the many pressing problems facing the world. 1. Introduction There are six reasons why now is a good time to review the teaching of econometrics. Over the last quarter century, there have been: (1). massive changes in the coverage, approaches, and methods of econometrics; (2). huge improvements in computer hardware and computational methods;† (3). improvements to software, data, and graphics capabilities, which have been at least as impressive; (4). considerable advances in teaching methods, from mathematical derivations written on blackboards, through overheads to live computer projection; (5). few discussions of computer-based teaching of econometrics since [18] proposed an approach based on PcGive [21], describe its history); (6). new approaches to teaching econometrics [see e.g., 27], emphasizing empirical model- ing. The last is the focus of this paper, where every student is taught while having continuous computer access to automatic modeling software (in Oxford, based on [22], within OxMetrics: see [6]). Computer-based teaching of econometrics is feasible at all levels, from elementary, through intermediate, to advanced. We cannot cover all those aspects, and will only briefly describe how we teach the first steps in econometrics, noting some tricks that help retain student interest, before moving on to model selection in non-stationary data. Such ideas can even be communicated to less mathematically oriented undergraduates, enabling them to †These have, of course, been ongoing from hand calculations at its start in the 1930s, through punched card/tape- fed mainframe computers, workstations, to powerful PCs and laptops. D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 349 progress in a year from introducing independent, identically distributed (IID) binary data to selecting cointegrated equations in the face of structural breaks. At the outset, we assume no knowledge of even elementary statistical theory, so first ex- plain the basic concepts of probability; relate those to distributions, focusing on location, spread and shape; turn to elementary notions of randomness; then apply these ideas to distri- butions of statistics based on data. There are six central themes: • likelihood; • testing assumptions; • economically relevant empirical applications; • mastery of software to implement all the procedures; • an emphasis on graphical analysis and interpretation; • rigorous evaluation of all “findings“. Derivations are only used where necessary to clarify the properties of methods, concepts, formulations, empirical findings, and interpretations, but also simultaneously upskill students’ mathematical competence. By adopting the common framework of likelihood, once that fun- damental idea is fully understood in the simplest IID binary setting, generalizations to more complicated models and data generation processes (DGPs) follow easily. A similar remark applies to evaluation procedures, based on likelihood ratio tests: the concept is the same, even though the distribution may alter with more complicated and realistic DGPs with time dependence, non-stationary features etc. (which include changes in means and variances as well as stochastic trends). The theory and empirical sections of the course proceed in tandem. Once a given statistical or econometric idea has been introduced, explained and illustrated, it is applied in a computer class where every student has their own workstation on line to the database and software. Sections 2 and 3 describe the first steps in theory and practice respectively, then §4 dis- cusses regression analysis graphically and as a generic “line-fitting” tool. Those set the scene for introducing simple models and estimation in §5, leading to more general models in §6, and model selection in §7. Section 8 notes some implications for forecasting–Clive’s other great interest–and §9 concludes. 2. Theory First Steps To introduce elementary statistical theory, we consider binary events in a Bernoulli model with independent draws, using sex of a child at birth as the example. This allows us to explain sample versus population distributions, and hence codify these notions in distribution functions and densities. Next, we consider inference in the Bernoulli model, discussing expectations and variances, then introduce elementary asymptotic theory (the simplest law of large numbers and central limit theorem for IID processes) and inference. D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 350 It is then a small generalization to consider continuous variables, where we use wages (wi) in a cross section as the example. The model thereof is the simplest case, merely wi = β + ui , where β is the mean wage and ui characterizes the distribution around the mean. Having established that case, regression is treated as precisely the same model, so is already familiar. Thus, building on the simple estimation of means leads to regression, and hence to logit regression, and then on to bivariate regression models. 2.1. The Way Ahead In the lectures, regression is applied to an autoregressive analysis of the Fulton fish-market data from [11]. The natural next step is to model price and quantity jointly as a system, lead- ing to simultaneity and identification, resolved using as instruments dummies for “structural breaks” induced by stormy weather at sea. In the system, over-identified instrumental vari- ables regression is simply reduced-rank regression. This makes it easy to move on to unit roots and a system analysis of cointegration, picking up model selection issues en route, and illus- trating the theory by Monte Carlo simulation experiments. Thus, each topic segues smoothly into the next. 3. Empirical First Steps Simultaneously, we teach OxMetrics and PcGive so that students can conduct their own em- pirical work. Here, we focus on the computer-class material, which moves in parallel, but ana- lyzes different data each year. We have collected a large databank of long historical time series from 1875–2000 (available at http://press.princeton.edu/titles/8352.html), and offer the students the choice of modeling any one of the key series, such as the unemploy- ment rate (denoted Ur), gross domestic product (g, where a lower-case letter denotes the log of the corresponding capital), price inflation (∆p, where P is the implicit deflator of G), or real wages (w − p). We assume students chose Ur , leading to a classroom-wide discussion of possible economic theories of unemployment based around supply and demand for a factor of production–but replete with ideas about “too high real wages” , “lack of demand for labour” , “technological change” , “population growth” , “(im)migration” , “trade union power” , “overly high unemployment benefits” etc.–as well as the relevant institutions, including companies, trade unions and governments. An advantage of such a long-run data series is that many vague claims are easily rebutted as the sole explanation, albeit that one can never refute that they may be part of the story. The next step is to discuss the measurement and sources of data for unemployment and working population, then get everyone to graph the level of Ur (here by OxMetrics as in fig. 1). It is essential to carefully explain the interpretation of graphs in considerable detail, covering the meaning of axes, their units, and any data transforms, espe- cially the roles and properties of logs. Then one can discuss the salient features, such as major events, cycles, trends and breaks, leading to a first notion of the concept of non-stationarity, an aspect that Clive would have liked. It is also easy to relate unemployment to the student’s own life prospects, and those of their fellow citizens. Everyone in the class is made to make a new comment in turn–however simple–about some aspect of the graph, every time. Neverthe- D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 351 1880 1900 1920 1940 1960 1980 2000 0.025 0.050 0.075 0.100 0.125 0.150 Ur ,t Figure 1: Graph of Historical Data on UK Unemployment Rate. less, it is usually several sessions before students learn to mention the axes’ units first. We aim to produce students who will be able to critically interpret published empirical findings, and sensibly conduct their own empirical research, so one cannot finesse difficulties such as non- stationarity and model selection. The non-stationarity visible in whatever series is selected is manifest(e.g., fig. 1), including shifts in means and variances, any “epochs” of markedly different behaviour, and changes in persistence. Figure 2 shows possible general comments, whereas fig. 3 adds specific details, most of which might arise in discussion. Thus, a detailed knowledge of the historical context is imparted as a key aspect of modeling any time series. This also serves to reveal that most of the major shifts are due to non-economic forces, espe- cially wars and their aftermaths. Next, the students are required to calculate the differences of Ur,t , namely ∆Ur,t = Ur,t − Ur,t−1 and plot the resulting series as in fig. 4. Again, everyone is asked to discuss the hugely different “look” of the graph, especially its low persistence, con- stant mean of zero, and how it relates to fig. 1, including the possibility that changes in the reliability and coverage of the data sources may partly explain the apparent variance shift, as well as better economic policy. ∆Ur,t is clearly not stationary, despite differencing, another key lesson. 3.1. Theory and Evidence A further feature of our approach is to relate distributional assumptions to model formula- tion. The basic example is conditioning in a bivariate normal distribution, which is one model for linear regression. In turn that leads naturally to the interpretation of linear models and their assumptions, and hence to model design, modeling, and how to judge a model. Here, the key concepts are a well-specified model (matching the sample distribution), congruence (also matching the substantive context), exogeneity (valid conditioning), and encompassing (explaining the gestalt of results obtained by other models of the same phenomena). These are deliberately introduced early on when only a few features need to be matched, as they will D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 352 1880 1900 1920 1940 1960 1980 2000 0.025 0.050 0.075 0.100 0.125 0.150 upward mean shift and higher variance ↓ upward mean shift→ downward mean shift and dramatically lower variance ↓ Business cycle epoch ↓ Units = rate Figure 2: General Comments about Unemployment Rate. be important when models become larger and more complicated, usually requiring computer- based automatic selection. In fact, it is difficult to formulate simple theoretical models of unemployment with any ability to fit the long non-stationary sample of data in fig. 1. Consequently, to start down the road of linking theory and evidence in a univariate model, we postulate a “golden-growth” explanation for deviations of unemployment from its historical mean, so assume that Ur,t is non-integrated yet non-stationary. The measure of the steady-state equilibrium determinant is given by: dt = RL,t −∆pt −∆gt (1) where RL,t is the long bond rate: precise definitions of the data are provided in [19], who also develops a model based on (1). When the real cost of capital � RL,t −∆pt � exceeds the real growth rate ∆gt , then dt > 0 so the economy will slow and Ur,t will rise; and conversely when dt < 0, so Ur,t will converge to its historical mean for dt = 0. Of course all the variables in (1) are jointly determined with Ur,t in any equilibrium, so the resulting “explanation” is conditional, but allows a simple yet viable model to be developed, with the possibility of later extension to a multivariate setting. As ever, we first look at the graphs of each series as in fig. 5 a, and discuss their properties, then consider dt in relation to Ur,t in panel b (where the series are adjusted for means and ranges to maximize the visual correlation). 4. Regression concepts Once regression theory has been explained, it is easily illustrated by a scatter plot, as in fig. 6a. Adding a single line allows one to explain the slope and intercept graphically. By further D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 353 1880 1900 1920 1940 1960 1980 2000 0.025 0.050 0.075 0.100 0.125 0.150 upward mean shift and higher variance ↓ upward mean shift→ downward mean shift and dramatically lower variance ↓ Business cycle epoch ↓ WWI → ← WWII ← Oil crisis ←Postwar crash ←Mrs T Boer war → US crash → ←Post−war reconstruction ←leave ERM Units−rate ←leave gold standard Figure 3: All Comments about Unemployment Rate. adding the projections from the data to the line, least squares can be understood visually as the line that minimizes the squared deviations: see fig. 6b. Such graphs now reveal that the Ur,t on dt relation is fine in the tails, but “erratic” in the middle. Having established the basics of regression, a more penetrating analysis moves on to the five key concepts that underpin linear regression interpreted as conditioning in a bivariate normal distribution: • exogeneity of the regressor; • IID errors; • normality; • linear functional form; • parameter constancy. Failure on any of these induces model mis-specification, and likelihood ratio, or equivalent, tests of the corresponding assumptions can be introduced. At each stage, we relate the maths derivations as needed for understanding the graphs–but always using the same basic prin- ciples: data graphs suggest a putative DGP and hence a model of its distribution function, leading to the likelihood function. Maximize that likelihood as a function of the postulated parameters, obtaining an appropriate statistic from the score equation, and derive its distri- bution. Finally, apply the resulting method to the data, interpret the results and evaluate the findings to check how well the evidence matches the assumed DGP. The same approach is just applied seriatim to ever more complicated cases. D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 354 1880 1900 1920 1940 1960 1980 2000 −0.04 −0.02 0.00 0.02 0.04 0.06 0.08 High variance Low variance Constant mean of zero Are changes random? ∆Ur Figure 4: Changes in the Unemployment Rate. 4.1. Regression as “non-parametric” To “demistify” regression analysis as just line fitting, use the “pen” in OxMetrics to have each students write his/her name on a graph, then run a regression through it: see fig. 6c. The pixels are mapped to world coordinates (which can be explained using the graphics editor), so become “data” in the (Ur,t , dt) space, and hence one can estimate a regression for that subset. Consequently, projections can even be added to the signature regression. Most students are intrigued by this capability, and many gain important insights, as well as being amused. Next, show how to join up the original data points to create “Phillips’ loops” [see 33], tracing out the dynamics as in fig. 6d. This serves to highlight the absence of the time dimension from the analysis so far, and is a convenient lead into taking account of serial dependence in data. Many routes are possible at this point–one could further clarify the underlying concepts, or models, or methods of evaluation. We illustrate several sequential regressions graphically as that leads to recursive methods for investigating parameter constancy: see fig. 7. Alter- natively, OxMetrics graphs provide an opportunity to introduce the basics of LATEXby naming variables, as shown in the graphs here, or by writing formulae on the figures. Even minimal LATEXskills will prove invaluable later as estimated models can be output that way, and pasted directly into papers and reports (as used below). 4.2. Distributions Graphics also ease the visualization of distributions. Plotting the histograms of Ur,t and ∆Ur,t with their interpolated densities yields fig. 8. Such figures can be used to explain non- parametric/kernel approaches to density estimation, or simply described as a “smoothed” histogram. More importantly, one can emphasize the very different features of the density graphs for the level of Ur,t and its change. The former is like a uniform distribution–many values are roughly equally likely. The distribution for ∆Ur,t is closer to a normal with some outliers. Thus, differencing alters distributional shapes. This can be explained as the uncondi- D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 355 1880 1900 1920 1940 1960 1980 2000 −0.1 0.0 0.1 0.2 aRL,t−∆pt ∆gt 1880 1900 1920 1940 1960 1980 2000 0.00 0.05 0.10 0.15 bUr,t dt Figure 5: Graphs of RL,t −∆pt and ∆gt ; Ur,t and dt . tional distribution of Ur,t versus its distribution conditional on the previous value, so panel b is plotting the distribution of the deviation of Ur,t from Ur,t−1. 4.3. Time Series and Randomness That last step also serves to introduce the key concept of non-randomness. Regression subsumes correlation, which has by now been formally described, so can be used to explain correlograms as correlations between successively longer lagged values: fig. 9 illustrates. The plots reveal that Ur,t has many high autocorrelations–indeed the successive autocorrelations almost lie on a downward linear trend–whereas ∆Ur,t has almost no autocorrelation at any lag. Thus, changes in Ur,t are “surprise-like”: again this comparison highlights the huge difference between unconditional and conditional behaviour. In turn, we can exploit the different behaviour of Ur,t and ∆Ur,t to introduce dynamics, by plotting Ur,t against its own lag Ur,t−1 then graphically adding the regression, as in fig. 10a (and panel b for ∆Ur,t). 4.4. Well-specified Models Now one can explain well-specified models as needing all the properties of the variables in a model to match simultaneously–in terms of dynamics, breaks, distributions, linear relations, etc.– otherwise there will be systematic departures from any claimed properties. Tests of each null hypothesis are then discussed, albeit using Lagrange multiplier approximate F-tests rather than likelihood ratio, namely: • Far for kth-order serial correlation as in [10] and [32]; • Fhet for heteroskedasticity as in [37]; D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 356 Ur,t × dt −0.2 −0.1 0.0 0.1 0.2 0.00 0.05 0.10 0.15 aUr,t × dt Ur,t × dt −0.2 −0.1 0.0 0.1 0.2 0.00 0.05 0.10 0.15 bUr,t × dt Ur,t × dt −0.2 −0.1 0.0 0.1 0.2 0.00 0.05 0.10 0.15 c . . Ur,t × dt Ur,t × dt −0.2 −0.1 0.0 0.1 0.2 0.00 0.05 0.10 0.15 dUr,t × dt Figure 6: Scatter Plots and Regressions of Ur,t on dt . • Freset for functional form following [35]); • Farch for kth-order autoregressive conditional heteroskedasticity, from [? ]; • FChow for parameter constancy over k periods as in [2]; and • χ2 nd(2) for normality (a chi-square test: see [5]). Below ∗ and ∗∗ denote significant at 5% and 1% respectively. Having established the basics, the scene is set for formal estimation of a regression. 5. Model estimation Estimating the static or long-run (later interpreted as “cointegrated”) relation Ur,t = β0 + β1dt + et yields: bUr,t = 0.050 (0.003) + 0.345 (0.052) dt (2) R2 = 0.26 bσ = 0.0315 FGUM(1,126) = 44.64∗∗ Here, R2 is the squared multiple correlation, bσ is the residual standard deviation, and co- efficient standard errors are shown in parentheses. The test FGUM is for significance of the general unrestricted model, that is the joint significance of all regressors (dt) apart from the intercept. The estimates suggest that unemployment rises/falls as the real long-run interest rate is above/below the real growth rate (i.e., dt ≶ 0). All the assumptions are easily tested, yielding Far(2,124) = 180.4∗∗; Farch = 229.9∗∗; Freset(1,125) = 0.33; Fhet(2,123) = 2.62; D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 357 −0.20 −0.15 −0.10 −0.05 0.00 0.05 0.10 0.15 0.20 0.000 0.025 0.050 0.075 0.100 0.125 0.150 10 regressions of Ur,t on dt for T/10 subsets by increasing values of dt. Ur,t × dt Figure 7: Regressions for Ur,t on dt for each Tenth of the Data. χ2 nd(2) = 15.0∗∗. These tests show that the model is poorly specified. Figure 11 records the fitted and actual values, their cross-plot, the residuals scaled by bσ, and their histogram and density with N[0,1] for comparison, visually confirming the formal tests. Once again, it is clear that the model is badly mis-specified, but it is not clear which assumptions are invalid. However, we have now successfully applied the key concepts to residuals. 5.1. Simple Dynamic Models Another univariate model worth illustrating is that of Ur,t on Ur,t−1, namely Ur,t = γ0 + γ1Ur,t−1 + εt . This form was implicit in fig. 10–and can also be related to the earlier graphs for ∆Ur,t : bUr,t = 0.006 (0.003) + 0.887 (0.040) Ur,t−1 (3) R2 = 0.79 bσ = 0.017 FGUM(1,126) = 485.7∗∗ χ2 nd(2) = 33.0∗∗ Far(2,124) = 3.8∗ Farch(2,124) = 0.55 Fhet(2,123) = 0.42 Freset(1,125) = 0.01 Most of the mis-specification tests are considerably improved, but the model in (3) is still mis-specified, with an obvious outlier in 1920, as fig. 12(b) shows. The long-run solution in (3) is 0.006/(1− 0.887) or 5.3% unemployment–which is close to the intercept in (2)–and although one cannot in fact reject the hypothesis of a unit root, that provides an opportunity to explain the rudiments of stochastic trends, possibly illustrated by Monte Carlo simulation of the null distribution. However, this is crunch time: having postulated our models of the D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 358 −0.02 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 5 10 15 Density plots aUr −0.06 −0.04 −0.02 0.00 0.02 0.04 0.06 0.08 0.10 10 20 30 40 b∆Ur Figure 8: Distribution of the Unemployment Rate Ur,t and its Change ∆Ur,t . DGP, we find strong rejection on several tests of specification, so something has gone wrong. Multiple testing concepts must be clarified: each test is derived under its separate null, but assuming all other aspects are well specified. Consequently, any other mis-specification re- jections contradict the assumptions behind such derivations: once any test rejects, none of the others is trustworthy as the assumptions underlying their calculation are also invalidated. Moreover, simply “correcting” any one problem–such as serial correlation–need not help, as the source may be something else altogether, such as parameter non-constancy over time. A more viable approach is clearly needed–leading to general-to-simple... 6. More General Models It is time to introduce a dynamic model which also has regressors, nesting both (3) and (2), namely Ur,t = β0+ β1dt + β2Ur,t−1+ β3dt−1+υt . Estimation delivers: bUr,t = 0.007 (0.002) + 0.24 (0.03) dt + 0.86 (0.04) Ur,t−1− 0.10 (0.03) dt−1 (4) R2 = 0.88 bσ = 0.013 FGUM(3,123) = 308.2∗∗ Far(2,121) = 2.5 χ2 nd(2) = 7.2∗ Farch(1,121) = 3.1 Fhet(6,116) = 4.2∗∗ Freset(1,122) = 4.2∗ Although (4) is not completely well-specified, it is again much better, and certainly dominates both earlier models, as F-tests based on the “progress” option in OxMetrics reveal. While illustrating progressive research, the exercise also reveals the inefficiency of commencing with overly simple models, as nothing precluded commencing from (4). Assuming cointegration has been explained, one can show that a unit root can be rejected in (4) (tur =−3.9∗∗ on the D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 359 0 5 10 15 20 −1.0 −0.5 0.0 0.5 1.0 ACF−Ur 0 5 10 15 20 −1.0 −0.5 0.0 0.5 1.0 ACF−∆Ur Figure 9: Correlograms for the Unemployment Rate Ur,t and its Change ∆Ur,t . PcGive unit-root test: see [1, 9]), so Ur and d are “cointegrated” (or co-breaking, as in [25]). Next, the long-run solution can be derived by taking the expected value of the error as zero, and setting the levels to constants such that:� 1− β2 � U∗r = β0+ � β1+ β3 � d∗, so: U∗r = β0 1− β2 + β1+ β3 1− β2 d∗, which yields U∗r = 0.052+ 1.02d∗ for the estimates in (4). The coefficient of d at unity is much larger than that of 0.35 in (2), and suggests a one-for-one reaction in the long run. 6.1. Extensions There is as much to discuss as ones desires at this stage. For example, there are few out- liers, but there are some negative fitted values (suggesting a logit formulation, which may also attenuate the residual heteroscedasticity). Challenge students to formulate alternative expla- nations, and test their proposals against the evidence–and see if they can encompass (4), by explaining its performance from their model. One can also check model constancy by formal recursive methods, building on the earlier graphical approach. Figure 14 records the outcome for (4): despite the apparently “wandering” estimates, the constancy tests–which are scaled by their 1% critical values–do not reject. It is surprising that such a simple representation as (4) can describe the four distinct epochs of unemployment about equally accurately. 7. Model Selection Having shown the dangers of simple approaches, general-to-specific model selection needs to be explained. In a general dynamic model, one cannot know in advance which variables will D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 360 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 0.11 0.12 0.13 0.14 0.15 0.05 0.10 0.15 a Ur,t × Ur,t−1 −0.05 −0.04 −0.03 −0.02 −0.01 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 −0.05 0.00 0.05 0.10 b∆Ur,t × ∆Ur,t−1 Figure 10: Unemployment Rate Ur,t and its Change Regressed on Their Own First Lag. matter: some will, but some will not, so selection is required. Indeed, any test followed by a decision entails selection, so in empirical research, selection is ubiquitous, however unwilling practitioners are to admit its existence. “Model uncertainty” is pandemic–every aspect of an empirical model is uncertain, from the existence of any such relation in reality, the viability of any “corroborating” theory, and the measurements of the variables, as well as the choice of the specification and every assumption needed in the formulation, such as exogeneity, constancy, linearity, independence etc. One must confront such issues openly if graduating students are to be competent practitioners. It is feasible to sketch the theory of model selection in the simplest case. We use the idea of choosing between two decisions, namely keeping or eliminating a variable, where there are two states of nature, namely the variable is in fact relevant or irrelevant in that setting. The mistakes are “retain an irrelevant variable” and “exclude a relevant variable”, akin to probabilities of type I and II errors. Consider the perfectly orthogonal, correctly specified regression model: yt = β1z1,t + β2z2,t + εt (5) where all variables have zero means, E[z1,tz2,t] = 0, the βi are constant, and εt ∼ IN � 0,σ2 ε � . Denote the t2-statistics testing H0: β j = 0 by t2 j , and let cα be the desired critical value for retaining a variable in (5) when t2 j ≥ c2 α. When either (or both) β j = 0 in (5), the probability of falsely rejecting the null is determined by the choice of cα–conventionally set from α = 0.05. There is a 5% chance of incorrectly retaining one of the variables on t2, but a negligible probability (0.0025) of retaining both. When one (or both) β j ̸= 0, the power of the t2-statistic to reject the null depends on the non-centrality β2 j /V �bβ j � ≃ Tβ2 j σ 2 z j /σ2 ε where E[z2 j,t] = σ 2 z j : this can be evaluated by simulation. Thus, all the factors affecting the D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 361 1900 1950 2000 0.00 0.05 0.10 0.15 Ur ,t Ûr ,t 1900 1950 2000 −1 0 1 2 3 Ur ,t scaled residuals −3 −2 −1 0 1 2 3 4 0.1 0.2 0.3 0.4 0.5 density of Ur ,t residuals N(0,1) 0 5 10 −0.5 0.0 0.5 1.0 correlogram of Ur ,t residuals Figure 11: Graphical Output from Ur on d. outcome of selection are now in place. 7.1. Understanding Model Selection The interesting case, however, is generalizing to: yt = N∑ i=1 βizi,t + εt (6) where N is large (say 40). Order the N sample t2-statistics as t2 (N) ≥ t2 (N+1) ≥ · · · ≥ t2 (1), then the cut-off between included and excluded variables is given by t2 (n) ≥ c2 α > t2 (n−1), so n are retained and N − n eliminated. Thus, variables with larger t2 values are retained on average, and all others are eliminated. Importantly, only one decision is needed to select the model even for N = 1000 when there are 21000 = 10301 possible models. Consequently, “repeated testing” does not occur, although path searches during model reduction may give the impression of “repeated testing”. Moreover, when N is large, one can set the average false retention rate at one irrelevant variable by setting α = 1/N , so αN = 1, at a possible cost in lower correct retention. Of course, there is sampling uncertainty, as the t2 j are statistics with distributions, and on any draw, those close to c2 α could randomly lie on either side–for both relevant and irrelevant variables. It is important to explain the key role of such marginal decisions: empirical t2- values close to the critical value c2 α are the danger zone, as some are likely to arise by chance D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 362 1900 1950 2000 0.05 0.10 0.15 aUr ,t Ûr ,t 1900 1950 2000 −2.5 0.0 2.5 5.0 b Ur ,t scaled residuals −4 −2 0 2 4 6 0.2 0.4 0.6 Density c density of Ur ,t residuals N(0,1) 0 5 10 −0.5 0.0 0.5 1.0 dcorrelogram of Ur ,t residuals Figure 12: Ur,t on Ur,t−1 Graphical Output. for irrelevant variables, even when α is as small as 0.001. Fortunately, it is relatively easy to explain how to bias correct the resulting estimates for sample selection, and why doing so drives estimates where t2 j just exceeds c2 α close to zero [see e.g. 24]. Students have now covered the basic theory of Autometrics [see 4], and, despite their inexperience, can start to handle realistically complicated models using automatic methods, which has led to a marked improvement in the quality of their empirical work. Nevertheless, a final advance merits discussion, namely handling more variables than observations in the canonical case of impulse-indicator saturation. 7.2. Impulse-indicator Saturation The basic idea is to “saturate” a regression by adding T indicator variables to the candi- date regressor set. Adding all T indicators simultaneously to any equation would generate a perfect fit, from which nothing is learned. Instead, exploiting their orthogonality, add half the indicators, record the significant ones, and remove them: this step is just “dummying out” T/2 observations as in [36]. Now add the other half, and select again, and finally combine the results from the two models and select as usual. A feasible algorithm is discussed in [23] for a simple location-scale model where x i ∼ IID � µ,σ2 x � , and is extended to dynamic pro- cesses by [31]. Their theorem shows that after saturation, eµ is unbiased, and αT indicators are retained by chance on average, so for α = 0.01 and T = 100, then 1 indicator will be retained by chance under the null even though there are more variables than observations. Thus, the procedure is highly efficient under the null that there are no breaks, outliers or data D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 363 1900 1950 2000 0.00 0.05 0.10 0.15 Ur ,t Ûr ,t 1900 1950 2000 −2 0 2 Ur ,t scaled residuals −3 −2 −1 0 1 2 3 4 0.2 0.4 0.6 density of Ur ,t residuals N(0,1) 0 5 10 −0.5 0.0 0.5 1.0 correlogram of Ur ,t residuals Figure 13: General Ur,t Model Graphical Output. contamination. Autometrics uses a more sophisticated algorithm than just split halves, [see 4], but the selection process is easily illustrated live in the classroom. Here we start with 2 lags of both Ur,t and dt and set α = 0.0025 as T = 126. Selection locates 8 significant outliers (1879, 1880, 1884, 1908, 1921, 1922, 1930, and 1939: the 19th century indicators may be due to data errors), and yields (not reporting the indicators):‡bUr,t = 0.004 (0.0015) + 0.15 (0.02) dt + 1.29 (0.06) Ur,t−1− 0.09 (0.02) dt−1− 0.39 (0.06) Ur,t−2 (7) R2 = 0.95 bσ = 0.008 Far(2,111) = 1.59 χ2 nd(2) = 7.98∗ Farch(1,1249) = 0.09 Fhet(14,103) = 1.03 Freset(2,111) = 1.41 The long-run solution from (7) is U∗r = 0.05+ 0.62d∗ so has a coefficient of d that is smaller than in (4). However, no diagnostic test is significant other than normality, and the model is congruent, other than the excess of zero residuals visible in the residual density. 7.3. Monte Carlo of Model Selection How to evaluate how well such general-to-specific model selection works? Everyone in the class generates a different artificial sample from the same DGP, which they design as a ‡Three more indicators for 1887, 1910 and 1938 are retained at α= 0.01, with a similar equation. D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 364 1900 1920 1940 1960 1980 2000 0.50 0.75 1.00 Ur,t−1 × ± 2SE 1900 1920 1940 1960 1980 2000 0.00 0.02 Intercept × ± 2SE 1900 1920 1940 1960 1980 2000 0.25 0.50 0.75 dt × ± 2SE 1900 1920 1940 1960 1980 2000 −0.2 0.0 0.2 0.4 dt−1 × ± 2SE 1900 1920 1940 1960 1980 2000 −0.025 0.000 0.025 1−step residuals with 0± 2σ̂ 1900 1920 1940 1960 1980 2000 0.5 1.0 1−step Chow tests 1% 1900 1920 1940 1960 1980 2000 0.5 1.0 break−point Chow tests 1% Figure 14: Ur,t Model Recursive Output. group, then they all apply Autometrics to their own sample. Pool the class results and relate the outcomes to the above theoretical “delete/keep” calculations–then repeat at looser/tighter selection criteria, with and without impulse-indicator saturation to see that the theory matches the practice, and works. 7.4. Evaluating the Selection At an advanced level, exogeneity issues can be explored, based on impulse-indicator satu- ration applied to the marginal model for the supposedly exogenous variable [see 30]. Here, that would be dt , so develop a model of it using only lagged variables and indicators: Au- tometrics takes under a minute from formulation to completion at (say) α = 0.0025, as for (7). dt = 0.55 (0.05) dt−1 − 0.16 (0.03) I1915 − 0.13 (0.03) I1917 + 0.24 (0.03) I1921 + 0.12 (0.03) I1926 + 0.10 (0.03) I1931 − 0.14 (0.03) I1940 − 0.09 (0.03) I1975 (8) bσ = 0.028 Far(2,116) = 1.32 Farch(1,124) = 0.19 χ2 nd(2) = 2.20 Fhet(2,116) = 4.72∗ Freset(2,116) = 4.52∗ Only the indicator for I1921 is in common, and the remainder are not, so must all co-break with (7). All the dates correspond to recognizable historical events, albeit that other important D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 365 Ur ,t Ûr ,t 1900 1950 2000 0.00 0.05 0.10 0.15 Ur ,t Ûr ,t Ur ,t scaled residuals 1900 1950 2000 −2 0 2 Ur ,t scaled residuals Ur ,t residual density N(0,1) −4 −2 0 2 4 0.2 0.4 Ur ,t residual density N(0,1) Ur ,t residual correlogram 0 5 10 −0.5 0.0 0.5 1.0 Ur ,t residual correlogram Figure 15: Graphical Description of Final Model of Ur,t . dates are nor found. Even that for 1921 (one of most eventful years for the UK) is 0.056 in (7) as against 0.24 in (8), so does not suggest that a break in the latter is communicated to the former, which would violate exogeneity. Adding the indicators from (8) to (7), however, delivers F(6,107) = 4.28∗∗ so strongly rejects exogeneity, even beyond the 0.0025 level used in selection. The main “culprit” is 1975 (which was omitted from (7) by Autometrics as it induced failure in several diagnostic tests), but interestingly, the long-run solution is now U∗r = 0.042+ 1.02d∗ so is back to the original. None of the indicators from (7) is significant if added to (8). 8. Forecasting First, one must establish how to forecast. Given the unemployment equation above, for 1-step forecasts, Ur,T , dT are known, the past indicators are now zero, but dT+1 needs to be forecast if (7) is to be used for bUr,T+1. Thus, a system is needed, and is easily understood in the 2-stages of forecasting dt+1 from (8):bdT+1 = 0.55 (0.05) dT (9) and use that in:bUr,T+1 = 0.004 (0.0015) + 0.15 (0.02) bdT+1+ 1.29 (0.06) Ur,t − 0.09 (0.02) dT − 0.39 (0.06) Ur,T−1 (10) D. Hendry and B. Nielsen / Eur. J. Pure Appl. Math, 3 (2010), 347-369 366 Brighter students rapidly notice that the net effect of d on the forecast outcome is essentially zero as substituting (9) into (10) yields 0.55× 0.15− 0.09 = −0.0075. Thus, the forecast model is no better than an autoregression in Ur,t . Indeed simply selecting that autoregression delivers (indicators not reported):bUr,t = 1.29 (0.06) Ur,t−1 − 0.34 (0.06) Ur,t−2 with bσ = 0.0096. This is the first signpost that “forecasting is different”. That “vanishing trick” ~Ur Ur 1960 1970 1980 1990 2000 0.00 0.05 0.10 1980 on → ~Ur Ur ~d d 1960 1970 1980 1990 2000 −0.05 0.00 0.05 1980 on → ~d d ~Ur Ur 1970 1980 1990 2000 0.00 0.05 0.10 0.15 1982 on → ~Ur Ur ~d d 1970 1980 1990 2000 −0.05 0.00 0.05 1982 on → ~d d Figure 16: Dynamic Forecasts of Ur,t and dt over 1980�2001. would have been harder to spot when the model was expressed in equilibrium-correction form to embody the long-run relation e = Ur − 0.05− d as a variable: ∆bUr,t = 0.37 (0.05) ∆Ur,t−1 + 0.17 (0.02) ∆dt − 0.07 (0.02) et−1 Since multiple breaks have already been encountered, it is easy to explain the real problem confronting economic forecasting, namely breaks. Simply extrapolating an in-sample esti- mated model (or a small group of models pooled in some way) into the future is a risky strategy in processes where location shifts occur. Here, the key shift would be in the equi- librium mean of 5% unemployment, and that has not apparently occurred over the sample, despite the many “local mean shifts” visible in figure 2. To make the exercise interesting, we go back to 1979 and the election of Mrs. Thatcher, and dynamically forecast Ur and d over the remainder of the sample as shown in figure 16 (top row) with ±2bσ error fans. REFERENCES 367 The forecast failure over the first few years in Ur is clear, associated with the failure to forecast the jump in d, following her major policy changes. Going forward two years and repeating the exercise (bottom row) now yields respectable forecasts. 9. Conclusion Computer-based teaching of econometrics enhances the students’ skills, so they can progress from binary events in a Bernoulli model with independent draws to model selection in non- stationary data in a year-long course which closely integrates theory and empirical modeling. Even in that short time, they can learn to build sensible empirical models of non-stationary data, aided by automatic modeling. We believe Clive would have approved. ACKNOWLEDGEMENTS The background research has been financed by the United King- dom Economic and Social Research Council through the funding of RES-062-23-0061 and RES-000-27-0179. We are indebted to Jennifer L. Castle, Jurgen A. Doornik, Vivien L. Hendry for many helpful comments on a previous draft, to Jennie and Jurgen for many invaluable con- tributions to the research and software, and in memory of Clive Granger’s long-term creative stimulus. References [1] A. Banerjee, J. J. Dolado, J. W. Galbraith, and D. F. Hendry. Co-integration, Error Cor- rection and the Econometric Analysis of Non-Stationary Data. Oxford University Press, Oxford, 1993. [2] G. C. Chow. Tests of equality between sets of coefficients in two linear regressions. Econometrica, 28:591–605, 1960. [3] M. P. Clements and D. F. Hendry. Forecasting Economic Time Series. Cambridge University Press, Cambridge, 1998. [4] J. A. Doornik. Autometrics. pages 88–121. 2009. [5] J. A. Doornik and H. Hansen. An omnibus test for univariate and multivariate normality. Oxford Bulletin of Economics and Statistics, 70:927–939, 2008. [6] J. A. Doornik and D. F. Hendry. OxMetrics: An Interface to Empirical Modelling. Timber- lake Consultants Press, London, 6th edition, 2009. [7] R. F. Engle. Autoregressive conditional heteroscedasticity, with estimates of the variance of United Kingdom inflation. Econometrica, 50:987–1007, 1982. [8] R. F. Engle, D. F. Hendry, and J-F. Richard. Exogeneity. Econometrica, 51, 1983. [9] N. R. Ericsson and J. G. MacKinnon. Distributions of error correction tests for cointegra- tion. Econometrics Journal, 5:285–318, 2002. REFERENCES 368 [10] L. G. Godfrey. Testing for higher order serial correlation in regression equations when the regressors include lagged dependent variables. Econometrica, 46:1303–1313, 1978. [11] K. Graddy. Testing for imperfect competition at the Fulton Fish Market. RAND Journal of Economics, 26:75–92, 1995. [12] C. W. J. Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37:424–438, 1969. [13] C. W. J. Granger. Some properties of time series data and their use in econometric model specification. Journal of Econometrics, 16:121–130, 1981. [14] C. W. J. Granger and D. F. Hendry. A dialogue concerning a new instrument for econo- metric modeling. Econometric Theory, 21:278–297, 2005. [15] C. W. J. Granger and P. Newbold. Some comments on the evaluation of economic fore- casts. Applied Economics, 5:35–47, 1973. Reprinted in T.C. Mills (ed.), Economic Fore- casting. Edward Elgar, 1999. [16] C. W. J. Granger and P. Newbold. Spurious regressions in econometrics. Journal of Econometrics, 2:111–120, 1974. [17] C. W. J. Granger and P. Newbold. Forecasting Economic Time Series. Academic Press, New York, 1977. [18] D. F. Hendry. Using PC-GIVE in econometrics teaching. Oxford Bulletin of Economics and Statistics, 48:87–98, 1986. [19] D. F. Hendry. Modelling UK inflation, 1875–1991. Journal of Applied Econometrics, 16:255–275, 2001. [20] D. F. Hendry. The Nobel Memorial Prize for Clive W.J. Granger. Scandinavian Journal of Economics, 106:187–213, 2004. [21] D. F. Hendry and J. A. Doornik. The impact of computational tools on time-series econo- metrics. In T. Coppock, editor, Information Technology and Scholarship, pages 257–269. Oxford University Press, Oxford, 1999. [22] D. F. Hendry and J. A. Doornik. Empirical Econometric Modelling using PcGive: Volume I. Timberlake Consultants Press, London, 2009. [23] D. F. Hendry, S. Johansen, and C. Santos. Automatic selection of indicators in a fully saturated regression. Computational Statistics, 33:317–335, 2008. Erratum, 337–339. [24] D. F. Hendry and H-M. Krolzig. The properties of automatic Gets modelling. Economic Journal, 115:C32–C61, 2005. [25] D. F. Hendry and M. Massmann. Co-breaking: Recent advances and a synopsis of the literature. Journal of Business and Economic Statistics, 25:33–51, 2007. REFERENCES 369 [26] D. F. Hendry and G. E. Mizon. The pervasiveness of Granger causality in econometrics. In R. F. Engle and H. White, editors, Cointegration, Causality and Forecasting, pages 102–134. Oxford University Press, Oxford, 1999. [27] D. F. Hendry and B. Nielsen. Econometric Modeling: A Likelihood Approach. Princeton University Press, Princeton, 2007. [28] D. F. Hendry and M. H. Pesaran. Obituary in memory of Clive Granger: An Advisory Board Member of the Journal. Journal of Applied Econometrics, 24:871–873, 2009. [29] D. F. Hendry and P. C. B. Phillips. Obituary: Sir Clive W.J. Granger. Econometric Theory, 25:1139–1142, 2009. [30] D. F. Hendry and C. Santos. An automatic test of super exogeneity. In M. W. Watson, T. Bollerslev, and J. Russell, editors, Volatility and Time Series Econometrics, pages 164– 193. Oxford University Press, Oxford, 2010. [31] S. Johansen and B. Nielsen. An analysis of the indicator saturation estimator as a robust regression estimator. pages 1–36. 2009. [32] B. Nielsen. Order determination in general vector autoregressions. In H.-C. Ho, C.-K. Ing, and T. L. Lai, editors, Time Series and Related Topics: In Memory of Ching-Zong Wei, pages 93–112. IMS Lecture Notes and Monograph Series, 52, New York, 2006. [33] A. W. H. Phillips. The relation between unemployment and the rate of change of money wage rates in the United Kingdom, 1861–1957. Economica, 25:283–299, 1958. Reprinted as pp243–260 in R. Leeson (ed.) (2000) A. W. H. Phillips: Collected Works in Contemporary Perspective. Cambridge: Cambridge University Press. [34] P. C. B. Phillips. The ET interview: Professor Clive Granger. Econometric Theory, 13:253– 303, 1997. [35] J. B. Ramsey. Tests for specification errors in classical linear least squares regression analysis. Journal of the Royal Statistical Society B, 31:350–371, 1969. [36] D. S. Salkever. The use of dummy variables to compute predictions, prediction errors and confidence intervals. Journal of Econometrics, 4:393–397, 1976. [37] H. White. A heteroskedastic-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48:817–838, 1980. [38] G. U. Yule. Why do we sometimes get nonsense-correlations between time-series? A study in sampling and the nature of time series (with discussion). Journal of the Royal Statistical Society, 89:1–64, 1926. Reprinted in Hendry, D. F. and Morgan, M. S. (1995), The Foundations of Econometric Analysis. Cambridge: Cambridge University Press.