HOM6_02_Persson_fifty_years_later   History  of  Meteorology  6  (2014)         7                   Fifty Years Later Anders Persson Swedish Meteorological and Hydrological Institute (SMHI) Stockholm, Sweden In 1974 I had the privilege of meeting and talking to Jacob (Jack) Bjerknes in Bergen, Norway, a city immortalized in the history of meteorology thanks to the "school" that Jack and his father founded in 1917. We were in Bergen for the 9th Nordic Meteorologist Meeting (NMM9), held every two years in one of the Scandinavian nations on a rotating basis. This was during a rare spell of dry and sunny weather, unusual for this part of rainy western Norway, and the hosts had invited two of their most legendary meteorologists: Bjerknes, who had come all the way from Los Angeles and Sverre Petterssen who was supposed to come from London. But these were the times of the Vietnam War and the Watergate scandal, and Sverre Petterssen had renounced his U.S. citizenship in an angry letter to the White House (Petterssen, 2001). Stateless and thus without any passport, he could therefore not leave the UK. Jack’s youthful memories Petterssen’s absence was to some extent compensated for by Erik Palmén from Finland who regularly attended the NMMs (fig. 2.1). His friendship with Jack dated back to the period between the world wars when, through their theoretical and practical works, they engaged in scholarly combat with their German and Austrian doubters and opponents (Friedman, 1989, p. 246; Bergeron, 1959, p. 462; Haurwitz and Haurwitz, 1939, p. 53 ff). I had hoped to learn more about this from Jack when we sat down in the park outside the Geophysical Institute. But recollections of his youth were freshest in Jack's memory. Born in Stockholm, with the first nine years of his childhood there, and then with almost half his life in the United States, he didn’t feel very "Norwegian": “Jeg føler meg mere svensk enn norsk,” he let slip. However, when the import of what he had said hit him, he corrected himself quickly: “men mest av alt nordisk!”1                                                                                                                 1 ”I feel more Swedish than Norwegian – but most of all Nordic!” 8   Persson,  Fifty  Years  Later           8   Fig. 2.1. Eric Palmén and Jack Bjerknes inspecting weather charts at NMM9 in 1974. Photo taken by Anders Persson. Jack’s lecture on climate change The NMMs featured numerous lectures on mixed meteorological topics. Jack Bjerknes chose to speak on his latest research. While all young career minded meteorologists used to fill the blackboard with mathematical equations, Jack just wrote A + B and drew a trapezoid. The trapezoid symbolized the Pacific and A + B the warm and cool surface water oscillations between the El Niño and La Niña phases, the more A, the less B etc. Now, 40 years later, it is easy to see that we had listened to a speech of historical significance in the meteorological-oceanographic sciences. At that time, within the broad meteorological community the “climate” was something you worked out in 30–year periods and which was determined by the variations in the Earth's path and axis tilt, perhaps also volcanic eruptions and human emissions after a nuclear war. With the El Niño (Southern Oscillation) attention was drawn to the atmospheric internal variability. In 1974 the oceanographic measurements were still rudimentary, to any extent with only surface water temperatures. Jack Bjerknes and his oceanographer colleagues at that time would have felt like children on Christmas Eve with a very generous Santa   History  of  Meteorology  6  (2014)         9   Claus if they had had access to today's networks of both surface and deep-based ocean measurements. The importance of satellites One day we visited the weather ship Polar Front II or “Mike” lying in Bergen harbour for overhaul. Its sister ship Polar Front I was on duty out in the Norwegian Sea. While expressing a high opinion of the weather satellite's future importance, Jack felt in 1964 that it would require a global network of weather ships with aerological equipment to reap the benefits of modern numerical forecasting. For the next two decades meteorologists would convert satellite radiance measurements into artificial radiosonde observations (SATEMs) which were then assimilated by the numerical forecast systems analysis in the same way as usual radiosonde observations . This was still the situation when I was employed at ECMWF in 1991. However, since the creation of these SATEMs involved interaction with another NWP system, it was seen as “incestuous,” and work had already started on the 4- dimensional variational analysis system (4DVAR) that would revolutionize the NWP and help to consolidate ECMWF as the leading NWP centre. 4DVAR The idea of variational analysis had originated with Soviet researchers but had been taken over and pursued by French meteorologists Olivier Talagrand, Phillippe Courtier, Florance Rabier, Jean-Noël Thepaut and others. The first innovation with 4DVAR was that the satellite radiance data could be assimilated much more smoothly if the model's "first guess" figured out what radiances the satellites "ought to" observe. Through sophisticated algorithms the differences were used to correct the analysed temperatures and humidities to make the emitted radiances as close as possible to the observed ones. The second innovation with 4DVAR was its dynamic consistency. In the same way as any forecast meteorologist who analyzes an 18 UTC map of the North Atlantic with the scattered ship observations is keen to look back on the earlier maps at 15 UTC, 12 UTC, and maybe 09 UTC to make a reassessment of the analyzes and observations in light of the later 18 UTC data, the mathematical algorithm in 4DVAR went back and forth over a 6-hour time window to make observations affect the analyses in a dynamically consistent way. It was intriguing to witness, how a single moisture observation thereby could change the wind and geopotential fields. These changes also prevented the humidity data from being wiped out because of physical-dynamic imbalances. Today's scientists trying to assimilate moisture information from the radar echoes face the same problem: without modification of the wind, the important moisture information is smoothed out in a few hours. The development of 4DVAR was very much a French affair and benefited from an almost endless stream of young female scientists from Météo France, including Florence Rabier, Mathilde Hervour, and Elisabeth Gérard, who had been recruited by Phillipe Courtier. I managed to break into this coterie when, once during one of our missions together, we discussed my section’s forecast error tracking using “group velocity thinking”. Back at ECMWF Courtier organized a project “to do with 8   Persson,  Fifty  Years  Later           10   mathematics what Anders is doing with his eyes.” In very simple words, it meant running the 4DVAR backwards in time (Rabier et al., 1993; Persson 1999, 2000). The 4DVAR technique has also found its use in climate research by enabling scientists to analyze the 3-dimensional structure of the atmosphere back into the 19th century, maybe into the 1780s, from only surface observations. The future of weather forecasters At the NMM9 in 1974 all this progress was not even dreamt of when we gathered one afternoon to discuss the future of human weather forecasters in the light of computer forecasts. Meteorologists were divided into two competing "camps": one claiming that in "5-10 years" there would be no need for any forecasters, another claiming that no computer could ever replace an experienced weather forecaster. In 1964, Jack Bjerknes had not seen any conflict between forecasters and NWP, and that was also the opinion both Sigbiørn Grønaas and I expressed in the 1974 debate. When I became more and more engaged at ECMWF in the 1980s, it turned out that there was a need for meteorologists who stood with one foot in both "camps." I was drawn into the information and education activities, with a focus on how best to use the ECMWF medium-term forecasts. I must admit that it was not totally clear what should be taught. Much of forecasting was intuitive and based on experience. If you wanted to learn to write poetry or cook Italian food, there was a plethora of books, but not on how to do weather forecasting using NWP. The “User guide to ECMWF forecast products” (2011), which I became in charge of, tried to remedy this want. I was not quite satisfied with it at the time, but there was nothing better. The limits of predictability In 1974 what would become ECMWF was taking shape.2 At the NMM9 we learned that Norway had decided to stay cautious, among other reasons because they doubted that 10-day forecasts were ever possible. At this time the useful predictability was 4-5 days, and there was great uncertainty about how much it could be extended. The question has been raised (Elsner and Honoré, 1992) why it took so long for Edward Lorenz 's classic predictability papers (Lorenz, 1963; 1969) to have influence? In 1972 I made a study of the predictability literature and found that in the 1960s he had been considered too pessimistic at a time when overly optimistic predictability studies pointed at 20+ days predictability. This was, however, shown by Döös (1969, p.53) to be due to misinterpreted verification statistics. But leading Scandinavian scientists still told me that there was an "energy gap" between synoptic and subsynoptic waves that would confine the forecast errors to the latter. On the other hand, studies had shown that the lack of observations over the North Pacific Ocean would set a limit of 5-6 days predictability over Europe. All those concerns lingered into the 1990s when they came to naught when the use of satellite data through 4DVAR made us almost independent of surface observations over the North Pacific and other oceans. The predictability of the global                                                                                                                 2 http://www.ecmwf.int/about/history/.   History  of  Meteorology  6  (2014)         11   forecasts is now considered to be 8-9 days and perhaps the “20 days optimists” from the 1960s will be right one day, albeit for the wrong reason. From this perspective, ECMWF and other NWP centres should be counted in the same high scientific class as CERN and the Hubble Space Telescope. While the former penetrates into the smallest components of matter and energy and the latter explores the extent of the universe, NWP raises the fundamental question of how far we can see into the future. Is "Laplace’s Demon" possible, an existence mastering all natural laws and knowing the initial conditions who can calculate everything that will happen or has happened? This challenge was a great inspiration to the father of NWP, Vilhelm Bjerknes and has remained so for many NWP modelers (see Bergeron, 1959, pp. 440-41) But from a down-to-earth practical point of view: the details of even the most high-resolution weather forecasts are inaccurate within a day or so. The “jumpiness” of NWP is an additional source of annoyance, since the next run may come up with a completely different development and thereby appear to disavow the earlier forecast. In 1986 I attended a meeting at ECMWF where a group of Dutch meteorologists emphatically pointed out that any forecast is pretty useless unless there is a way to tell how likely it is (Tennekes et al., 1986). Inspired by Americans visionaries (Chuck Leith, E S Epstein, Phil Thompson) this initiated research first in "forecast forecast skill" and then, when computer capacity allowed, in ensemble technique (Lewis, 2005). This was the second major meteorological revolution in the 1990s I had the privilege to take part in. An often-overlooked asset at ECMWF, at that time lacking in many other research institutions, was their excellent restaurant! It encouraged cross-fertilizing contacts across sectional and departmental boundaries. It gave young scientists opportunities to meet more established colleagues and visitors. You might one day find yourself discussing group velocity with Brian Hoskins in the lunch queue, next day follow Eugenia Kalnay and Tim Palmer arguing over perturbations techniques over the roast beef and at the end of the week, over a cup of coffee, listen to Ed Lorenz’s recollections when he was a weather forecaster in the Pacific during WWII. Ensemble forecasts While the work on 4DVAR tried to reduce the analysis errors, the work with ensemble system was almost the opposite. A large number (first 32 later 50) of alternative analyses, perturbed within analysis uncertainty, were created. If 10-day forecasts, run from these modified analyses, were in general in agreement with each other, we could be pretty sure of the prognosis irrespective of any faulty or misinterpreted observations upstream. In case they differed much, we could at least get an idea of what would not happen, and probability estimates for possible developments. This statistical approach to weather forecasting faced some difficulties in finding support among meteorologists in both "camps." In spite of their disagreements, they were united by their common classical deterministic Newtonian training. Statistics, "the science of uncertainty" was at best seen as a kind of perfume: it's good that it exists, but it is a pity that it needs to be used. To express uncertainty in any form was seen as worst as “cowardliness”, at best as a way to conveniently cover your back. The realisation that in a world with non-perfect forecasts the uncertainty information had a positive value, 8   Persson,  Fifty  Years  Later           12   increased the usefulness of the forecasts, still had a long way to go. This became the main theme in the 2011 edition of the “User Guide to ECMWF Forecast Products”, the one I am most satisfied with. Statistics in meteorology If the haunting spectre in 1964 for many in the meteorological community was the computer, it today seems to be the growing influence of statistical thinking that challenges the classic deterministic thinking. But statistical know-how is needed in our age of “Big Data” (Mayer-Schönberger and Cukier, 2013) The weather forecaster stands in a flow, or rather “overflow” of information—as do stockbrokers, politicians, military—and just like them acts as an “intuitive statistician” (Kahneman, 2011). They have to make quick decisions with conflicting indications from various models, the sudden appearance of new information from radar screens or oceanographic buoys. However, the assessment of uncertainty is complicated by a human tendency to be overconfident; to rather look for arguments that support an idea than those that contradict it. Underestimation of randomness may fool the forecasters to see systematic behaviors in the models where there are none. Then there is the problem of how to present forecast uncertainty in a form that is understood and - not least, gives a fair chance to make the right decision. With the increasing emphasis on extreme or "high impact" weather resistance against probability forecasts is slowly giving way. Even the most hardcore determinists realize they may have to shelve their planned mountaineering even if the probability of snow and storm gusts is less than 100%. In a New Year's reflection in 2005 the then SMHI's Director Maria Ågren held the view that also the climate debate needs to be more probabilistic: probabilities of severe climate deterioration need not be 100% or even 95% to justify action. I think Jack Bjerknes would be pleased to see that, despite prophecies to the contrary, the weather forecasters are still with us and perhaps there have never been so many around as today, not least in the private companies. Maybe they remain, not "despite" the computer, but "thanks” to the computer. Thanks to the high quality of the weather forecasts, people can now begin to take them seriously into account. But to do that most efficiently they need to know more about their uncertainties and possible extremes. Here lie the challenges for the future. References Bergeron, T., 1959: Methods in Scientific Weather Analysis and Forecasting. In An Outline in the History of Idea and Hints at a Program, in The Atmosphere and The Sea in Motion, Scientific Contributions to The Rossby Memorial Volume, Ed. B. Bolin, pp. 440-474. Stockholm,   Rockefeller   Institute   Press   and   Oxford   University  Press. Döös, B.R., 1971: The Status and Outlook of Numerical Weather Prediction, Zeitschrift f. Meteorologie 22:1-2, 46-53.   History  of  Meteorology  6  (2014)         13   ECMWF Seminar Proceedings 6-10 September 1999, pp. 123-38, http://www.ecmwf.int/publications/library/do/references/show?id=85676 Elsner, J.B. and J.C. Honoré, 1994: Ignoring Chaos, Bull. Am. Met. Soc. 75:10, 1846-47. Friedman, R.M., 1989: Appropriating the Weather, Vilhelm Bjerknes and the Construction of Modern Meteorology, Ithaca, NY, Cornell Univ. Press, 251 pp. Haurwitz B. and Haurwitz, E., 1939: Pressure and Temperature Variations in the Free Atmosphere over Boston, Harvard Meteorological Studies, 3, Cambridge, Mass., Harvard University Press; London, H. Milford, Oxford University Press, 74 pp. Kahneman, D., 2011: Thinking, fast and slow. Allen Lane, Penguin Books, 499 pp. Lewis, J., 2004: Roots of Ensemble Forecasting, Mon. Wea. Rev. 133:7, 1865-85. Lorenz, E., 1963: Deterministic nonperiodic flow, J. Atm. Sci. 20, 130-41. _____.. 1969: The predictability of a flow that posses many scales of motion, Tellus 21, 289-307. Mayer-Schönberger, V. and K. Cukier, 2013: Big Data, A Revolution That Will Transform How We Live, Work and Think, John Murray, 242 pp. Persson, A. 1999: Monitoring the ECMWF forecast system by dynamical methods, pp. 150-153. 13th conference on Numerical Weather Prediction, Denver, Colorado, 13-17 September 1999, Am. Met. Soc., Boston. _____. 2011: User Guide to ECMWF forecast products, available at http://www.ecmwf.int/products/forecasts/guide/user_guide.pdf Petterssen, S,. 2001: Weathering the Storm: Sverre Petterssen, the D-Day Forecast, and the Rise of Modern Meteorology, J.R. Fleming, ed. Am. Met. Soc. Boston, 329 pp. Rabier, F, P. Courtier, M. Hervou, B. Strauss and A. Persson, 1993: Sensitivity to forecast error to initial conditions using the adjoint model. ECMWF Tech. Memo 197, http://www.ecmwf.int/publications/library/do/references/show?id=83728 Tennekes, H, A.P. Baede and J.D.Opsteegh, 1986: Forecasting forecast skill, Workshop on Predictability in the Medium and Extended Range 17-19 March 1986, available at http://www.ecmwf.int/publications/library/do/references/list/16234 Anders Persson (b.1944) chose a career in meteorology in 1964, the year of Bjerknes’s essay, “Half a century of change in the meteorological scene,” in the mistaken belief that meteorology was a simpler science than quantum mechanics or relativity theory. Starting as a weather forecaster at SMHI and then as a senior scientist at ECMWF and the British Met Office, he has, at close range, followed the spectacular development of numerical weather prediction (NWP), one of science’s great success stories.