lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) PEACE RESEARCH AND INFORMATION SYSTEMS Carl Beck University of Pittsburgh In a recent article I wrote that cal in a way that will be attended "the need for information retrieval to by academic and policy col- systems in the social sciences is leagues they must be able to docu- both real and apparent, but given ment their position effectively, the ability of many researchers to Their ability to be effective is gather idiosyncratic research sup- going to depend upon their ability port, the need is not perceived as to manipulate a basically uncont- acute. The evidence that is not reliable information environment, perceived as acute is that few Effective access to information is scholars are willing to amend their required if theory buffeted about usual behaviors to participate in by information is to be the basis the building of an effective infor- for conclusions reached as a result mation base which could be shared of peace research, by social scientists. With the seeming inability of anyone to stem Unfortunately, we live in an the information explosion and with unstructured information environ- few positive steps having been ment particularly in the social taken recently to improve access, sciences. It seems beyond anyone's the information environment in control, even UNESCO's. We are which we operate has continued to dependent upon information struc- deteriorate. Unfortunately, that tore to organize and facilitate deterioration is still not per- access to that information environ- ceived by most researchers as ment. Yet every existing informa- acute, and the literature is still tion structure provides only par- concerned more with technology and tial access for a number of reasons conceptualization than on results. that I will discuss later. There have been some important contribu- The impact of a less than a-ie- tions; the work of Alan Newcomb; quate information environment is the work of the International Poli- particularly felt by those engaged tical Science Association and the in a multi-disciplinary study such work of my own University Center as peace research. In peace for International Studies in the research there are no established development of the United States authorities and no established and Political Science Information Ser- formalized schools of thought, vice are examples. If this report therefore, the peace researcher stresses the characteristics of must engage in a widely ranging United States Political Science information search behavior in Information Service it is only order to touch all of the relevant because I am better acquainted with information bases. Peace research- the ins and outs of that service, ers are also challenged by being in But, I believe, that even those who a field which is policy related. have participated in these activi- Peace research is by its very ties would recognize the partial nature bound to intrude upon nature of their contribution. As national sensitivities, ideological more data are collected and dis- sensitiv ities , and/or academic sen- cussed the information environment sitivities. If peace researchers are to go beyond the merely polemi- - 3 - lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) expands and the ability of an the International Studies Associa- individual researcher to control tion, one of whose thematic see- that environment contracts. Par- tions is a Peace Studies Section; tial access to information has many and those associated with my role debilitating consequences. It as Director of the University Cen- leads to waste of research ter for International Studies, a resources; it strengthens academic multi-disciplinary center concerned imperialism and orthodoxy; it rein- with internationalizing the teach- forces tendencies toward the reifi- ing, research and service facili- cation of authorities; it even ties and programs at the University reinforces global divisions such as of Pittsburgh. There exists in the North-South division. It also these cases a symbiotic relation- makes it very difficult for new ship between information systems fields of inquirty to develop, or and substantive research. We have to sustain themselves. I believe found that the more effort we put that the problem of information into developing interactive infor- access is endemic in the social mation systems the more effort we sciences and particularly acute for will able to devote to the analyti- persons engaged in peace research. cal characteristics of research rathern than the bibliographic. In The remarks in this paper are doing so we learned a great dela focused on problems and possibili- about the conceptual, technical, ties for the integration of docu- and organizational problems con- mentation and information relevant fronting the development of an for peace and conflict studies into information system and the dissemi- social sciences information systems nation of information from one such and for the use of such systems by center to another, peace researchers. There is an additional emphasis upon the deliv- The frustrations stem from a ery of such information to indivi- decade of committee work in such duals in developing nations, alt- ill fated activities as the Infor- hough I tend to believe that the mation Retrieval Committee of the problem confronting scholars in any Council of Social Science Data field that is multi-disciplinary. Archives, and other international social scientific and policy and national committee's whose related is probably no greater in efforts here produced many reports, developing nations than the same workshop statements, guidelines, problem is in most developed but whose contributions to anything nations -- in fact, it may even be operative that improves structured easier at least to initiate fields access to information has been very of inquiry in those nations in that limited. These experiences have existing educational institutional reinforced a very strong prejudice structures are less formalized. which you will see running through- out this paper: to concentrate Much of what I have suggested only on THE Thesaurus, THE informa- stems from three positive experi- tion program, THE information ences and from one very strong retrieval system, and to believe at frustration stemming from other the same time that this constitutes experiences. The positive sets of a direct and positive contribution experience are those associated to improving information access is with the development of the United to engage in an illusionn. The States Political Science Informa- illusion is satisfying and, indeed, tion System, those associated with the development of improved concep- my role as Executive Director of lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) tualizations and models is important and will shape the future, but we can't risk the pre- sent for the future. Since this type of prejudice has, in the past, been described often as American pragmatism I might as well plead guilty to it. However, just for the record it should no longer be considered as American pragmaticism as American research support agen- cies such as the National Science Foundation, the Office of Educa- tion, the Ford Foundation, etc., have made it abundantly clear that they believe the opeaational dimen- sions of system design and develop- ment to be beneath them. Someone has been very successful in pre- suading them that the "American" concern with getting a system run- ning is not as respectable as the European concern with informatics. I believe we need both; we can deliver the something that works and that meets information needs with very little effort and very little funding. Indeed, I think one could easily document that any investment in information systems would soon be recouped. The often discussed formula is that 25% of a research budget goes into biblio- graphic searching. My guess is that 25% is too low (Committee on Scientific Technical Information, 1970, 1976a, 1976b) In a discussion of how to get relevant information into informa- tion systems appropriate for peace research we have to begin with some truisms. I will not dwell on the truisms of what constitutes an effective information system. We all know that an information system to be useful and to be used must meet the needs of its users. In the conceptualization and design of an information service the nature of the inforamtion to be made available and the nature of the information behaviors of the users to be served must be analyzed or if that is not possible at least inferred from known information structures and known behaviors with space left in the conceptualization and design for unknowns. There- fore, we have to start with the question of what constitutes peach research. David Singer suggests there are three major streams to peace research: (1) the pure sci- ence school who justify their research on grounds of intellectual curiosity; (2) the applied science school that believe that their research will end human suffering; and (3) the radical critique school. Singer suggests that three major substantive issues shape the field: strategic deterrence, social reform, and economic devel- opment (Singer, 1976). Wh a t a r cepts that we approac ing a na research a terms and is used to Political national Abstracts Abstracts , the major e some o mark pe h this q rrow p nd then concepts search Sc ience Pol it and we get terms . f the ace re uest io rof i le see wh as t the Un Do c urn e ical Ps a view major con- search? If n by build- of peace at types of hat profile ited States nts, Inter- Science ycholog ical of some of Agreements, including such items as alliances, contracts, blocs, fronts, state agreements, private agreements, international organi zations; Conflict, including such items as aggression, agi- tation, economic con- flict, ideological con- flict, social conflict, war ; Crisis management, including such items as conflict management. lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) diplomacy, strategy, bargaining, peace keep- ing, arms control; 4. Weaponary, including such items as arms, missies, military characteristics; 5. Structures, including structural prerequisites for world order, peace keeping forces, interna- tional law; 6. Peace science, including the search for behavioral and structural predictors of peace, correlates of war, events and even ana- lysis; 7. Psychologyical condi- tions, ranging from aggression to conscious- ness (International Peace Research Newsletter, 7(3)); and 8. Special methodologies, such as fights, games, simulations . And of course with all of these the array of methodologies, con- cepts and analytical modes that can be used to discuss them. Peace r will want very wide b tainly enc social scie ogy as wel some peace concepts s extracts f clinical ps apy will pr Additionall be concern impacts and able to mon ized transa esearchers as to be able to ody of 1 iteratu ompassing all nces and social 1. If we add researchers con uch as consc rom the liter yc ho logy and ps obably also be y researchers ed with direc they will wa itor events and ctions as we 11. a group search a re, cer- of the psychol- to that cern with iousness , ature of yc bother- relevant . wi 11 al so t policy nt to be spec ial- The inform at i is characterized information n researchers is I believe the si ment constitute difficult probl peace research or in getting pe mation into ope systems so that will use such sy cussing sources this paper let traditional fas parameters of th peace research i on environment that by the aggregated eeds of peace therefore immense. ze of this environ- s the single most em in developing information systems ace research infor- rating information peace researchers stems. Before dis- as a major item of us discuss in more hion some of the e development of a nformation system. Terminological control. Information systems vary in the way in whch they search textual informa- tion from full text searching to some system of terminological control in which both indexers and users are acquainted with a codebook of terms. Weak information systems rest upon a set of des- criptors; strong informa- tion systems have a structured thesaurus. With testing and refine- ment the Political Sci- ence Thesaurus will meet all of the terminological requirements of peace research information sys- tems in English. Such testing is now underway at the University Center for International Stu- dies. A revised thesau- rus, along with appropri- ate changes in the computer files, will be available sometime in 1980. The limitation is that this Thesaurus is only in English and our conceptu- lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) alizations of language and linguistic structure are not developed enough to allow for direct translation from a The- saurus in one language to another. However, even with the sloppiness that will be built into direct translation we ought to be experimenting with it by running a translated Thesaurus against exist- ing information data bases. The fact that most computer readable information services are in English, and even more limiting, drawn from English language sources, limits the possibility of building peace research from many different per- spectives and traditions (Sartori, e_t a_l . , 1975). Levels of Information. The design of an informa- tion system or the design of a format for machine readable descriptions of textual information always involves trade- offs. I believe that one area that we cannot allow to be weak and partial is in the number of levels of information that should be included. These include a document number, author, contribu- tors, title, source, abstract, tables, figures and charts, cited authors, subject descrip- tor (developed from the Thesaurus), geographic descriptors (developed from the Thesaurus) , and significant proper names. I wish that we had added one more, headings in the article, and I wish that we had included the title of the citation as well as the author. However, since we have been operating on no funds except the subsidy that the University of Pittsburgh is providing to cover our deficit, it is perhaps astonishing that we have come as far as we have. We made the strong levels of informa- tion decision in the belief that by so doing no one would ever have to go back to these documen- tary representations of scholarly articles to add information, no matter who the clients might be for the information ser- vice. I believe we were right. We were also right in not allowing authors to write their own abstracts. Our experience shows that authors tend to abstract the article they wish they had written rather than the article they wrote . Info Syst ter soci of h d isc info syst THE esta term at appr info info wo ul "str info syst rmat em . bee al r ours ussi rmat ems sy blis "st 1 ea oach rmat rmat d ong " rmat ems ion Si nee ame a esearch have ng and ion in the stem hed. rong" t St one to the ion pr ion sys use in r ion when t Retrieval the compu- tool of , millions been spent desig ning retr ieval hope that could be I used the o describe systems levels of oblem in tems. I the term egards to retr ieval hose sys- lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) terns are easily understood, easily mani- pulated and fully flexi- ble. Its internal nature should be left to those who have to mesh hardware and software require- ments. RECON is still an excellent model. So are DIALOG (Lockheed) , ORBIT (SDC) , STAIRS (BRS), WISE (a New York Consortium), TRIAL (Northwestern Univ- ersity) , AND PIRETS (University of Pitts- burgh) . The design of such systems and changing of the necessary programs is now a matter of rout- ine. The swapping of data and information bases is much more impor- tant a process in getting information to be used than is the swapping of programs. We have run USPSD tapes on DIALOG, on ORBIT, on TRIAL, and on PIRETS, and on homegrown programs for small IBM computers, large CDC com- puters, etc. We are always glad to discuss with any institute inter- ested in swapping infor- mation bases, mutual par- ticipation in the design and implementatin of a retrieval system for access to the swapped materials. Dissemination, What do we know about the format requirements of users of a computer-readable information system? Actually, we do not know very much. We feel that users do not like to read computer output. I know I do not. We feel that some scholars want tradi- tionally structured information and that other scholars want information sources that they can manipulate. A strong information system will then produce multi- ple products, A weak one will produce its product in only one or two for- mats. In USPSIS we pro- duce the following: an annual publication of the entire contents of the yearly file; a set of derivative publications in which certain docu- ments which clusters around a theme are pulled from the file and printed on an annual basis (Eth- nic Studies, Strategic Studies, Intercul tur al Studies, Russian and East European Studies, Asian Studies, and if we could expand our sources -- see below — peace research) ; individualized searches in which clients may write or call and discuss their information needs from which we will gener- ate a custom designed profile and run it against all of our hold- ings (currently about 600,000 entries); tape leasing in which an institute may lease the tapes receiving a new one each quarter; tape leas- ing to commercial systems which we do through both Lockheed and SDC, and soon may be even micro- fiche and microfilm. All of these outputs seem easy to generate. What is not easy is to know which will please cl ients . lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) 5. Sources. Here is the weakness of all existing answers to the question of how can we help peace researchers meet their information needs. Even the literature of the United States, which is the best covered, is badly covered. We can demonstrate this by ask- ing what sets of sources are potentially of inter- est of peace researchers and how much of a parti- cular set can they access on a sustained basis. Please note that I have raised the question, not in terms of articles in specific journals or spe- cific occasional paper series because I believe it is wasteful and inef- ficient in terms of financing and in terms of meeting the needs of the diverse clientele which constitute peace research to select journals for inclusion in a peace research system. The computer should do the selection: one person's peace and conflict stu- dies literature may not be another's. In the interest of helping researchers we should be able to tell them what materials are covered by an information system. If article are the level of inclusion in a system the client will never really know whether or not a journal has been fully covered by his or her search. When a deci- sion is made to include a journal in a system all articles in that journal should be included so that a user of the system does not have about full cov so that late abstractor/inde not have to go fill in hole data/informatio can be more ea ped than created . It noted that I a low level conce swapping rathe higher level co as networking believe that s possible now networking requ o rgan i za t ional nological devel be ef f ic ient . require much ping , as I d i in the conclusi paper , can be d a fret erage and r another xer does back and s . Al so n files sily swap- networks should be m using a pt such as r than a ncept such because I wapping is and that ires both and tech- opment to It doesn' t but swap- scussed it on of this one today. a) Journals: It is a safe statement that through Sociological Abstracts, Psychologi- cal Abstracts, United States Political Sci- ence Documents, the journal literature published in the U.S. relevant to peace research is handled and handled well. Clients who wish to search this literature can do so in published format, machine mani- pulable format, and in some cases tapes can be purchased or exchanged. Through the work of Alan New- comb and International Political Science Abstracts some of the international litera- ture can also be searched. The English abstracts of lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) International Politi- cal Science Abstracts can be searched by computer through the United States Politi- cal Science Informa- tion Service. The most effective organi- zational arrangement for improving the cov- erage of journal lit- erature is by the development of national or regional equivalents to the work of any of the above. Certainly, this is the stated goal of at least one of the above informa- tion services and dis- cussions are now under way with a number of institutions on how to bring this objective to fruition. b) Books. It is a fair statement that no existing service ade- quately handles access to books for peace researchers. In the future when the Jour- nal of Economic Liter- ature goes into a com- puter based information system some progress will have been made. A major issue with this is how to describe the contents of a book adequately to help the client determine whether a study is relevant to the client's needs. Those of us who believe in strong information services argue that books should be ana- lyzed on a chapter basis in as structured a format as journal articles. Unfortu- nately, at this moment no one provides the resources even to do the experimentation necessary to determine whether the cost is worth the effort. Experiments on how to describe books for the purpose of inclusion in machine readable information systems are critically impor- tant. Perhaps the Journal of Economic Li teratur-i' s approach of a critical descrip- tive review is ade- quate. One service will be exploring soon the possibility of some subvention from book publishers for inclusion of their products in the ser- vice. c) Convention papers. Given the expansive nature of peace research interests, the lack of definition of the field, and given the wide range of authorities -- par- ticularly of informa- tional authorities the life of informa- tion in peace research is very short. Much of the information exists in ephemera or in fugitive files. Three major examples of this type of infor- mation are convention papers, peace research conference proceed- ings, and occasional or working papers. ERIC is about the only - 10 - lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) d) system that handles convention papers and even there it does not do it particularly well. Most associa- tions no longer even list convention papers. The only one that I know that does so on a continuous basis and makes them available for purchase after the convention is the International Studies Association. Yet convention papers are not hard to organ- ize. Often they can be grouped around a panel which makes the collection of such papers relatively easy. A hopeful sign is the work being done at the Carleton School of International Affairs under the leadership of Jane Baumont. She is col- lecting the titles of convention papers, computerizing them and doing searches using Key World in Context (KWIC) procedures. It is at least a start. Conf ere ings . No. 3 tional Ne wslet confere to peac 1 isted . rather ever , interes many o f pa pe r s part o tion en we can mater ia nee proceed- In Vol XVII, of the Interna- Peace Research ter , f i f teen nces important e research are This seems typical . How- i t wi 1 1 be ting to see how the conference ever become f the informa- vironment which access. These Is, I bel ieve , are of particular interest to a peace researcher because of the nature of the field as discussed above . e) Occasional papers and working papers. At times some of us have argued that publica- tion is a symbolic act. By the time an article is published it may be known throughout the invisi- ble college network by those who have their antenna out regarding the work of the author. A media of information that is important to the peace researcher is working papers of institu- tions. Many insti- tutes have working paper series. John Fletcher has shown what can be done in regards to letting the world know about Eco- nomic Working Papers. This fall, Economic Working Papers will begin computerization and may be available through commercial information systems. Unfortunately, Eco- nomic Working Papers contains neither abstracts nor a struc- tured terminological control system but Mr . Fletcher is building a terminological control system inductively. His efforts deserve support. f) Research Directory including research in 11 lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) TECHNOLOGICAL FORECASTING AND SOCIAL SCIENCE INFORMATION SERVICES As ever greater demands are current issue of the lASSIST News- placed on various resources, letter while a second questionnaire government, industry, and education will appear in the Summer issue. A around the world are attempting to report of the findings will be pre- better understand future technolog- sented in the Fall issue, ical developments so that better and more coherent planning can take The Delphi Method is designed to place. As with other elements of elicit from a group of experts -- technological societies, those peo- in this case members of lASSIST — pie engaged in the delivery of informed opinions regarding the information services must have some future. It does this by first idea of the future in order to meet seeking from the experts sugges- demands placed upon them by the end tions about what developments user. One of the problems with the (problems, events, techniques, delivery of such services is that equipment) will develop. The time we meet both technological and con- frame you should consider is the ceptual problems. period from 1980 through 2000. The first questionnaire to be presented The technological problems are is, therefore, essentially a blank bound up in developments of compu- sheet of paper, ter and communicat ins devices, whiole the conceptual problems Using all the imagination and relate to the issue of what data information at your disposal we shall be saved and archived. Alt- would like you to make some sugges- hough the need for forecasting is tions about what events and/or evident, the techniques for pro- problems you believe will confront ceeding with such forecasts is not the delivery of information ser- at all apparent. One of the pri- vices over the next twenty years, mary methods used to forecast and From these open-ended responses assess developments in technology will be developed a set of items is the Delphi Method. Delphi is which will be presented in the sec- useful under conditions where hard ond questionnaire -- you will then trend-line data are not available be asked to rate each of the items or where typical trend models are in terms of the likelihood that an thought to lack descriptive power event will take place, an estimated for future developments. date by which the event will take place, and the desirability that In order to assess future needs the event take place. From the set and problems concerning the deliv- of responses to the second ques- ery of machine readable informa- tionnaire an analysis will be tion, with this issue we are begin- developed which will present the ning a Delphi Study of the future findings of the study, developments in information tech- nology and service. The first questionnaire appears in the INSERT - 1 lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) We are embarking on this study naire by Air Mail, in an effort to further involve the members of lASSIST in developing PLEASE NOTE goals and objectives for the future of information services and tech- In order to prepare the second nology. In this insert space is questionnaire we need as much time provided for your responses. When as possible. It is imperative that you have completed as much of the you complete and return your ques- questionnaire as possible, please tionnaire as quickly as possible, return it to: We will structure the second ques- tionnaire using those responses Thomas Wm. Madron received by May 15, 1979. Your Academic Computing and Research help and cooperation will be Services greatly appreciated. This is an Western Kentucky University opportunity to provide all of us Bowling Green, KY 42101 USA with some information which should be useful in planning at our home If you live outside the United inst i tut ionas as well as for lAS- States, please return the question- SIST. Thanks. WITH RESPECT TO EACH CATEGORY NOTED BELOW, PLEASE ANSWER THE FOLLOWING QUESTION: "What conditions will be facing those of us who deliver information services in the 1980s and 1990s?" Provide as many res- ponses as you believe desirable in the form of short, declarative sentences . PROBLEMS FOR INFORMATION SERVICES: 1. 4. USER DEMANDS ON INFORMATION SERVICES: INSERT lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) 1, 3. 4. NEW TECHNIQUES FOR DELIVERING INFORMATION SERVICES 1, 3. EQUIPMENT NEEDS AND OPPORTUNITIES: 1. INSERT - 3 lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) OTHER EVENTS OR CONDITIONS; 1. THANK YOU FOR YOUR PARTICIPATION INSERT - 4 lASSIST Newsletter, Tol. 3, No. 1 (Winter 1979) progress. Peace researchers need to know who is doing what in the field and what research is being undertaken. Directo- ries of research in progress and biograph- ical directories are important information bases for peace researchers. Newslet- ters such as Interna- tional Peace Research contains invaluable information on who is doing what and what institutes are produc- ing. But a more systematic and more easily updated system is required. g) Policy Documents. Aside from the New York Times Information Bank what is there? Yet peace researchers need to know about transactions, ublic affairs, pronounce- ments by individual political actors. A model for this type of policy related infor- mation system in SCOR- PIO at the U.S. Library of Congress, but SCORPIO is not public and until it is it is of little use to peace researchers. h) Specific Data Sets. In Vol. XVII. No. 3 of the International Peace Newsletter it is suggested that peace researchers should pay attention to such cri- tical matters as food production. Yet how do we find data that we can manipulate for analytic purposes on food production? What is needed is an infor- mation system on data files available for secondary analysis. The characteristics of such a system are dis- cussed by Paul E. Peters in SIGSOC Bul- letin, Vol 6, No. 2-3, in 1974, Experimental work on developing such files was done by the University Center for International Stu- dies as early as 1970. However, despite many international confer- ences on this subject, there is no service available. 6. Swapping and Networking. The information environ- ment of the peace researcher is immense and continuously growing probably at an expanding rate. The structure for accessing the environment in a continuous cont- rolled manner are inade- quate. What can be done to improve this situa- tion? There is always room for improvement in both the technology and terminology dimensions of an information service, but the major impediment is neither conceptual nor technical, it is organi- zational. We have had our sights set on THE system rather than on facilitating system interchange. If through inter-institutional agreements research institutes would sector out the information envi- ronment and take on res- - 12 - lASSIST Newsletter, Vol. 3, No. 1 (Winter 1979) ponsibility for building machine-readable files of a sector with agreed upon standards on delivery time, on levels of infor- mation, and on termino- logical control, and then on conditions for swap- ping, swapping of files could become routine rather than the unusual. USPSIS would be glad, as I am sure many research institutes or services would be glad, to swap files, UNESCO could be of great assistance by pro- viding seed money and organizational backing for such swaps. This many modal model would come closer to satisfying the needs of peace researchers than any one publication, or any one tape, or any one service. As the swaps became net- works an infra-structure for peace researchers would develop, and per- haps even a demonstration model for international collaboration could emerge . REFERENCES Committee on Scientific Technical Information (1970). "Director of Federally Supported Informa- tion Analysis Centers." COSA- TI-70-1. PB 189 300. Spring- field, VA: National Technical Information Services. -- {1976a) . "Progress of the United States Government in Scientific and Technical Commu- nications." PB 180 867. Springfield, VA: National Technical Information Services. -- (1976b), "Proceedings of the Forum on Federally Supported information Analysis Centers." November 7-8, 1976. Sponsored by COSATI Panel 6, PB 177 051. Springfield, VA : National Technical Information Services. Fred W. , Tower of Sartori, Giovanni, Riggs and Teune, Henry. Babel : On the Definition and Analysis of Concepts in the Social Sciences . International Stud ies Association, Pitts- burgh, 1975. Singer, J. David. "An Assessment of Peace Research." Interna - tional Security . 1(1), Summer 1976, 118-37. 13