06_Jasna Petkovic:tipska.qxd 49 Jasna Petković1, Milica Jovanović1, Maja Levi-Jakšić1, Sanja Marinković1, Marija Djordjević1 1 University of Belgrade, Faculty of Organizational Sciences, Serbia Ranking Technology Forecasting Journals by Using Data Envelopment Analysis UDC: 005.521:050.486 005.311.11 DOI: 10.7595/management.fon.2015.0018 1. Introduction There are many overlapping forms of forecasting technological developments, such as technology intelli- gence, forecasting, roadmapping, assessment, and foresight. (Madnick & Woon, 2003-2009). Many of these forecasting forms use similar tools, at the same time accomplishing similar goals. However, there is a gen- erally accepted tendency in government to use phrases that separate theory from practice, such as “as- sessment” and “foresight”, while economics uses phrases that link theory and practice, such as “roadmapping” and “competitive technological intelligence”. There are international differences as well, launched by different social expectations on behalf of the market and the government as well. Industrial roadmapping, a largely private sector led initiative, emerged and became prevalent in the United States of America, while foresight, a government sponsored activity, became preferred alternative in Europe. These forms of forecasting - national technology foresight, roadmapping, and competitive technological intelli- gence - gained weight at different times, but the authors have invested little effort to clarify their similarities and differences (Coates et al., 2001; Campbell, 1996). Technology forecasting usually focuses on specific technologies, but sometimes the scope is more en- compassing (Reuven & Dongchui, 1995). A company might roadmap a set of related technologies and prod- ucts; an industrial branch might roadmap the range of emerging technologies that could make potential impact on a given sector, or the state could roadmap technologies that cover its entire economic base. Management 2015/76 The goal of this paper is to present the ranking of the journals in the field of technology forecasting (TF) through the application of the Data Envelopment Analysis - DEA. Over the past four decades, we have witnessed a rapid development in the field of technology forecasting. This development, both in theory and in practice, has been fuelled by the increase in complexity, even greater competition, and rapid changes in the business environment. Back in the past, little attention was paid to the importance and conceptual development in the field of TF, which caused numerous overlaps of the forms of TF development and its influence. The changes that have resulted in the emergence of information technology and modern manufacturing technology have actually increased the need for application of technology forecasting, as well as for explosive growth in scientific and engineering liter- ature worldwide. The paper shows the ranking of 39 journals in the field of TF over the period from 1999 to 2011 through the application of the Data Envelopment Analysis, as well as through analyzing trend changes in pub- lishing professional publications in the field of technology forecasting. Also pointed out are the shortcomings of previous methods of ranking journals by using impact factors, as one of the most important indicators of the quality of journals, as well as the possibilities of applying indicators obtained by using other methodologies. The results of such method of ranking should point out to publishing trends to the researchers engaged in this field, and direct them to journals in which they could present the results of their research. The aim of this research is to show that more than one criterion must be taken in order to create the rank of a journal from a specific area. In this way we get a more realistic ranking of leading journals from the observed area, since other necessary el- ements that contribute to the importance of a journal for a relevant field are also taken into consideration. Keywords: Technology forecasting, Data Envelopment Analysis, Ranking, Journal Methodologically, both national foresight studies and roadmapping usually bring together people of differ- ent expertise and interests, and use instruments and procedures that allow participants to simultaneously adopt a micro view on their own disciplines, as well as a more comprehensive view on most important or shared objectives (Zhu & Porter, 2002; Gordon & Glenn, 2003). It has been noticed through literature review that the trends and directions of the development of scientific areas are determined by analyzing tendencies in publishing expert results (Togia & Tsigilis, 2006; Leydes- dorff, 2008; Rafols et al., 2012). This way of analyzing the actual condition of a certain field is also used by scientists in the field of management of technology and innovations (Linton & Thongpapanl, 2004; Linton, 2009; Thongpapanl, 2012; Ratinho et al., 2015), so this paper also shows that the authors have made an ef- fort to examine current trends in publishing research in this field. We have also presented the possibility of ranking journals with the help of alternative ranking methods - the Data Envelopment Analysis, which has proven to be convenient for this type of ranking. (Tüselmann et al., 2015). 2. Trend in TP Publications In 2003, the Technology Futures Analysis Methods Working Group (TFAMWG) sought to lay out a framework from which to advance the processes and the methods. They combined different forms of technology fore- casting studies under the term Future/oriented Technology Analyses (FTA) and classified different forms as follows (TFAMWG, 2004): • Gathering and interpreting information: Technology monitoring, technology watch, technology alerts. • Converting that information into actionable intelligence: Technical intelligence and competitive in- telligence. • Anticipating the direction and pace of changes: Technology forecasting. • Relating anticipated advances in technologies and products to generate plans: Technology roadmapping. • Anticipating the unintended, indirect, and delayed effects of technological changes: Technology assessment, and forms of impact assessment, including strategic environmental as- sessment. • Effecting development strategy, often involving participatory mechanisms: Technology foresight, also national and regional foresight. How much the FTA research publication is out there? Figure 1 shows the results of querying Web of Science for “Technological forecasting” or “Technology forecasting”. The activity seems encouraging for the FTA. Figure 1. Number of hit counts for “Technology forecasting” OR “Technological forecasting ” from Web of Science between 1996. and 2009. (Firat, 2010) 50 2015/76Management 51 Management 2015/76 Type # of Articles # of Authorships % of Articles Academic 567 779 58% Gov′t/NGO′s/Institutes 174 210 18% Industry 109 142 11% Other 128 - 13% In 2006, Alan Porter (Zhu & Porter, 2002) prepared a literature profile of the FTA domain helping to characterize the growing body of knowledge. This study shows that the number of scholarly articles relat- ing to FTA is increasing. The study also examines the sectoral mix of institutions involved in the FTA as shown in Table 1 below. Note that the second grouping consolidates several difficult to distinguish types - governmental and non-governmental organizations, and other such institutes. Publication of the FTA articles is strongly led by the academic community - which has the greatest stake in such publications - but the sub- stantial participation by government and industry is also notable. Table 1. Leading Authoring Organizations by Sector (Firat, 2010, p. 24) Where is FTA work being published? Alan Porter’s study (Zhu & Porter, 2002) for the period from 1996 to 20061 lists 11 journals with 10 or more publications, where “Technological Forecasting & Social Change” ranks first with strong representation of leading technology management journals. Table 2 displays the most impotant journals in the field of technology management. The “Journal of Cleaner Production” focuses on sustainable development, while “Solid State Technology” shows a number of technology roadmapping ar- ticles. (Porter & Van der Duin, 2007, pp. 183-207). Table 2. Leading FTA Journals (1996-2006) Table 3 represents (sorted) areas of articles in which we recognized 20 out of 36 key words that show 15 times, or more. 1 Search results downloaded from the “Web of Science” on October 16, 2006. Process repeated on October 30 to capture cited references as well. This seems like a reasonable information resource to use in that it provides excellent coverage of journal articles in the sciences (SCI), de- cent coverage of engineering, good coverage of social sciences (SSCI), and additional treatment of humanities (AHCI). It does not include conference papers and books. Leading FTA journals (# of Articles) Technological Forecasting & Social Change (114) International Journal of Technology Management (52) Futures (49) Research-Technology Management (26) Abstracts of Papers, American Chemical Society (14) Technovation (13) Journal of Cleaner Production (12) Journal of Forecasting (12) R & D Management (11) Solid State Technology (11) Technology Analysis & Strategic Management (11) 52 2015/76Management Table 3. Key Terms in the FTA Articles (Porter & Van der Duin, 2007a, p. 190) Based on the results of search of more than 25 words in different combinations (browsing did not show ad- equate results, for example, “future studies” or “scenario” and ”trend analysis” because those terms are too general), the presumption is that the relevant articles do not contain general FTA terms (for example, tech- nology foresight or forecasting). For that reason, Delphi, TRIZ, backcasting and cross-impact are browsed explicitly. Unfortunately, the two predominant phrases - “Delphi” and “Technology assessment” - proved highly problematic. Delphi captured certain physics research as well as articles mentioning the company of that name. Technology assessment mainly yielded “health technology assessment”, its own arena of detailed evaluations of medical technologies and programs. (Porter & Van der Duin, 2007a, p. 184). To deal with these issues, all results were combined in VantagePoint2. Duplicates were removed. An itera- tive process was then used: • Removing inessential articles • Base revision • Repeated check Different entries were browsed (titles, key words, journals, files) so as to avoid irrelevant articles. The set of separated records has then been browsed in order to detect the presence of the FTA terms. The volume of admissibility has included the approach of decision support. Although this approach is not per- fect, it has allowed for keeping a certain number of materials related with Delphi and Technology assessment, which would be discarded otherwise. 3. Empirical results and discussion In this chapter, the ranking of journals in the field of technology forecasting was carried out by applying the Data Envelopment Analysis - DEA. In previous research, DEA has proven very adequate when it comes to multi-criteria ranking, and it has also been used as a tool for journal ranking (Tüselmann et al., 2015). This method is highly suitable because, while creating the end score, one can use criteria that employ different measure units. In addition, one more advantage of the DEA method for this way of ranking is to be found in the fact that weight coefficients are not previously determined (fixed) values, but only the perimeter is given 2 Software for Market analysis and forecasting Key FTA terms # Technology Assessment 92 Technology 48 Innovation 43 Management 36 Future 35 Science 33 Delphi 32 Forecasting 28 Delphi method 26 Delphi study 25 Technology foresight 25 Model 22 TRIZ 21 Foresight 19 Sustainability 19 Systems 19 Models 17 Sustainable Development 17 Backcasting 16 Technology Forecasting 15 within which weight coefficients should range. In this way, every observed unit (in this case, a journal) is al- lowed to choose on its own a combination of criteria that is most suitable for gaining as high a rank as pos- sible (Tüselmann et al., 2015). The DEA models generally have two forms: input or output oriented model. Input orientation has a goal of minimizing the inputs necessary for creating the specified quantity of outputs. On the other hand, the out- put orientation tries to maximize outputs at the specified level of input values (Popovic, 2006a). This paper uses a dual, output oriented DEA model, or the so called “envelopment problem”. Dual model is used to construct a hypothetic composite unit beyond the existing units, all this for a given unit. In case this is possible, the observed unit is inefficient, and if not, than it is efficient. Although there are no inputs that could be used while ranking, the DEA model has been applied because of the listed feature saying that it can use diverse criteria, which is the case in the ranking problem that is the subject of this paper. In the output oriented DEA model, the values for dual weights show the importance that every DMU had while defining the input and output of the composite unit, and they are being determined so as none of the inputs of compos- ite unit ( ) is higher than the value of that input for kth DMU. By using dual weights chosen in this way, a necessary quantity ( ) that kth DMU should generate in order to be effective is calculated for each output (Popovic, 2006b). If the observed kth DMU generates a small amount of outputs, then the intensity factor λj shows how much it should increase its outputs proportionally so it should become efficient. When in optimal solution only λkj has a positive value out of all λj ( j= 1, 2, ...,n), then kth DMU is on the very edge of efficiency and it is not possible to construct a composite unit out of remaining DMU, that could generate greater amount of outputs with the same level of inputs as kth DMU (Savic, 2012). Dual model has a following form: (1) p.o. (2) (3) (4) However, such model cannot be used for ranking, because more effective units are not comparable having in mind that their efficiency scores are mutually equal, i.e., all have value 1. This is the reason why we will use a modification of the basic DEA model, the super-efficiency grade that will allow for the ranking of ob- served units, and it was proposed by Andersen and Petersen (Andersen & Petersen, 1993). The only mod- ification here is leaving out the unit for which the efficiency is measured in the limitation from (1 - 4) relation, i.e., the limitation in the primary model has a following form: (5) While dual model now has the following form: (6) p.o. (7) (8) (9) (j=1,..,n) represents a weighting coefficient allocated to DMUj while assessing the efficiency of the ob- served DMUk, and speaks of its importance while determining goals if the observed unit is found inefficient. This model differs from basic DEA models, because the unit that is being assessed (DMUk) is excluded from the production set that forms the efficiency frontier (Savic, 2012). 53 Management 2015/76 54 2015/76Management The basic questions that the authors wanted to obtain answers to by ranking journals in this way are: Which are currently the leading journals in the field of technology forecasting? What are the trends like in publish- ing publications in this field? Has anything been changed since Porter’s (Zhu & Porter, 2002) study? What are the key problems that the researchers in this field are dealing with? Impact factor is one of indicators of the quality of a journal, but it is not sufficient and it is not the only indicator that should be used for this pur- pose (Woodside, 2009; Hirsch, 2005; Harzing & Van der Wal, 2009). Based on Alan Porter’s study (Porter & Van der Duin, 2007a, pp. 183-207), a total of 25 journals have been ranked in this chapter, with an addition of 14 new journals that are considered to be leading in this area. These 14 journals have been selected by browsing the JournalCitationReports3 online base, and we have selected the journals that deal with this topic with the highest impact factor. The research has been con- ducted for the period from 1999 to 2011. The journals have been ranked based on four criteria: Total number of published articles, Total number of published articles in the FTA field, Impact factor and Number of quotations per article. The total number of pub- lished articles has been taken as basic output result of publishing one journal, but it has the least importance in this research, since it does not say much of the quality of publishing results in the FTA field. For that rea- son, we have taken into consideration the total number of articles in the FTA field that was obtained based on browsing key words. As one of the basic indicators for journal ranking (Franceschet, 2010), a biannual im- pact factor has also been taken into consideration. However, the information obtained from literature shows that the impact factor is not a sufficient indicator for the comparison of journals (Woodside, 2009; Togia & Tsig- ilis, 2006), so one needs to consider the quotation of papers as well. While creating the impact factor, one must relate the overall number of published papers with the number of quotations, so it may seem there is a data redundancy. However, the biannual impact factor does not consider the overall period of publishing that the authors wanted to take into consideration, so they tried to overcome the potential problem which refers to “nearsightedness” of a biannual impact factor by involving the abovementioned criterion (Togia & Tsigilis, 2006). In addition, it must be stressed that the criterion Number of quotations per article refers to all publica- tions, and not only to those from the FTA field. In the end, the first two criteria are put into relation with the fre- quency of journal editing, i.e., with Annual number of published journals, so we can say that this indicator has not been used explicitly, but it is used for gaining relative values of the first two criteria. While generating a DEA model, it is necessary to allocate the range of importance to each of the used cri- teria, which has been done in this paper through author’s subjective assessment. Total number of articles has been graded as least important, while Number of quotations per article is twice more important in com- parison with Total number of articles. Impact factor and Total number of articles in the field are listed as the most important factors and are twice more important than Number of quotations per article, i.e., four times more important than Total number of articles. These data have been collected from the ”Web of Science”4 site, which contains official information on all journals on the SCI and SSCI lists. As for data regarding the number of articles in the FTA field, on the above- mentioned site, a number of articles for 18 key words in this area have been searched for every journal, and they were identified by Porter (Porter & Van der Duin, 2007a, p. 190): • Delphi • technology assessment • (forecast OR forecasting OR forecasts) SAME (technology OR technologies)) • (“technology roadmap” or “technology roadmaps” or “technology roadmapping”) • technology foresight • roadmapping • Tech intelligence • (analysis SAME technologies SAME emerging) • futures research • TRIZ • Backcasting • cross-impact 3 Available on http://jcr.incites.thomsonreuters.com 4 Available on http://webofknowledge.com/ • (“foresight program” or “foresight programme”) • technology monitoring • technology watch • national foresight • technology SAME prospecting • Tech mining In order to perform journal ranking, based on defined criteria and collected data, the Data Envelopment Analysis has been used. In order to solve DEA models (6-9), the EMS5 software was used. Solving the basic DEA model was made possible by including a virtual input in the analysis, and the value of this input equaled 1 for all journals. In this way, the consistency of data and results was not undermined, taking into consideration that the effi- ciency could be observed as a ratio of a weighted sum of the output and weighted sum of the input. Table 4 shows journals ranked in accordance with four stipulated criteria (where the first two criteria are put in re- lation with Annual number of editions in order to get their relative value), for the period from 1999 to 2011. Table 4. Ranked FTA journals for period from 1999 to 2011 55 Management 2015/76 5 EMS - Efficiency Measurement System, it is used for calculating the efficiency using DEA method, Available on http://www.holger-scheel.de/ems/ Journal title Tot. number of mag. Number of art. in the field Impact factor Number of quotations per document Annual number of editions Technological Forecasting & Social Change 1013 245 1.709 2.362 9 International Journal of Technology Management 1227 50 0.516 0.742 16 Futures 1084 78 1.287 1.500 10 Research-Technology Management 759 39 0.885 0.682 6 Abstracts of Papers of the American Chemical Society 38631 0 9.907 9.896 52 Technovation 1024 30 3.287 4.085 12 Journal of Cleaner Production 1846 29 2.727 3.115 18 Journal of Forecasting 509 16 0.930 1.159 8 R & D Management 465 23 2.507 3.929 5 Solid State Technology 2312 17 0.271 0.63 12 Technology Analysis & Strategic Management 505 31 0.701 0.868 5 Journal of Engineering and Technology Management 219 10 1.032 1.806 4 IEEE Transactions on Industrial Informatics 325 0 2.990 4.143 4 Journal of Product Innovation Management 560 4 2.109 2.661 6 Journal of Materials Processing Technology 8763 2 1.783 2.370 42 International Journal of Production Economics 2939 5 1.760 2.545 12 Journal of Quality Technology 434 0 1.564 1.873 4 Computers and Industrial Engineering 1783 3 1.589 2.632 8 Strategic Management Journal 921 0 3.783 4.566 13 Business Strategy and The Environment 426 0 1.960 2.320 6 Journal of Information Technology 353 0 2.321 2.140 4 International Journal of Forecasting 712 14 1.485 1.678 4 Manufacturing and Service Operations Management 372 1 1.475 1.975 6 Organizational Research Methods 342 0 3.257 3.268 4 Management Science 1706 1 1.733 2.174 12 While collecting data, it was noticed that one of the journals, “Abstracts of Papers of the American Chemi- cal Society”, has incomparably higher values according to almost all criteria, and therefore, it was marked inadequate for comparison with other journals, i.e., it was left out from further comparison. The results of the application of the DEA method on the remaining 24 journals are shown in Table 5. It should be noted that the third column, “Efficiency score” gave a final score that was obtained using the DEA method, and jour- nal ranking was performed based on it. Because of the output oriented model, journals that have a lower score have accomplished a higher rank. In addition, the remaining four columns represent the values of weighted coefficients (ponders) which were allocated to each of the used criteria, and indicate how much each of those criteria contributed to the creation of the efficiency score. In this way, higher values of ponder were given to those criteria that allow the observed unit to accomplish a higher rank based on set limitations. Table 5. Application DEA method on FTA journals The obtained results show that the best journal in the FTA field is ”Technological Forecasting & Social Change”. This journal was ranked first because it publishes a large number of papers from this area, so this criterion was given ponder 1, while other criteria are neglected. Also, in addition to this journal, other two that rank highest are ”International Journal of Production Economics”, which has equable good values under all criteria, and ”Strategic Management Journal”, which ensured its good ranking due to a high impact factor. Therefore, the journal that ranks highest is ”Technological Forecasting & Social Change”, such is the case in Alan Porter’s study (Porter & Van der Duin, 2007, pp. 183-207), while second-ranked and third-ranked jour- nals from Porters (Porter & Van der Duin, 2007) list, namely the “International Journal of Technology Man- agement” and the “Futures” are ranked lower according to the DEA method (on 24th and 15th position, respectively), because in spite of a large number of papers from the observed area, they have a significantly lower impact factor compared to other journals, as well as the number of quotations per article. Since the 56 2015/76Management DEA method comprised more information by performing ranking based on four criteria, it is only logical that the “International Journal of Technology Management” and the “Futures” are poorly ranked because of the lower values under other criteria. As can be observed, the DEA method is quite practical for application for these purposes, and the list of the leading journals in the FTA field could be easily updated on regular basis. The ranking list of journals based on application of the DEA method is as follows: Table 6. Leading FTA Journals (1999-2011) In addition, one could determine if there was a change in the ranking between journals ranked by Porter (Porter & Van der Duin, 2007), by using comparative analysis of the number of FTA articles for the period from 1996 to 2006, and from 1999 to 2011. 57 Management 2015/76 JOURNAL Efficiency score 1 Technological Forecasting & Social Change 0.287 2 Strategic Management Journal 0.869 3 Internat. Journal of Production Economics 0.929 4 Technovation 1.037 5 Computers and Industrial Engineering 1.073 6 Journal of Materials Processing Technology 1.108 7 R & D Management 1.117 8 IEEE Transactions on Industrial Informatics 1.124 9 Organizational Research Methods 1.139 10 Journal of Cleaner Production 1.173 11 International Journal of Forecasting 1.273 12 Management Science 1.379 13 Journal of Product Innovation Management 1.427 14 Journal of Information Technology 1.472 15 Futures 1.577 16 Journal of Quality Technology 1.663 17 Business Strategy and The Environment 1.765 18 Solid State Technology 1.720 19 Research-Technology Management 1.802 20 Manufacturing and Service Operations Management 2.052 21 Technology Analysis & Strategic Manag. 2.095 22 Journal of Engineering and Techn. Manag. 2.332 23 Journal of Forecasting 2.613 24 International Journal of Techn. Management 2.898 Table 7. Comparative display of ranked journals (1996-2006/1999-2011) It can be noted that significant changes in ranking have occurred only for journals ”Abstracts of Papers”, “American Chemical Society”, which did not publish any article in the FTA field in the observed period, and “Technology Analysis & Strategic Management”, which improved its ranking by six positions. Also, the total number of articles published in the FTA field, in the period from 1999 to 2011, is higher in comparison with the previous period, which indicates the trend of growth in the number of articles in the field of technology forecasting. In addition, we pointed out a further direction of research that would include empirical research related with the development of education in the field of Technology management in Serbia and worldwide, as well as the need for certain knowledge that is obtained by practice. However, this research also has certain deficiencies when it comes to the selection of ranking criteria. As for the Number of quotations per document, it would be more adequate to update it for articles that refer to the field of technology forecasting. In addition, bearing in mind the shortcomings of the impact factor (Wood- side, 2009), it is necessary to make corrections as regards this indicator. Namely, there are other method- ologies that could help in determining journal ranking more objectively (Oosthuizen & Fenton, 2014). One of the solutions would be using SNIP (Source Normalized Impact per Paper), which takes into considera- tion the popularity of a certain field due to index normalization, thus solving the problem of “weight” of pub- lishing papers in certain areas. As for future research, it would be convenient to use SNIP for one of the criteria used for journal ranking, after which journal ranks should be compared to results displayed in this paper. 58 2015/76Management Leading FTA journals according to Porter's study Number of articles 1996-2006. Rank 1996- 2006. Number of articles 1999-2011. Rank 1999- 2011. Change in ranking Technological Forecasting & Social Change 114 1 245 1 0 International Journal of Technology Management 52 2 50 3 -1 Futures 49 3 78 2 1 Research-Technology Management 26 4 39 4 0 Abstracts of Papers, American Chemical Society 14 5 0 11 -6 Technovation 13 6 30 6 0 Journal of Cleaner Production 12 7 29 7 0 Journal of Forecasting 12 8 16 10 -2 R & D Management 11 9 23 8 1 Solid State Technology 11 10 17 9 1 Technology Analysis & Strategic Management 11 11 31 5 6 REFERENCES [1] Andersen, P., & Petersen, N. C. (1993). A Procedure for Ranking Efficient Units in Data Envelopment Analysis1993. Management Science, 39 (10), 1261-4. [2] Campbell, R. (1996). A methodological study of the utilization of experts in business forecasting, PhD Dissertation, UCLA. [3] Coates, V., Faroque, M., Klavins, R., Lapid, K., Linstone, H.A., Pistorius, C., & Porter, A.L. (2001). On the future of technological forecasting, Technol. Forecast. Soc. Change 67 (1),1 - 17. [4] Firat Ayse Kaya. (2010). Early Growth Technology Analysis: Case Studies in Solar Energy and Geothermal Energy, Working Paper CISL#02. [5] Franceschet, M. (2010). Journal Influence Factors. Journal of Informetrics 4(3), 239-248. [6] Gordon, T.J., & Glenn J.C. (Eds.). (July 2003). Futures research methodology, Version 2.0 Millennium Project of the American Council for the United Nations University.. [7] Harzing, A.-W., & Van der Wal, R. (2009). A Google Scholar H-Index For Journals: An Alternative Metric To Measure Journal Impact in Economics And Business. Journal of the American Society for Informa- tion Science and Technology, 60(1), 41–46. [8] Hirsch, J.E. (2005). An Index To Quantify An Individual’s Scientific Research Output. Proceedings of the National Academy of Sciences of the U.S.A. 102(46), 16569–16572. [9] Leydesdorff, L. (2008). Caveats for the Use of Citation Indicators in Research And Journal Evaluations. Journal of the American Society for Information Science and Technology. 59(2), 278-287. [10] Linton, J.D., Thongpapanl, N. (2004). Perspective: Ranking the Technology Innovation Management Journals. The Journal of Product Innovation Management. 21(2), 123-139. [11] Linton, J.D. (2009). Technology Innovation Management’s Growing Influence and Impact. Technova- tion. 29 (10), 643–644. [12] Linton, J.D. (2011). How Influential are Technology Innovation Management Journals—Technology In- novation Management Journal 2010 Impact Factors in Comparison With Financial Times 45. Techno- vation. 31(9), 425–426. [13] Madnick, S., & Woon, W.L. (2003-2009). Technology Forecasting Using Data Mining and Semantics, MIT/MIST Collaborative Research. [14] Oosthuizen, J.C., & Fenton, J.E. (2014). Alternatives to the Impact Factor. The Surgeon, 12(5), 239–243. [15] Savić, G. (2012). Komparativna analiza efikasnosti u finansijskom sektoru, doktorska disertacija, Fakultet organizacionih nauka, Beograd. [16] Porter, A.L. P., & P. in van der Duin. (2007). Knowing Tomorrow? How Science Deals with the Future, Eburon Academic Publishers, 183-201. 59 Management 2015/76 This paper studies journals that publish papers in the FTA (Future/oriented Technology Analysis) field. The research has been conducted for the period from 1999 to 2011, and it was based on Alan Porter”s study (Porter & Van der Duin, 2007). Apart from ranking according to number of papers in the FTA field, the ranking was also carried out based on several criteria, by using the DEA method. While the results of ranking according to number of papers in FTA field show similar ranks, with small oscillations compared to previous research (Porter & Van der Duin, 2007), the ranks obtained by the DEA method show sig- nificant deviations. These deviations emerged because of introducing new criteria in the analysis (total number of articles, im- pact factor, and number of quotations per article). Some of the leading journals from the previous analysis now ranked lower because their values were low for other criteria. This research aims to show that more than one criterion must be taken for the creation of journal ranking in a certain field (FTA, in this case). In this way, more realistic ranking of leading journals in the observed area is obtained, since other necessary factors that contribute to the importance of the journal for a relevant field are taken into consideration. Also, the analysis of journals that publish papers in the field of technology forecasting has been performed. The DEA method has been used in order to rank journals in this field. In the end, a possible development of the research is noticed through a selection of precise indicators for rank creation through the methodology used by the DEA. The results of such a way of ranking should indicate the publishing trends to researchers in this field, and to orient them towards journals in which they could present the results of their research. Also, the further research direction is pointed out, which would comprise another significant indicator for journal ranking, and that is the SNIP (Source Normalized Impact per Paper). Conslusion 60 2015/76Management About the Author [17] Popović, G. (2006a). Ocena efikasnosti kreditnih programa pomocu analize obavijanja podataka, mag- istarski rad. Beograd, FON. [18] Popovic, G., Martic, M., & Bulajic, M. (2006b). Primena DEA metode u rangiranju banaka Srbije. [19] Rafols, I., Leydesdorff, L., O’Harea, A., Nightingale, P., & Stirling, A. (2012). How journal rankings can suppress interdisciplinary research: a comparison between Innovation Studies and Business & Man- agement. Research Policy, 41(7), 1262–1282. [20] Ratinho, T., Harms, R., & Walsh, S. (2015). Structuring the Technology Entrepreneurship publication landscape: Making sense out of chaos. Technological Forecasting & Social Change. http://dx.doi.org/10.1016/j.techfore.2015.05.004 [21] Reger, G. ( 2001). Technology foresight in companies: from an indicator to a network and process per- spective.Technology Analysis & Strategic Management, 13(4), 533– 553. [22] Reuven, L. R., & Dongchui, H. (1995). Choosing a technological forecasting method. Industrial Man- agement, 37(1), 14. [23] Thongpapanl, N. (2012). The Changing Landscape of Technology and Innovation Management: An Updated Ranking of Journals in The Field. Technovation, 32(5), 257–271. [24] Togia, A., & Tsigilis, N. (2006). Impact Factor And Education Journals: A Critical Examination and Ana- lysis. International Journal of Educational Research. 45(6), 362-379. [25] Tüselmann, H., Sinkovics, R., & Pishchulov, G. (2015). Towards a consolidation of worldwide journal rankings - A classification using random forests and aggregate rating via data envelopment analysis, Omega 51, 11-23. [26] TFAMWG, (2004). Technology Futures Analysis Methods Working Group (TFAMWG). Technology futures analysis: toward integration of the field and new methods. Technological Forecasting & Social Change 71 (3), 287-303. [27] Woodside, A.G. (2009). Journal and Author Impact Metrics: An Editorial. Journal of Business Research. 62, 1-4. [28] Zhu, D., & Porter, A.L. (2002). Automated extraction and visualization of information for technological intelligence and forecasting, Technological Forecasting and Social Change 69, 495-506. Receieved: March 2015. Accepted: September 2015. Jasna Petković University of Belgrade, Faculty of Organizational Sciences, Serbia jasna.petkovic@fon.bg.ac.rs Jasna Petković is an assistant professor at the Faculty of Organizational Sciences, University of Belgrade, Serbia, where she earned her MSc and Phd degrees. She is an author and co-author of several books and more than 60 scientific papers. Her research and teaching interests are in the fields of technology management, technology forecasting and SMEs development. Milica Jovanović University of Belgrade, Faculty of Organizational Sciences, Serbia Milica Jovanović, MSc., works as a teaching assistant at the Faculty of Organizational Sciences, Department for Technology, Innovation and Development Management. She is completing her PhD studies at the Faculty of Organizational Sciences, University of Belgrade - Information systems and management. She has been involved in 3 consulting projects and 1 strategic project funded by the Ministry of Education, Science and Technological Development of the Republic of Serbia. Her main research interests include: technology and innovation management, quantitative management, sustainable development and performance management. Maja Levi Jakšić University of Belgrade, Faculty of Organizational Sciences, Serbia Maja Levi Jakšić, PhD is a full professor at the Faculty of Organizational Sciences, head of Department for Technology, Innovation and Development Management. She has been teaching as visiting professor at universities in Greece, United Kingdom, Japan, and France and is a dual diploma Master programme leader with Middlesex University from London. Prof Levi Jaksic has published more than 20 books and monographs, about eighty articles in journals and more than ninety papers in the proceedings of domestic and international scientific conferences. She has also been a project leader and member on about twenty domestic and international scientific projects. She is a member ofthe Editorial Board of Technovation. Sanja Marinković University of Belgrade, Faculty of Organizational Sciences, Serbia Sanja Marinković, PhD is an assistant professor at the Faculty of organizational sciences, Department for Technology, Innovation and Development Management. She is an author and co-author of several books and more than 60 scientific papers. She is a lecturer at dual award master programme International Business and Management, validated by Middlesex University, London. Her research and teaching interests are in the fields of technology and innovation management, sustainable development and SMEs development. Marija Djordjević University of Belgrade, Faculty of Organizational Sciences, Serbia Marija Djordjević is student of doctoral studies at the Faculty of Organizational Sciences, Belgrade University, Department of Management, and an assistant for the development of postal technologies in the PE „Post of Serbia“. She completed her graduate studies at the Faculty of Organizational Sciences, University of Belgrade. First of all, she graduated from the Department of Management and organization, on the Operations Management module. She earned her Master’s degree from the Department of Engineering and Operations Management, on the Engineering Management module. She is an author and co-author of seven scientific papers in journals and proceedings of domestic and international scientific conferences. Her research area of interest includes engineering process, performance measurement, and technology and inovation management. 61 Management 2015/76