5 Competitive Intelligence and Complex Systems Brigitte Gay University of Toulouse Toulouse Business School, France b.gay@esc-toulouse.fr Received June 1, revised form 10 September, accepted 27 September 2012 ABSTRACT: The economy reflects a dynamic interaction of a large number of different organizations and agents. A major challenge is to understand how these complex systems of interacting organizations form and evolve. The systemic perspective presented here confers an understanding of global effects as coming from these ever changing complex network interactions. Another main endeavor is to capture the interplay between individual firms’ alliance strategies and the dynamic interactions between all firms. In this paper, we advocate the use in competitive intelligence of a complex systems approach originating in statistical physics to understand the intricate meshes of interfirm interactions that characterize industries today, their dynamics, and the role major organizations play in these industries. KEYWORDS: Competitive intelligence, real-world networks, statistical physics, VisuGraph Introduction Because we need in competitive intelligence (CI) to analyze massive data in real time, using network visualization techniques to explore intricate interactions among organizations or agents is not enough. Large data streams require quantitative tools. The complex network approach developed in statistical physics is particularly adapted to the analysis of large networks. We advocate that recent developments in this field would help address the many issues CI is confronted with at this level. Physicists using a complex network approach have tried to infer the structural properties of large empirical networks. Statistical regularities have been observed in very diverse real-world networks (communication, biological, or social networks, etc.) when they were compared with network models generated with different stochastic algorithms, in particular small-world and scale-free network models (Watts and Strogatz, 1998; Barabasi and Albert, 1999). Progress in statistical physics was hence initially made with the identification of a series of unifying principles and statistical properties found in most empirical networks examined. Researchers anticipated that these studies would result in a better knowledge of the evolutionary mechanisms of complex networks as well as of their dynamical and functional behavior. We know however that the onset and outcomes of growing networks can be very different. In addition complex networks have heterogeneous structures that vary and extend over many possible levels. Comparing interfirm network structures across different industries has indeed recently revealed that though real-world growing networks may apparently share many properties they can be in fact very different (Gay, 2011). Moreover, they can deploy overtime very dissimilar architectures and switch from one particular structure to an altogether Available for free online at https://ojs.hh.se/ Journal of Intelligence Studies in Business 3 (2012) 5-14 mailto:b.gay@esc-toulouse.fr https://ojs.hh.se/ 6 different one. Major players also operate differently in the systems analyzed (Gay, 2011). Researchers must learn to evaluate the differences in the processes that take place on complex networks and start understanding the idiosyncrasies of both systems and agents’ behaviors. A recent shift in research on complex networks is to investigate more fully differences in network structures that may epitomize different behaviors. In this way the subtle dynamics that shape the different systems are also investigated. In particular, new contributions on structural properties have been made thanks to the development and use of novel metrics in statistical physics. They have for example provided evidence for the presence in networks of hierarchies (when applying k-core decomposition methods), communities ordering, and assortative mixing (Barrat et al., 2004; Girvan and Newman, 2002; Milo et al., 2002; Newman, 2002; Seidman, 1983; Shen-Orr et al., 2002). We use here some of these metrics to try to establish an understanding that complex webs of interactions that characterize industries today, as well as their evolution and dynamics, must also be considered a fundamental goal in competitive intelligence. To achieve this, we analyze the alliance networks of firms interacting in two different industries, the pharmaceutical industry (network 1) and the equity industry (network 2). We demonstrate that the two networks maintain many differences and that understanding networks dynamics is essential. 1. Looking for statistical ‘irregularities’ in network structures and firm position Networks are conceptualized here graph- theoretically, i.e. as objects containing nodes and links. A network is thus, in very general terms, a graph whose nodes identify the elementary constituents of the system, the interconnections between these constituents being represented by the linkages in the network. The nodes here are firms and the links actual contracts between firms. To assess the nature of the particular structure of the two experimentally observed interfirm alliance networks, we use 5 main network properties:  degree,  degree centralization (Freeman, 1979),  the k-core (Seidman, 1983),  the assortative coefficient (Newman, 2002), and  degree modularity (Newman, 2006). To capture the dynamics of the processes, we use different time windows. We then use the resulting adjacency matrices to construct the network metrics utilized. We study network 1 from 1998 to 2007 and network 2 from 1993 to 2008. Fig.1 shows the topology of the two networks (static data). The two graphs make evident the difficulty of analyzing intricate meshes of interfirm transactions. Both graphs also show that some firms or hubs (larger nodes on the graph) make many more transactions than others. The fact that there is a power structure in each industry is manifested, though we know nothing more about this phenomenon here, past the fact that some players in each industry deal a lot more than others. We do not know about its consequences, its level of influence on the industry in general and on its members. 7 Fig. 1. Network 1 (Left), Network 2 (Right). Node size is scaled to standardized network degree, or deal number, in the total network, reflecting variation in the extent of degree connectivity among the organizations. The darker lines indicate the presence of repeat ties between firms. Network structure in network 2 is highly cohesive. The presence of a hierarchical structure is also apparent in both networks. Hubs are tightly interconnected in Network 2, but not in Network 1. We will now use some network metrics to see if we can learn more from the power structures we observe and to refine the analysis. 1.1. Analysis of Network 1 The first metric we used is degree centralization. The degree of a node in a network is the number of links connecting it with other nodes. Degree centralization indicates how centralized an entire network is and is hence a macro-level measure. It is calculated as the sum of the differences between the maximum and each individual’s centrality score, normalized to range from 0 to 1 by dividing by the theoretical maximum centralization. A star network has maximum centralization, with value 1. Our data reveal that the interfirm network power structure varies and actually weakens over time, as demonstrated by variations in the centralization index (Fig. 2). Network 1 is first highly centralized, with few hubs. More hubs however increasingly participate in the network, though to an ever less extent and the network power structure hence weakens progressively. It is widely assumed that most social networks have a ‘community structure’, where nodes can be part of a tight group, while others may act as bridges between them. We use a new community centrality measure that identifies the participation of each node (central or not) within one or more communities in a network, defined by the leading eigenvectors of a characteristic matrix or ‘modularity’ matrix of the network (Newman, 2006). This measure helps to better understand how hubs/central firms operate within the different structures. In the first period, central players have small values for their community centrality, indicating that they operate globally (Fig. 3). The situation is however reversed in the latest period as central organizations have higher values for their community centrality. Firms with a higher degree tend then to exert control within communities rather than across the overall network structure. The network power structure thus evolves from globally to locally effective. 8 Fig. 2. Decrease in degree centralization overtime – Network Fig. 3. Change in community degree - Network 1 We also measured degree assortativity to link organizational and in particular hubs behavior to the structuring of the interfirm network. The assortative coefficient measures degree correlations. In other words, correlated graphs are classified as assortative if nodes tend to connect to their connectivity peers, and as disassortative if nodes with low degree are more likely connected with highly connected nodes. Networks of neutral mixing of their degree show none of these tendencies. Correlations are measured by the assortativity coefficient r, or Pearson correlation coefficient for the degrees at either side of an edge (Newman, 2002). The theoretical range is [-1, 1]. 0 5 10 15 20 25 30 35 40 Period 1 Period2 Period 3 Period 4 C e n tr a li za ti o n I n d e x ( % ) 0 0,5 1 1,5 2 2,5 0 0,1 0,2 0,3 0,4 0,5 C o m m u n it y C e n tr a li ty Degree Centrality Community Centrality, period 1 Community Centrality, period 4 9 We find strong variation with assortativity in the network (Table 1). We find a value for the assortativity coefficient of r = - 0.483 in the first period indicating strong disassortative mixing, i.e., hubs are primarily connected to less connected firms. Hubs are thus not linked together. Though the network remains disassortative, r increases continuously until the network shows neutral mixing in the latest period with r = 0,015. Firms then interact with all kinds of firms, similar or not. Linkages of hubs among themselves are therefore not a feature of the interfirm network as it is never assortative. 1.2 Analysis of Network 2 Centralization data also establishes that, as network 1, network 2 is dependent upon hubs for (Fig. 4). However we observe 2 peaks, the network is first highly centralized; centralization decreases afterwards and then regains some momentum though power is at that time distributed among more central organizations. These variations in the network power structure constitute key points that alert to change, individual as well as systemic. To probe these modifications further, we used the k-core decomposition method. Fig. 4. Network 2 centralization Network 1 r coefficient Period 1 -0,483 Period 2 -0,306 Period 3 -0,103 Period 4 0,015 Table 1: Assortativity coefficient The k-core decomposition is based on a recursive pruning of the least connected nodes (Fig. 5). The nodes displayed in the most internal of the shells of the network are those forming the central core of the network. Applying this method allow us to identify the inherent layer structure of a network and thus gain information about its hierarchical structure and the placement of hubs (globally central if in the innermost k-cores and locally central if hubs are merely members of the outer k- cores). Applying the k-core decomposition method, we investigated which firms made it to the central core by looking at the correlation between the degree of the nodes and what is called the coreness value. We determined the existence of 22 consecutive k-cores. 0 5 10 15 20 25 30 Period 1 Period 2 Period 3 Period 4 Period 5 C e n tr a li za ti o n I n d e x ( % ) 10 We find that hubs with the highest degree are within the inner shell of the network (only 2 hubs out of the first 58 firms with high degree are in the 18- and 20- core respectively for the period 1989- 2008, while all others belong to the innermost set of nodes, the 22-core; 89% of the nodes in the inner core are hubs). There is therefore a clear global hierarchical structure. There is no k-core fragmentation; the remaining nodes forming a k- core systematically belong to the same connected component (static and temporal analysis). Fig 5 (bottom) reveals the importance of examining dynamic displays of interactions, including the links between major players in the innermost k- core. We know the extent of the involvement of each hub in the system during each period (5 periods in total) as the nodes have been replaced by color-coded histograms that account for their degree centrality (standardized total number of transactions per firm) at each time point. VisuGraph visualization software allows positioning nodes according to their activity as they occur by (Gay and Loubier, 2009, 2012). We thus find that some hubs are highly active at all times whereas others have dropped their activity significantly after the second period. Interestingly, from period 4 onwards, more hubs become extremely active, thus explaining away the centralization data in Fig. 4. An easy way to pursue the analysis is to ‘tag’ the nodes/firms with additional data such as firms’ date of creation, number of employees, market capitalization, country of origin, etc. In Fig. 6, we investigate whether the same category of major players operate at all times. We find that basically one category of major players in this industry (pink nodes) operate during the first peak of centralization, shown in Fig. 4 while a second category, including more major players (blue nodes) becomes very active from period 4 to period 5, explaining away the second centralization peak, as well as the decrease in the centralization index observed during this second peak (Fig. 4). 11 Fig 5. Central players’ localization (Top) and networking activities (bottom) – Network 2. The top figure shows the number of major players in the different states (USA). Two categories of hubs are active (blue circles for category 1, red circles for category 2). The bottom figure highlights the activity in the latest 2 periods (periods 4 and 5) of more major players (within blue circle). This change is due to the sudden increase in transacting activity of the ‘category 2’ players displayed in the top figure. 12 Fig. 6. Dynamics of the transacting activity of major players from period 1 to 5 – Network 2. Pink rectangles represent category 1 players and blue rectangles category 2. When we compare the images in Fig. 5 (Bottom) to that in Fig. 5 (Top), we see the relevance of thinking of firms as interconnected organizations. The top figure gives the impression that a few isolated clusters of key firms operate in the US. In fact all major players in this industry are interacting between themselves, independent of their location, and they time their activity. To verify if these major players not only interact between themselves but also operate globally, we measured the community degree index. Fig.7 shows the results for community centrality. They highlight that while community centrality is correlated with degree (R 2 = 0,75), the two are not perfectly correlated. The effect is stronger for major players: they clearly transcend borders (lack of correlation Dc/community centrality for hubs). We find the same results whether we look at static or dynamic data. Therefore hubs control the network at all times even when their number increases and another category of actors surpasses the previous one. 13 Fig. 7. Community Centrality - Network 2 Using degree assortativity, we find that major players mix not only with other major players but also with less connected, smaller, organizations. The network is indeed non-assortative for the whole period (r = - 0.008) and falls under the slightly assortative mixing if we look at discrete time periods (r values ranging from 0,006 to 0,04). The non-assortative nature of the network establishes that major players interact among themselves but also with peripheral players. A more detailed analysis (data not shown) reveals that hubs tend to interact repeatedly between themselves while their deals with peripheral players are rarely repeated even though hubs interact with many different small players at all times. These small players allow hubs to have rapid expansion strategies when needed or to cope with uncertainty and crisis situations such as the subprime crisis. 3. Discussion What have we learnt from these data? For network 1, we’ve demonstrated that the influence of central players, strong at first, weakened rapidly through time and most importantly, that their influence is global in period 1 and only local in the last period examined. Not shown was that central players in period 1 were high tech players with radical innovations while as hubs were mostly among the top 10 global pharmaceutical companies in the last period. We’ve also shown that the relations were mostly asymmetrical, i.e. hubs interacted essentially with peripheral players. Hubs did not interact among themselves, except at the very end. For network 2, we showed again that the power structure varied, but with 2 distinct phases and peaks. Using the k-core decomposition method linked to dynamic visualization techniques, we quickly demonstrated that one category of major firms in the equity industry dominated the first phase. The 2nd phase is explained by the sudden arrival of a new category of big players in this industry. This time, major players interacted heavily between themselves and we could measure the differential co-involvement of the different hubs 0 1 2 3 4 5 6 7 0 0,05 0,1 0,15 0,2 C o m m u n it y C e n tr a li ty Dc 14 through time. Big firms in this industry also interact with smaller, less active firms. These peripheral firms are always dependent upon hubs and hubs clearly have a global influence for all periods of times. 4. Conclusion We want to stress the importance of considering the links between organizations or agents in CI. Economic and financial systems are built on interdependencies. Understanding their dynamics is crucial as these networked systems change rapidly. Though we’ve only used a very small set of metrics, we’ve proven that inter-organizational networks are very different and evolve differently. None of the metrics used here give a sense of ‘universality’ or of common mechanisms regarding the growth and dynamics of complex networks. The major role of big firms in both fields does not come as a surprise. We’ve demonstrated that hubs operated differently between networks, that different categories of hubs intervened, that hubs interact tightly or not at all among themselves, and that the power structure of a network can collapse (in this case, -in Network 1-, when it is led by highly innovative firms). We’ve also shown that major companies in both industries have different strategies and timing, and can operate globally or locally. This was done using a very small set of network metrics and a visualization software that can render networks dynamic, another key goal for CI analysts. Ongoing work consists of matching visualization techniques with statistical physics, accessible directly on graphs. This will give more input on systems and their agents. “More is different”. We highlight the importance of progressing in the field of statistical physics to help CI practitioners address differences between economic and financial systems, as system dynamics evolve rapidly due to endogenous as well as exogenous events (bubbles and busts, radical change, globalization, new rules, etc.). We also call attention to the importance in CI of understanding the interplay between micro- and macro- behavior (i.e. the influence the strategy of individual firms may have on the macro systems they are embedded in and the constraint/influence that these economic or financial systems in turn may exert on individual organizations). The ultimate goal is to give managers guidelines to help them understand the different environments in which they operate and position their firms. Some firms can also change/govern economic/financial environments or alter their power structure. References Barabási, A. L., & Albert, R. (1999). Emergence of scaling in random networks. Science, 286, 509– 12. Barrat, A., Barthélemy, M., Pastor-Satorras, R., & Vespignani, A. (2004). The architecture of complex weighted networks. Proceedings of the National Academy of Sciences USA, 101 (11), 3747–3752. Freeman, L. C. (1979). Centrality in social networks: Conceptual clarification. Social Networks, 6:223–258. Gay, B. (2011) Universal dynamics on complex networks, really? A comparison of two real- world networks that cross structural paths (…) but ever so differently. In: “Social Network Mining, Analysis and Research Trends: Techniques and Applications”, Eds, I-Hsien Ting, Tzung-Pei Hong and Leon S.L. Wang, National University of Kaohsiung, Taiwan, In: IGI Global, NY, USA. ISBN 978-1-61350-513-7 (hardcover) -- ISBN 978-1-61350-514-4 (ebook) ISBN 978- 1-61350-515-1 (print & perpetual access). Gay, B., & Loubier, E. (2009). Dynamics and evolution patterns of business networks. Paper presented at the international conference on Advances in Social Network Analysis and Mining (ASONAM), Athens, Greece. Gay, B., & Loubier, E. (2012) « Worldwide Dynamic Evaluation of Complex Business Networks.” In: "Dynamic Analysis for Social Network". Editor: Carlos Andre Pinheiro, iConcept Press. ISBN 978-14610987-3-7. Girvan, M., & Newman, M. E. J. (2002). Community structure in social and biological networks. Proceedings of the National Academy of Sciences USA, 99, 7821–26. Milo, R., Shen-Orr, S., Itzkovitz, S., Kashtan, N., Chklovskii, D., & Alon, U. (2002). Network motifs: simple building blocks of complex networks. Science, 298, 824–27. Newman, M. E. J. (2002). Assortative mixing in networks. Physical Review Letters, 89, 208701. Seidman, S. (1983). Internal cohesion of LS sets in graphs. Social Networks, 5, 97–107. Shen-Orr, S. S., Milo, R., Mangan, S., & Alon, U. (2002). Network motifs in the transcriptional regulation network of Escherichia coli. Nature Genetics, 31, 64–68. Watts, D. J., & Strogatz, S. H. (1998). Collective dynamics of ‘small-world’ networks. Nature, 393, 440–442.