21 CUSTOMER DEVELOPMENT, INNOVATION, AND DECISION-MAKING BIASES IN THE LEAN STARTUP Jonathan L. York California Polytechnic State University jlyork@calpoly.edu Jeffrey E. Danes California Polytechnic State University jdanes@calpoly.edu ABSTRACT This paper reviews current research relevant to new product development, customer development, and the lean startup. Customer development and the lean startup are a new and increasingly used form of entrepreneurship process, which rely on hypothesis testing but not in the traditional sense; the entrepreneur is encouraged to scan the environment, collect information, and form and evaluate educated guesses so as to make accurate judgments and decisions. The present research provides a review of the customer development model for entrepreneurial activities and a critique of this hypothesis testing methodology. We then consider ways in which to improve decision making within the startup via a systematic study of System 1 ( intuition) and System 2 ( reasoned and rational) decision-making styles. This paper has significant implications for entrepreneurs, entrepreneurship support organizations, such as incubators and accelerators, and entrepreneurship educators, all of whom are increasingly practicing and teaching this process. Keywords: biases and heuristics, customer development, entrepreneurship, innovation, lean startup, new product development INTRODUCTION This paper discusses a newly emerging and increasingly popular approach to entrepreneurial practice known as customer development and the “lean startup” process (Blank, 2007; Blank & Dorf, 2012; Ries, 2011; Maurya, 2012). For purposes of this paper, the lean startup process is defined as an approach to entrepreneurial and innovative activities that emphasizes placing resources into the creation of customer value, viewing all other activity as waste until a fit is found between the S TRATEG Y JOURNAL OF SMALL BUSINESS Journal of Small Business Strategy Vol. 24, No. 2 22 product and the intended market. The lean startup relies upon a process called customer development, which is a method of creating and testing assumptions regarding the end business model for the startup. The lean startup process is rapidly being adopted by university entrepreneurship programs, accelerators, entrepreneurial organizations like Startup Weekend, communities, and even the NSF (2012), which is employing this methodology for its Innovation Corps Sites Program. However, there has been very little if any systematic analysis of the lean startup and customer development methodologies. Therefore, we place customer development within the theoretical context of new product development; the methods by which the entrepreneur determines the viability of the innovation as well as the needs, features, and functionality for the product clearly fall under the overarching framework of new product development whether in an entrepreneurial or more stable environment. We review customer development as an entrepreneurial practice within the context of earlier product development models such as Cooper’s New Product Development (NPD) (Cooper 1988; Cooper 2008) and Koen’s (2004) new concept model for the fuzzy front end (FFE). Within this theoretical framework, we turn to a key issue in customer development. More than earlier forms of NPD, customer development relies heavily on hypothesis testing, but not in the traditional sense. Instead, the entrepreneur is encouraged to survey the environment, collect information, and form and evaluate educated guesses so as to make accurate judgments and decisions. Customer development recognizes that there may be a place for “intuition” within the entrepreneurial process and responds to the need to validate this intuition with more formal processes that make sense in the highly uncertain and time-sensitive startup environment. Below we briefly review formal methods of hypothesis testing and review Blank’s notion of "hypothesis testing” and find it lacking in rigor. Using the concept of System 1 (intuitive) and System 2 (systematic, reasoned) thinking (Stanovich and West, 2000), we then suggest methods which could be integrated into the customer development process to mitigate the significant risks of these biases, and place these in the context of entrepreneurial practice and teaching. CUSTOMER DEVELOPMENT IN THE CONTEXT OF NPD New product development over the past two decades has been heavily influenced by the “stage-gate” model, developed by Cooper (1988, 2001). A significant benefit of the traditional NPD model is its structure, allowing companies to follow a roadmap and prescriptive steps guiding their process. Yet, according to Cooper, the traditional new product development process was designed for incremental product development and “... may be inappropriate …” (Cooper, 2001, p. 151) when applied to breakthrough and platform projects. A limitation of the early, traditional NPD model is the emphasis on incremental product improvement with a cursory focus on ideation and discovery. Other limitations include a defined linear process with the implication that “looping back” to an earlier stage denotes a gating mistake or gating error in development. In addition, product development cycles, sped up today by competitive markets and enabled by technologies such as rapid prototyping (even on the desktop with 3D printing for example) and internet distribution, can be slowed significantly by the traditional NPD Journal of Small Business Strategy Vol. 24, No. 2 23 approach. In addition, stage gate may actually lead to a d ifficulty delivering final products because each stage has higher costs and may take longer as the company moves through the process (Anderson, 1993). The traditional NPD model proposed by Cooper (1988) and as modified (Cooper, 2001) may be viewed as a very successful attempt at a very logical and coherent methodology of launching new products. Cooper emphasizes the need for customer- facing activities in the earliest stages of NPD, with the realization that many projects fail because of an overemphasis of the technical tasks over the marketing/business-oriented tasks during what he calls the “homework” or predevelopment stage (Cooper 2011; Cooper 2013; Cooper, 2013a). As Edgett (2011) notes, activities taken before the formal design and development of the product play a key role in determining success or failure. Cooper (2014) himself has recognized the need for a more flexible process and has recently proposed “the Triple A System” of an adaptive, agile and accelerated modification to Stage Gate (p. 21). There is an abundance of research showing how greater emphasis on the front-end activities (meeting the needs of customers) explains why some products are more successful than others (Henard & Szymanski 2001). In studying a number of industries, Koen (2004) observed that communication and decision making is circular, nonlinear, and non-sequential, rather than following the linear model of NPD. The new concept model (NCD) developed by Koen (2004) was designed to emphasize that ideas are thought to flow, circulate, and iterate between and among five elements: opportunity identification, opportunity analysis, idea generation, idea selection, and concept definition. This contrasts with the traditional, sequential NPD process where “looping back” may be viewed as a gating error as opposed to a reasoned, positive correction. In contrast to the traditional NPD process, new concept development processes include a greater focus on idea generation activities such as in-depth interviews, brainstorming, and idea management tools (e.g., crowdsourcing) aimed at potential and current customers, lead users, employees, and other stakeholders. Early innovation FFE activities are often disorganized, unpredictable, and unstructured, as opposed to the later phases of new product development which are typically a more structured and formalized process. Koen defines the FFE as all activities that come before product development (Stage 3) of the five steps of the traditional NPD process. Likewise, Brentani and Reid (2012) define FFE as “the time and activity prior to an organization’s first screen of a new product idea, “ (p. 70), implying the FFE “ends” at the point of traditional “product development” activities. In other words, it can be argued that NPD and FFE processes are product-centric and downplay the ensuing business development activities, taking for granted that these processes are embedded in a larger company’s already existing structure. Blank (2007) on the other hand carries his customer development approach all through the new product development and launch process, arguing that any functional systematization (e.g., “company building”) occurs and continues after product launch, and that customer development is essentially never truly “completed.” Table 1 describes some key differences between traditional new product development (NPD), the fuzzy front end (FFE) and as described by Koen et al. Journal of Small Business Strategy Vol. 24, No. 2 24 Table 1: Differences between Traditional NPD, FFE, and CD New Product Development (NPD) Fuzzy Front End (FFE) Customer Development (CD) Nature of Work Disciplined and goal- oriented with a project plan Experimental, often chaotic. “Eureka” moments. Can schedule work— but not invention Iterative, with continual influx of new information processing hypotheses Commercialization High degree of certainty at conclusion Unpredictable or uncertain Higher degree of certainty after completion of CD process Funding Date Budgeted Variable — in the beginning phases many projects may be “bootlegged,” while others will need funding to proceed Only for minimum viable product until business and sales model developed Revenue Expectations Predictable, with increasing certainty, analysis, and documentation as the product release date gets closer. Often uncertain, with a great deal of speculation Must know sales and pricing model for first stage introduction Activity Multifunction product and/or process development team Individuals and team conducting research to minimize risk and optimize potential All members involved in extensive “outside of the building,” largely 1:1 customer contact Measure of Milestone achievement Progress Strengthened concept Minimum viable product; product/market fit Expenses Increase with each stage Increase with each stage Revenue can begin after MVP identified Decision Process Go/No Go Kill stages Indeterminate Pivots to new directions Journal of Small Business Strategy Vol. 24, No. 2 25 (2002) and Koen (2004), and customer development as described by Blank (2007). Customer Development (CD) Over the past several years, a new approach to the fuzzy front end, loosely grouped under the rubric of “customer development,” has been advocated and promoted by Blank (2007) and others, such as Blank and Dorf (2013), Cooper and Vlaskovits (2013), and Maurya (2012). The customer development model considers four interlocking and circular stages: 1) customer discovery, a focus on understanding customer problems and needs, 2) customer validation, the identification of a scalable and repeatable sales model, 3) customer creation, creating and driving end user demand, and 4) company building, the transitioning of the organization from one designed for learning and discovery to efficient execution. A unique feature is that, following the initial “hunches” of the entrepreneurial team regarding their business model, customer development is almost entirely a p rocess of direct contact with customers and others outside the company for knowledge acquisition and hypothesis testing (Blank & Dorf, 2012). All assumptions are to be challenged through the first phase of CD. A product should be launched (“put into the hands of customers”) as soon as possible to increase the level of feedback. As Blank (2007, p. 21) notes, “In essence, Customer Discovery and Customer Validation corroborate your business model. Completing these first two steps verifies your market, locates your customers, tests the perceived value of your product, identifies the economic buyer, establishes your pricing and channel strategy, and checks out your sales cycle and process.” Only then, according to Blank, does the company move to creating the full business plan and company development. Proponents of customer development assert that the greatest risk for failure for any innovative product is not development of the product but the lack of product-market fit. Good fit drives growth. Poor fit can manifest itself in a lack of customers, and/or difficulty in identifying customers, and/or the higher costs involved in finding customers for the innovation. Methods of mitigating this risk of poor fit are of considerable value to the effort. As such the focus is on the early stages of innovation. Proponents of a structured CD approach (Reis, 2012; Blank & Dorf, 2012) recommend a significant degree of “certainty” in the product-market fit before the company pursues building a formalized, complete business plan. Mullins (2013) also argues for significant customer development activity prior even to the launch of a business. As well as arguing for more traditional analysis of factors such as competition and market conditions, he proposes a process of customer interviewing very similar to customer development approaches. Another key feature of CD is product simplicity, often called the “minimum viable product.” In this model, the product is a set of “minimal requirements,” which meet the needs of the core group of early adopters or users. Also, cash burn is kept to a minimum until creating and driving end user demand (customer creation), and unlike the traditional NPD model, there are no kill gates, only continual iteration and reintroduction. Using the principles of the lean startup (Ries, 2011), Blank & Dorf (2012) argue for shipping product to early adopters rapidly and then following up with customer driven improvements, creating an iterative product development process. The lean startup methodology allows for significant changes throughout the process, often called “pivots,” (Ries, 2011). Journal of Small Business Strategy Vol. 24, No. 2 26 DelVecchio, White and Phelan (2014) note that with traditional NPD, “If customer feedback indicates a major redesign is required, the product will likely cease moving on to the next stage and the gatekeeper will “kill” the development of the product.” (p. 7) CUSTOMER DISCOVERY AND HYPOTHESIS TESTING Proponents of customer discovery emphasize hypothesis development and testing as an integral part of the customer development process. Below we review Blank’s interpretation of “hypothesis testing,” followed by a brief discussion of more formal models of hypothesis testing (frequentist and Bayesian). Next we make an argument that a more attainable goal is to focus on the frailties of entrepreneurial decision-making, and then consider ways to improve it. A key process in customer development is the testing of hypotheses about the problems faced by customers, the minimum feature set of the product, the product/market fit, product improvements, etc. Hypothesis development is concerned with how we determine what we initially think is “true” about the world and, given appropriate information, what then appears to be actually “true” about the world. The outcome of such testing guides decision making. Proponents of CD define a three- stage process, which includes a first stage of generating hypotheses, followed by testing the hypotheses, and finally, using the results of testing to evaluate the hypothesis. The challenge for the entrepreneur, as Blank (2013) suggests, is to generate and test hypotheses rapidly and frequently, and evaluate and reevaluate results before coming to the decision-making stage and identifying a course of action. Depending on the type of new product or new business, this can be done in a completely iterative process (such as for a website that is upgraded constantly) or in a step-by-step process, such as for a physical product where prototypes must be tested regularly. For Blank, hypothesis generation is essentially a founding team process based upon intuition into the market, beginning with educated guesses about the nature of the problem(s) and the target market the company proposes to address. Hypothesis testing follows these intuitions about the business model and is conducted by what Blank (2007) calls “getting out of the building” – in other words, talking to customers, users, and experts, to determine whether the problem, which the startup posits, is indeed significant and what the scope and characteristics of that problem may be. The hypothesis evaluation step at this stage is again one of team review and discussion, a group process. The next step is a continuation of the first, wherein a prototype or first iteration of product, what Ries (2011) deems the “minimum viable product,” is exposed again to the customer and hypotheses regarding its value are again tested. For Blank and other adherents of customer development, this process does not end, as each new decision involves another set of hypotheses that must be generated, tested, and evaluated. Each tested hypothesis yields a decision, a course of action. For Blank, this process is almost entirely derived from subjective information obtained by direct one-to-one contact with potential customers, suppliers, partners, etc. As noted, Blank’s (2007) CD process is built on a continual cycle of hypothesis testing, decisions, and corrections, which lends itself well to situations of highly novel product introductions where there is direct contact with potential consumers. In this way, entrepreneurial decision-making Journal of Small Business Strategy Vol. 24, No. 2 27 may indeed be viewed as a form of scientific process of discovery and learning rather than a purely intuitive process of recognition and alertness to market conditions. (Harper, 1999). Blank’s customer development hypothesis development and evaluation is a Bayesian decision-making process, as proposed by Fischhoff and Beyth-Marom (1983). That is, one develops hypotheses (prior belief) based upon experience or intuition, identifies data sources useful to evaluate hypotheses, observes or gathers data to evaluate the truth of competing hypotheses, aggregates data into an overall appraisal of likelihood of a hypothesis being accurate, and selects a course of action based upon that evaluation. The ultimate goal is to make good decisions. Yet, as is well known, true Bayesian decision making requires significant amounts of data, which the entrepreneurial environment may not provide, nor the entrepreneur be able to implement. Hypothesis testing in scientific efforts is grounded in rigorous statistical thinking. Formal models for hypothesis testing are logical and systematic processes. The essence of hypothesis testing is concerned with how one decides whether there is a match between what we initially think is true and what the evidence (data) shows is not true. That is, is the evidence strong enough to reject the corresponding null hypothesis? In practice, effective decision making follows from “truth” as determined by statements of hypotheses (educated guesses), the testing (evaluation) of hypotheses, and the verification of hypotheses. However, Blank’s (2007) entrepreneurial hypothesis testing methods are informal and tend to rely on intuitive thinking as noted above and discussed below. Blank (2007) refers to “hypothesis testing” as talking to customers, users, experts, etc., to determine whether what the startup sees as a problem is indeed one, and what the scope and characteristics of that problem may be. As we will see later, this form of informal hypothesis testing is prone to certain biases and heuristics which can impair its accuracy. There are two general types of hypothesis testing methods: frequentist and Bayesian. The classical frequentist-based hypothesis testing is a formal statistical process; however, within the social context of entrepreneurial activities, classical hypothesis testing is problematic given limited information, subjective opinion, and the need for rapid decisions. On the other hand, Bayesian inference is a normative method in the sense of prescribing how hypotheses should be evaluated given prior information and new information. The Bayesian model is better suited to the analysis of subjective information and opinion, which makes it much more relevant to entrepreneurial decision-making and customer development. Nevertheless, the data requirements are tedious and thus appear to be impractical relative to decision making in the entrepreneurial context. The complications of elementary Bayesian decision making are discussed in Lindley (1985). Therefore, instead of focusing on hypothesis testing, either classical or Bayesian, we focus on a more attainable goal, reducing the biases inherent in entrepreneurial decision making, especially in the context of customer development. Risks of Customer Development from Biases and Heuristics To set the stage for reducing bias in decision making, we now examine some of the most significant biases and risks in customer development hypothesis testing and then, using the concepts of system thinking (Stanovich & West, 2000), we examine research findings relevant to Journal of Small Business Strategy Vol. 24, No. 2 28 reducing the bias in decision making. We then propose ways in which the practitioner of customer development in the entrepreneurial context can improve decision-making. Entrepreneurs tend to be overly active, face time constraints, and hence, tend to rely on intuition. According to Stanovich and West (2000), intuition as a basis for decision making is fast, automatic, effortless, implicit, and emotional (referred to as System 1). System 2 refers to reasoning, which is slower, conscious, effortful, explicit, and logical. Levels of System 2 thinking include unstructured, clinical, and assisted (e.g., training). An excellent discussion of the implications of Stanovich and West’s (2000) work is given in Stanovich (2010) and in Kahneman (2011). Entrepreneurs often lack important information regarding a decision, fail to notice available information, and face time and cost constraints; hence, they tend to rely on intuitive System 1 thinking. But, reliance on System 1 thinking, intuition, has certain inherent weaknesses which may result in poor decisions. As Kahnemann (2011) notes, expert intuition is most trustworthy in an environment that is regular and predictable, and where the expert has had sufficient practice to learn the regularities of this environment, scarcely descriptive of the startup situation. Given that many entrepreneurial activities rely upon subjective and sparse information, this decision making is prone to significant errors in judgment. The presence of such forms of error in inquiry is well documented (Tversky & Kahnemann, 1974; Holcomb, Ireland, Holmes & Hitt, 2009). Busenitz & Barney (1997) established specific differences in two biases (overconfidence and representativeness) between entrepreneurs and managers in large organizations. Schade and Koellinger (2007) review a detailed list of perceptual biases and heuristics in general that affect entrepreneurial decisions, including whether to start a business. Several of these biases and heuristics seem to carry the most risk for the customer development process: Selection bias. In its most common manifestation, the data that an entrepreneur gathers may be biased if she looks only to friends, colleagues, and known sources for testing her hypotheses. Blank argues that “getting out of the building” is a hedge against this bias, but many entrepreneurs still gravitate to comfortable and confirmatory sources. (Holcomb et al., 2009) Representativeness bias. In the dynamic and uncertain startup environment, it is perhaps natural to generalize from small, non-random samples of data. Yet, to the extent that this bias interacts with the selection bias noted above, the validity of customer development information gathered can be severely compromised. Acquiescence bias. Related to selection bias, this represents the tendency of respondents to give the answers they think the entrepreneur wants to hear, rather than their unvarnished opinion. This is often referred to in the startup world as “not wanting to tell someone their baby is ugly.” Confirmation bias. People tend to favor or interpret information in a way that confirms their prior beliefs. If we “believe” that a problem exists, we will listen for any evidence that “confirms” this belief and ignore all else. Overconfidence bias. “Overconfidence is the tendency for people to overestimate their knowledge, abilities and the precision Journal of Small Business Strategy Vol. 24, No. 2 29 of their information” (Bhandari & Deaves, 2006, p. 5). In many ways, overconfidence is essential for the entrepreneur to be able to act in an environment of such uncertainty (Koellinger, Minniti & Schade, 2007), but when overconfidence leads to the blocking out of new evidence or alternative perspectives, it can be detrimental to the customer development process. Optimism bias. Kahneman (2011) notes that optimism is an essential characteristic of the entrepreneurial mindset but that it can lead to a strong tendency to ignore the relevant evidence. Cooper, Woo & Dunkelberg (1988) found entrepreneurs indeed experienced extreme optimism no matter the degree to which they were likely to succeed based on objective factors such as experience and the nature of the new business. In fact, optimism may be detrimental in the decisions of what kind of risk to take on, but essential in the actual implementation of the business once these decisions are made. For the startup entrepreneur, placing himself in the appropriate environment and with a group of teammates who will help temper this optimism bias in the early stages can mitigate the risk of poor decisions. Slater, Mohr, and Sengupta (2013) note that an organizational culture which emphasizes team interaction; however informal, leads to more successful radical product innovation capability. For innovators, this means “getting outside one’s head” and allowing all input, even that critical of the idea. Reducing Bias in Customer Development Processes An entire body of work has evolved over the last 20 years to illuminate the differences between intuitive (System 1) thinking and System 2, a more rational, logical and calculating approach. As we have noted above, the startup environment thrives on intuitive (System 1) entrepreneurs – without them there would be no startups. Purely relying upon this intuitive and error-prone approach is too often a recipe for failure; yet the prior “System 2” methodologies used in innovation development, such as the traditional NPD process, may be inappropriate substitutes for the entrepreneur due to time and information constraints, and do not take into the account the highly uncertain nature of the entrepreneurial environment. Customer development seems to address this gap by offering a framework and techniques for testing the hypotheses that the startup has created through its more intuitive processes. Nevertheless, even moving to a “hypothesis testing” approach in the startup does not remove the risk that these very same biases will manifest themselves in the sources of information, the type of information sought, and the interpretation of that information. In the following section, we suggest certain approaches that may help to mitigate the risks of System 1 biases in the customer development process. Table 2 cross references these bias-mitigating techniques against the specific biases inherent in customer development. There is significant research that shows that System 1 thinking can be improved and that System 2 thinking can be encouraged. For example, a r eview of research by Fischhoff (1982) on four strategies as potential solutions for biased decision making found extended training and feedback, coaching, and other interventions to improve judgment. In specific, they found training and feedback to be superior to (1) offering warnings about the possibility of bias; (2) describing the direction of a bias; and (3) providing small doses of feedback. However, extended training with feedback only produced only moderate improvements Journal of Small Business Strategy Vol. 24, No. 2 30 Table 2: Bias Mitigation Techniques in Customer Development Bias Description Mitigation techniques Selection bias seeking information from “friendly" confirmatory sources resulting in unrepresentative sampling of the target market(s) founding team members pair up in “getting out of the building” and check each other’s assumptions at all times careful attention to selection of interview targets Representativeness bias generalizing from small, non- random samples of data and/or information from respondents who do not represent the target market(s) conducting customer development interviews as an iterative and continuous process, checking previous generalizations against new data Acquiescence bias respondents’ tendency to give the answers they believe the entrepreneur wants to hear carefully structuring customer development interviews to avoid “yes/no" answers; not asking respondents to speculate on future behavior, but focusing on past and current behavior Confirmation Bias interpreting information to confirm prior beliefs maintaining open-ended interview discussion focused on problems, not proposed solution improper linear model development to simulate data-driven decision where actual measurement data is impractical if not impossible Overconfidence bias overestimating the knowledge and precision of customer suggestions and/or the entrepreneurs information engaging mentors and advisors to provide an unbiased perspective on the information gathered and to create an environment of System 2 thinking around the startup premortem exercises looking at potential causes of failure rather than assuming success; a belief that only the paranoid survive Optimism bias the entrepreneur’s belief that he/she is unlikely to experience negative outcomes or fail analogical reasoning - engaging in “reference class forecast" activities comparing to other similar startups"consider the opposite” activities to force more critical thinking Some general bias mitigation activities include: • locating in an environment with other startups and advisors (incubator, co-working) where external feedback is readily available • undergoing training in customer development activities to strengthen information-gathering skills • conducting all customer development activities as a team • engaging a mentor or advisor to play the devil’s advocate role in all customer development activities Journal of Small Business Strategy Vol. 24, No. 2 31 in decision making. The training research reviewed by Fischoff (1982) may be viewed as encouraging System 2 type thinking. The training research reviewed by Fischoff (1982) may be viewed as encouraging System 2 type thinking. In the startup environment, team members can provide checks and balances, in a sense offering to each other the initial training and feedback that can prevent against the injection of bias into the customer development process. In fact, Blank (2005) insists that all members of the founding team, regardless of discipline, take part in the “out of the building” hypothesis testing activities in order to eliminate the natural tendency for engineers to make their assumptions, business types to make theirs, and the result to be a compromise between opinions rather than some form of “fact,” a strongly held prior belief. There are other mechanisms that tend to encourage System 2 thinking. Kahneman and Lovallo (1993) discuss attempting to take an outsider’s perspective, that is, trying to remove oneself mentally from a specific situation or to consider the class of decisions to which the current problem belongs. For example, an entrepreneur could look at similar type startups to get what Flyvbjerg (2009) calls a “reference class forecast” of how much money it took to get to launch, how long until profitability, etc. Taking an outsider’s perspective has been shown to reduce decision makers’ overconfidence about their knowledge (Gigerenzer, Hoffrage, & Kleinbölting, 1991), the time it would take them to complete a task (Kahneman & Lovallo, 1993), and their odds of entrepreneurial success (Cooper et al., 1988). Decision makers may also be able to improve their judgments by asking a genuine outsider for his or her view regarding a decision. Kahneman (2013) provides a lucid discussion of this research. To do this requires the entrepreneur to be open to negative input, not an easy task. Encouraging people to “consider the opposite” of whatever decision they are about to make tends to reduce errors in judgment caused by various biases: overconfidence, the hindsight bias, and anchoring (Larrick, 2004; Mussweiler, Strack, & Pfeiffer, 2000). Reduced error in judgment has also been achieved by having groups rather than individuals make decisions, training individuals in formal reasoning, and making people accountable for their decisions (Larrick, 2004; Lerner & Tetlock, 1999). Thus, while the startup team is most accountable for decisions during the customer development activities, having an outside set of advisors, or an advisory board can insert a level of bias-reduction into the process. Analogical reasoning research has examined how System 2 thinking can be leveraged to reduce System 1 errors. Analogical reasoning can be used to reduce bounds on awareness (see Bazerman & Chugh 2005). Stemming from work by Thompson, Gentner, and Loewenstein (2000), Idson, Chugh, Bereby-Meyer, Moran, Grosskopf, and Bazerman (2004) found people who were encouraged to comprehend common principles underlying seemingly unrelated tasks improved in their ability to discover solutions in a different task that relied on the same set of underlying principles. For example, an entrepreneurial team can study the success or failure of a startup in a separate but related field to open up their awareness to contradictory rather than only confirmatory evidence. Recent research on joint-versus-separate decision making indicates that people can Journal of Small Business Strategy Vol. 24, No. 2 32 move from intuitive System 1 thinking toward the rational System 2 thinking if they contemplate and then choose between multiple options simultaneously rather doing the same separately for each option. Research by Bazerman, White, and Loewenstein (1995) suggests that choice options should be evaluated jointly rather than sequentially. For example, suppose the team is considering the match between three different product embodiments with three different target markets. Their research shows that evaluating the elements of the set jointly, collectively at one point in time, is superior to their evaluation sequentially at different points in time. Consistent with this work, Bazerman, Loewenstein, and White (1992) have found people exhibit more willpower when they weigh their choices jointly rather than separately. That is, a collective decision has more resolve than an aggregation of individual decisions. The entrepreneurial environment itself can be staged to encourage better decisions given System 1 thinking. Thaler and Sunstein’s (2003) work on “choice architects” suggests that mentors, accelerators, and incubators can design situations so as to encourage better choices, given known decision biases. Having an environment around the entrepreneurial team of people who can question the input from the hypothesis testing and provide neutral feedback can serve as a de facto “System 2” overlay upon the process. In addition, the entrepreneurs in these programs can be trained, and supported, in reducing bias in their customer development processes, and be put into cross-team situations that encourage analogical reasoning to expand bounds of awareness. An example of a technique that can be employed is the premortem method suggested by Klein (2007). In essence, the entrepreneurial team “makes” a key decision in the planning stage, and then is asked to fast forward a year or two and imagine that the decision has failed disastrously. By proactively discussing the hypothetical reasons for this failure, the team can counteract the overconfidence bias. One of the most successful System 2 decision-making mechanisms is the improper linear model research associated with Robyn Dawes. Dawes and Corrigan (1974), in their seminal article, report simple linear models to be superior to clinical-type judgment and to simulate a System 2 t ype of decision making. Dawes and Corrigan (1974) report that simple linear models work well with input variables that have a conditionally monotonic 1 relationship with the output. Numerous studies report that linear models produce predictions that are superior to those of experts across an impressive array of domains, see Hastie and Dawes (2010). Moreover, Dawes (1971) has shown that even improper linear models outperform clinical judgments. The remarkable implication of the above research shows that clinical and intuitive System 1 thinking can be transformed into rational System 2 by 1) identifying potential predictors of an outcome, and then 2) providing a guess at the weights in a linear model. The weights could be as simple as: -1, 0, or +1, for a negative, neutral, or positive relationship or be a simple rating system: 0,1,2,3,4 followed by the identification of the algebraic sign (+,-) denoting the relationship of the variable with the outcome. 1 A sequence {a(n)} such that either (1) a(i+1)>=a(i) for every i >=1, or (2) a(i+1)<=a(i) for every i>=1. Journal of Small Business Strategy Vol. 24, No. 2 33 Linear regression produces optimal weights; however, Dawes (1971) has shown that linear models with non-optimal weights are also superior to intuitive (System 1) judgments. This holds promise for the creation of models wherein there is little possibility of fully accurate measurement. When error is introduced through measurement rather than the item being measured, deviations from optimal weighting do not make much practical difference in the quality of decisions. Good predictions may be obtained by selecting variables that have some validity for predicting the outcome. An improper linear model with subjective weights guessed at by the user is likely to be just as accurate in predicting new cases as is ordinary least squares regression. The classic example is the Apgar score for new born babies, as described by Kahneman (2011, p. 277). “One day over breakfast, a medical resident asked how Dr. Apgar would make a systematic assessment of a newborn. “That’s easy,” she replied. “You would do it lik e this.” Apgar jotted down five variables (heart rate, respiration, reflex, muscle tone, and color) and three scores (0, 1, or 2, depending on the robustness of each sign). Realizing that she might have made a b reak through that any delivery room could implement, Apgar began rating infants by this rule one minute after they were born. A baby with a total score of 8 or above was likely to be pink, squirming, crying, grimacing, with a pulse of 100 or more-in good shape. A baby with a score of 4 or below was probably bluish, flaccid, passive, with a slow or weak pulse- in need of immediate intervention.” Applying Apgar’s score, the staff in delivery rooms finally had consistent standards for determining which babies were in trouble, and the formula is credited for an important contribution to reducing infant mortality. The Apgar test is still used every day in every delivery room.” The predictive accuracy of the improper linear model, the Apgar score, has subsequently been validated against extensive outcome data (Moster, Lie, Irgens, Bjerkedal & Markestad, 2001; Carter, McNabb & Merenstein, 1998). The amazing success of subjectively defined weighting schemes has an important practical implication for the entrepreneur as it is possible to develop a useful predictive model without any prior data collection such as that required for building a regression model. For example, a startup can develop a simple and useful improper linear model by creating a set of key variables that the entrepreneurial team feels is valuable to measure, providing a subjective weighting system to these variables, and then using the customer development process to assign scores to each of these. A core framework for this would be the Business Model Canvas (Osterwalder & Pigneur, 2010) which breaks down business models into nine graphically connected segments. The startup might determine, for example, that the key variables from the Business Model Canvas are the potential revenue models, the customer segments, and the presence of key partners in the distribution channel. By simultaneously evaluating each of its possible alternatives here, and assigning each a subjective “score,” the entrepreneurial team has essentially created a predictive improper linear model. It is critical that these evaluations be conducted in a simultaneous rather than sequential Journal of Small Business Strategy Vol. 24, No. 2 34 manner; in other words, the process suffers if the company eliminates one possibility before moving on to the next instead of scoring them all. As noted, even though these items and their weights are user- defined and non-optimal, a decision based upon this process brings non-biased System 2 thinking into the process. DISCUSSION, CONCLUSIONS, AND FUTURE RESEARCH The review and study of customer development and lean launch methodologies is important to innovation and entrepreneurial activities, as these processes are becoming well instantiated at the practitioner level. The speed required for successful innovation will not abate, and thus, both academics and practitioners will continue to seek more effective frameworks and processes. The concept of lean startups, driven by such tools as customer development, promises to remain an important part of the entrepreneurial approach. As noted, the National Science Foundation recently adopted the Customer Development approach to the commercialization of NSF funded innovation through its i-Corps program (NSF, 2012), and many leading universities have restructured their innovation and entrepreneurship education curriculum around this methodology ( c.f. Stanford, Berkeley, MIT). Larger companies are starting to explore variations of this process for “intrapreneurship” activities as well (Karlsson & Nordström, 2012), in industries as diverse as manufacturing (Blank, 2013) and healthcare (Silva & Nascimento, 2013.) The process of customer development is intended to not only accelerate the process of innovation (the FFE) but also reduce the risk of error by continual iteration on the product-market fit before significant investment. There is little in this process that isn’t a part of one or another of the earlier new product development frameworks, but CD’s emphasis on full team participation, continuing face-to-face customer contact, and reliance on “out of the building” work have the potential of eliminating some of the biases inherent in the innovation process. On the other hand, customer development has its own risks that need to be acknowledged and addressed. For example, there can be a tendency, especially for the first-time entrepreneur, to become more focused on the activity of customer development than the outcomes, in essence to rely too much on fine tuning these results at the expense of their intuition. In other words, customer development is no more inoculated from “paralysis by analysis” than any other NPD processes. Furthermore, there is a key role for actual data collection and analysis in the early stages of the startup (Croll & Yoskovitz, 2013) through the introduction of key metrics and testing. Customer development can provide the direction for choosing these metrics, so that assumptions and hypotheses can be tested empirically wherever possible. As noted, educators are increasingly relying on lean startup and customer development activities for college and even high school level entrepreneurship courses. This provides a unique opportunity to educate their students about entrepreneurial biases, and to prepare them with the tools and techniques to minimize biases in both information gathering and decision-making. Based upon the findings of Fischhoff (1982), this training can be effective and will ideally translate to later entrepreneurial activities. The same can apply to entrepreneurial support groups such as incubators and accelerators and, in this case, preparing their mentors and advisors to be alert for these entrepreneurial biases can Journal of Small Business Strategy Vol. 24, No. 2 35 encourage deeper System 2 approaches. Even using different companies in the incubator as both a referent class for comparison as well as a friendly “check and balance” can take best advantage of the group setting. Several interesting research directions also emerge from this review. The prevalence of the Business Model Canvas and Lean Canvas methodologies could be studied to identify which subsets of questions in customer development are most relevant for product-market fit. Is there a specific order in which the customer development hypothesis testing model increases the chances of success, and, if so, can innovators and their partners (such as venture capital firms and business accelerators) create specific programs to take advantage of this? For example, is it more effective to talk to a group of potential customers in one field at a time, or to test several customer segments simultaneously with regard to bias reduction? In addition, the use of simple linear models with customer development and lean startup offers the framework to validate the evaluation methods. Just as the Apgar score, an improper linear model, was validated against subsequent data, testing the improper linear models inherent in the customer development process against a set of established outcome measures would provide another means of reducing bias from the process while still recognizing the intuitive side of the entrepreneurial process. Future research could focus on the development and testing of such improper linear models around a tool often used in the lean startup process such as the Business Model Canvas. REFERENCES Anderson, R. "Can stage-gate systems deliver the goods?" Financial Executive. (1993): 34. Bazerman, M. H. & Chugh, D. (2005). Bounded awareness: Focusing failures in negotiation. In L. Thompson (Ed.), Frontiers of Social Psychology: Negotiation. (pp.7-26) East Sussex, UK: Psychology Press. Bazerman, M. H., Loewenstein, G. F., & White, S. B. (1992). Reversals of preference in allocation decisions: Judging an alternative versus choosing among alternatives. Administrative Science Quarterly, 37(2), 220-240. Bazerman, M.H., White, S.B., & Loewenstein, G.F. (1995). Perceptions of Fairness in Interpersonal and Individual Choice Situations. Current Directions in Psychological Science, 4, 39-43. Bhandari, G., & Deaves, R. (2006). The demographics of overconfidence. The Journal of Behavioral Finance, 7 (1), 5- 11. Blank, S.G.(2013). Why the Lean Start-Up Changes Everything. Harvard Business Review, 91(5), 64-+. Blank, S. G., & Dorf, B. (2012). The Startup Owner's Manual: The Step-By- Step Guide For Building a Great Company. K&S Ranch, Incorporated. Blank, S.G. (2007). Four Steps to the Epiphany: Successful Strategies for Products that Win, Palo Alto, CA: Cafepress. Journal of Small Business Strategy Vol. 24, No. 2 36 Brentani, U., & Reid, S. E. (2012). The Fuzzy Front End of Discontinuous Innovation: Insights for Research and Management. Journal of Product Innovation Management, 29(1), 70-87. Busenitz, L. W., & Barney, J. B. (1997). Differences between entrepreneurs and managers in large organizations: Biases and heuristics in strategic decision- making. Journal of Business Venturing, 12(1), 9-30. Carter, B. S., McNabb, F., & Merenstein, G. B. (1998). Prospective validation of a scoring system for predicting neonatal morbidity after acute perinatal asphyxia. The Journal of Pediatrics, 132(4), 619-623. Cooper, A. C., Woo, C. Y., & Dunkelberg, W. C. (1988). Entrepreneurs' perceived chances for success. Journal of Business Venturing, 3(2), 97-108. Cooper, B., & Vlaskovits, P. (2013). The Lean Entrepreneur: How Visionaries Create Products, Innovate with New Ventures, and Disrupt Markets. Hoboken, NJ: John Wiley & Sons. Cooper, R. G. 1988. “The New Product Process: A Decision Guide for Management,” Journal of Marketing Management, 3(3): 238-255. Cooper, R. G. 2001. Winning at New Products: Accelerating the Process from Idea to Launch, 3rd edition, Cambridge, MA: Perseus. Cooper, R.G. 2011. Winning at New Products: Creating Value Through Innovation. 4th Edition, New York: Basic Books. Cooper, R.G. 2013 New Products: What Separates the Winners From the Losers and What Drives Success? In K. Kahn, (Ed.), PDMA Handbook of New Product Development 3rd Edition (pp.3-34), Somerset, NJ: Wiley. Cooper, R. G. 2013 (a). Invited Article: Where Are All the Breakthrough New Products?: Using Portfolio Management to Boost Innovation. Research-Technology Management, 56(5), 25-33. Cooper, R. G. (2014). Invited Article: What’s Next?: After Stage-Gate. Research-Technology Management, 57(1), 20-31. Croll, A., & Yoskovitz, B. (2013). Lean Analytics: Use Data to Build a Better Startup Faster. Sebastopol, CA: O'Reilly. Dawes, R. M. (1971). A case study of graduate admissions: Application of three principles of human decision making. American Psychologist, 26(2), 180-188. Dawes, R. M., & Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81(2), 95. DelVecchio, J., White, F., & Phelan, S., Tools for Innovation Management: A Comparison of Lean Startup and The Stage Gate System, USASBE Proceedings, 2014, In Press. Edgett, S. J. (2011). New Product Development: Process Benchmarks and Performance Metrics. Ancaster, Ontario, Canada: Stage-Gate International. Journal of Small Business Strategy Vol. 24, No. 2 37 Fischhoff, B. (1982). Debiasing. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment Under Uncertainty: Heuristics and Biases. Cambridge: Cambridge University Press: 422 – 444. Fischhoff, B., & Beyth-Marom, R. (1983). Hypothesis evaluation from a Bayesian perspective. Psychological review, 90(3), 239. Flyvbjerg, B. (2009). Survival of the unfittest: why the worst infrastructure gets built—and what we can do about it. Oxford Review of Economic Policy, 25(3), 344-367. Gigerenzer, G., Hoffrage, U., & Kleinbölting, H. (1991). Probabilistic mental models: a Brunswikian theory of confidence. Psychological review, 98(4), 506. Harper, D. A. (1999). How Entrepreneurs Learn: A Popperian Apprach and Its Limitations. Handelshøjskolen i København. Hastie, R., & Dawes, R. M. (Eds.). (2010). Rational Choice In An Uncertain World: The Psychology Of Judgment And Decision Making. Sage. Henard, D. H. and D. M. Szymanski. 2001. “Why Some New Products Are More Successful Than Others, Journal of Marketing Research ,” 38( 3) August: 362-375. Holcomb, T. R., Ireland, R. D., Holmes Jr, R. M., & Hitt, M. A. (2009). Architecture of entrepreneurial learning: exploring the link among heuristics, knowledge, and action. Entrepreneurship Theory and Practice, 33(1), 167-192. Idson, L. C., Chugh, D., Bereby-Meyer, Y., Moran, S., Grosskopf, B., & Bazerman, M. (2004) Overcoming focusing failures in competitive environments. Journal of Behavioral Decision Making, 17(3), 159-172. Kahneman, D. (2011). Thinking, Fast and Slow. New York: Macmillan. Kahneman, D., & Lovallo, D. (1993). Timid choices and bold forecasts: A cognitive perspective on risk and risk taking. Management Science, 39, 17-31. Karlsson, J., & Nordström, J. (2012). Implementing Lean Startup Methodology in Large Firms-Benefits, Challenges and Solutions. (Unpublished Thesis). Chalmers University of Technology, Gothenburg, Sweden. Karniouchina, E.V. 2011. “Are Virtual Markets Efficient Predictors of New Product Success? The Case of the Hollywood Stock Exchange, “ Journal of Product Innovation Management, 28(4): 470–484. Klein, G. (2007). Performing a Project Premortem. Harvard Business Review, 85(9), 18-19. Koellinger, P., Minniti, M., & Schade, C. (2007). “I think I can, I think I can”: Overconfidence and entrepreneurial behavior. Journal of Economic Psychology,28(4), 502-527. Koen, P., Ajamian, G., Burkart, R., Clamen, A., Davidson, J., D'Amore, R., ... & Wagner, K. (2001). Providing clarity Journal of Small Business Strategy Vol. 24, No. 2 38 and a common language to the" fuzzy front end". Research-Technology Management, 44(2), 46-55. Koen, P.A. G. Ajamian, S. Boyce, A.Clamen, E. Fisher, S. Fountoulakis, A. Johnson P. Puri, R. Seibert. 2002. “Fuzzy-Front End: Effective Methods, Tools and Techniques,” In: P. Belliveau, A Griffen and S. Sorermeyer (Eds.) PDMA Toolbook for New Product Development, (pp. 2-35), New York: Wiley. Koen, P.A. 2004. "Fuzzy Front End for Incremental, Platform and Breakthrough Products and Services", In K. Kahn, G. Castellion and A Griffen (Eds.) PDMA Handbook (pp. 81-91) New York: Wiley. Larrick, R. P. (2004). Debiasing. In D. J. Koehler & N. Harvey (Eds.), Blackwell Handbook of Judgment and Decision Making. Oxford, England: Blackwell Publishers. Lindley. D. V. (1985), Making Decisions, New York, Wiley. Lerner, J. S., & Tetlock, P. E. (1999). Accounting for the effects of accountability. Psychological Bulletin, 125(2), 255- 275. Maurya, A. (2012). Running Lean: Iterate from Plan A to a Plan That Works. O'Reilly Media. Minniti, Zacharakis, Spinelli, Rice, & Habbershon, Entrepreneurship: The Engine of Growth, Greenwood Publishing Group, 2007. Moster, D., Lie, R. T., Irgens, L. M., Bjerkedal, T., & Markestad, T. (2001). The association of Apgar score with subsequent death and cerebral palsy: a population-based study in term infants. The Journal of pediatrics, 138(6), 798- 803. Mullins, J.W. (2013). The New Business Road Test: What Entrepreneurs and Executives Should Do Before Launching a Lean Start-Up. London: Pearson Financial Times. Mussweiler, T., Strack, F., & Pfeiffer, T. (2000). Overcoming the inevitable anchoring effect: Considering the opposite compensates for selective accessibility. Personality and Social Psychology Bulletin, 26, 1142- 1150. Osterwalder, A., & Pigneur, Y. (2010). Business model generation: a handbook for visionaries, game changers, and challengers. Hoboken, NJ: Wiley. Ries, E. 2011. The Lean Startup, New York: Crown Publishing Group. Schade, C., & Koellinger, P. (2007). Heuristics, biases, and the behavior of entrepreneurs. In M. Minniti (Ed.) Entrepreneurship, the Engine of Growth, Vol. 1, (pp. 141-163) Westport, CT: Praeger. Silva, M. B., & Nascimento, M. A. Lean Startup Applied In Healthcare: A Viable Methodology for Continuous Improvement in the Development of New Products and Services. Retrieved from http://www.cti.gov.br/mcpl2013/papers _MMCP13/0054/0054_FI.pdf. Slater, S. F., Mohr, J. J., & Sengupta, S. (2013). Radical Product Innovation Capability: Literature Review, Synthesis, and Illustrative Research Journal of Small Business Strategy Vol. 24, No. 2 39 Propositions. Journal of Product Innovation Management, 31, (3). Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate. Behavioral & Brain Sciences, 23, 645- 665. Stanovich, Keith (2010), Rationality and the Reflective Mind, Oxford University Press, USA. Thaler, R. H., & Sunstein, C. R. (2008). Nudge. New Haven: Yale University Press. Thompson, L., Gentner, D., & Loewenstein, J. (2000). Avoiding Missed Opportunities in Managerial Life: Analogical Training More Powerful Than Individual Case Training. Organizational Behavior & Human Decision Processes, 82(1), 60-75. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124- 1131. Jonathan L. York, Ph.D. is an associate professor of entrepreneurship in the Orfalea College of Business at Cal Poly. Dr. York is also the Founding Faculty Director of the Cal Poly Center for Innovation & Entrepreneurship, and is a former serial entrepreneur whose research interests are now centered in entrepreneurial search processes and entrepreneurial cognition. He has a Bachelor's Degree from Yale University and a Ph.D. from Michigan State. Jeffrey E. Danes, Ph.D. Michigan State University, is professor of Marketing Analytics at Cal Poly in the Orfalea College of Business and works at the intersection of psychology, probability, and behavioral economics. His expertise is judgment under uncertainty. He seeks to solve problems in product pricing, branding, new product development, and data-driven decision making. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.