Mutual fund knowledge assessment for policy and decision problems Brian Scholl1, Angela Fontes2,* 1Office of the Investor Advocate, 100 F Street NE, Washington, DC 20549, USA 2NORC at the University of Chicago, 55 East Monroe, Chicago, IL 60602, USA Abstract We develop a measure of mutual fund investment knowledge that complements existing fi- nancial literacy measures. Our question battery was administered to 3,444 survey respondents. We validate the index with factor analysis identifying two latent components, and descriptive regressions demonstrating the additive value of our index beyond general financial literacy in explaining variation in financial well-being, investment ownership, and fee calculation profi- ciency. Despite mutual funds’ importance in household savings, our index suggests that the public lacks adequate understanding of them. We demonstrate the utility of our index for studying selected decision and policy problems. © 2022 Academy of Financial Services. All rights reserved. JEL classification: G53 (Household Finance: Financial Literacy); G11 (Portfolio Choice; Investment Decisions) Keywords: Financial literacy; mutual funds; Investor decision-making; Measurement; Human capital; Knowledge; Exchange-traded funds (ETFs) The Securities and Exchange Commission disclaims responsibility for any private publication or state- ment of any Commission employee or Commissioner. The Commission has expressed no view regard- ing the data, analyses, findings, and conclusions contained in this publication, which does not necessarily reflect the views of the Commission, the Commissioners, or staff of the Commission. This paper is part of a series of papers that the Office of the Investor Advocate is publishing. Papers in this series investigate a broad range of issues relevant to the Commission’s mission and are disseminated to stimulate discussion and critical comment. Inclusion of a paper in this series does not indicate a Commission determination to take any particular action. *Corresponding author. Tel.: +1-608-630-1373. E-mail address: Angela@FontesResearch.com 1057-0810/22/$ – see front matter © 2022 Academy of Financial Services. All rights reserved. Financial Services Review 30 (2022) 31–56 1. Introduction Mutual funds are extremely important to household investment portfolios, potentially pro- viding well-diversified investment management options for most investors, and serving as the main investment vehicle in retirement and educational savings accounts.1 With the tran- sition to defined contribution retirement plans well underway, nearly 80% of investors hold employer-sponsored retirement plans, putting the onus of investment management largely on individuals themselves. The majority of these individuals report ownership of mutual funds and exchange traded funds (ETFs)—with more investors reporting ownership of funds than stocks and bonds combined; some estimates suggest that the number of individual mu- tual fund shareholders was 99.5 million in 2018 (ICI, 2010, 2018).2 Although mutual funds are often targeted towards individual investors, their complexity may be an obstacle that inhibits investor choice. To better understand the precise knowledge gaps that may lead to potential investor decision pitfalls, we develop an easily deployable and re- spondent-friendly battery of 11 true-false technical mutual fund questions that allows us to mea- sure respondent knowledge of key concepts important to investment decision making. Our battery is the result of extensive input from extant literature, financial regulators, a close reading of the mandated disclosure documents intended to communicate important product features to investors, substantial cognitive interviewing, multiple rounds of testing, and considerable data analysis following standard practice in index development (in particular, see DeVellis, 2016). Administration is comprised of a set of progressively more complicated technical questions that seek to test knowledge of basic concepts such as risk, fees, regulatory protections, marketplace characteristics and performance. We field the battery on a large, nationally representative, prob- ability-based panel, allowing us to provide a broad perspective on U.S. households. To summa- rize respondent knowledge, we use these individual questions to construct a simple index that well-encapsulates the level of technical mutual fund knowledge of the investing public. We believe our index is highly useful in the context of regulatory policy and academic work on investor decision making. Policymakers have an interest in making sure mutual fund products work as intended so that investors achieve their investment goals, with inves- tor protection one of the main goals of the U.S. Securities and Exchange Commission (SEC), the body that regulates mutual funds.3 Regulators operationalize policies that advance this goal with rules limiting or prohibiting certain activity and (perhaps most impor- tantly) by mandating disclosures that provide transparency about decision-relevant features such as fees and risks, which can facilitate informed investor decision making.3 The interre- lated nature of financial knowledge and disclosure is highlighted by U. S. Securities & Exchange Commission (2012), a SEC staff report on financial literacy mandated by Congress as part of the Dodd-Frank Act; the report notes troubling disparities in knowledge, particularly in selected demographic groups, but also responds at length to specific Congressional mandates to probe into highly related issues in disclosure. In a disclosure-centric regulatory regime, the burden of choosing investments ultimately falls on the investor. Disclosure of information about a particular product does not mean that investors can understand the concepts, understand their implications or make appropriate investment choices. Such conceptual gaps may forestall potential investors from investing 32 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 leaving them unprepared for important goals such as college or retirement, but for those that do invest, a knowledge gap related to important technical specifics may result in choices that have real consequences for a household’s balance sheet. For example, an investor that fol- lows conventional wisdom by investing $100,000 in one of the lowest priced S&P index funds (“Fund A”) with returns averaging 7% per year would pay approximately $1,300 in fees over 20 years, and would have a final balance of approximately $385,000. But an inves- tor that makes a single mistake in their purchase decision by selecting one of the higher priced S&P index funds (“Fund B”)—a mistake that could arise by simply picking the more expensive option from a broker’s menu of funds—would pay approximately $80,000 in fees over the same period and end up with less than $240,000. The consequences of fees are well known to professionals, policymakers, and academics, but it is important to ask: how widely known is this to the public who may not be as experienced or knowledgeable?5 Moreover, what other misconceptions do they hold about mutual funds that could lead to other costly mistakes? Identifying these deficiencies may help in developing more targeted interventions to complement broad financial education initiatives. In the context of investor decision making, our index complements existing measures of general financial knowledge that have been associated with a broad range of economic out- comes. A large body of evidence has elevated attention to financial knowledge and its impli- cations for various financial outcomes (Lusardi, 2019, 2012, 2008; Lusardi & Mitchell, 2014; Lusardi, Mitchell, & Curto, 2014; van Rooij, Lusardi, & Alessie, 2011), such as wealth accumu- lation, debt management, general financial management, and uptake of financial advice (Scholl & Hung, 2018). Typical measures of financial literacy attempt to glean knowledge of general economic principles, which may make them helpful for understanding such concerns as overall well-being and asset accumulation, but these measures may be insufficiently specialized for the policy-making context of financial market regulators. For example, the useful and widely accepted three-question financial literacy questions advanced by Lusardi and Mitchell (Lusardi & Mitchell, 2014, 2008) tests literacy in terms of questions on purchasing power and inflation, risk, and interest compounding. In extended versions of this standard battery (Lusardi, 2008; Lin et al., 2019; van Rooij, Lusardi, & Alessie, 2011), further developments and refinements of these fundamental measurement concepts are made to examine the knowledge of increasingly com- plex principles of economics, finance, and investment. Yet, in the $145,000 investment mistake illustrated above, the knowledge gap that leads to this investor decision-making error relates to technical features of funds and the market for funds, rather than general economic knowledge. Leading up to this “asset selection deci- sion” between Fund A and B based on an evaluation of the characteristics of the funds, an individual presumably has previously made the decision to invest in securities (“participa- tion decision”) and determined that a particular asset class such as mutual funds are right for them (“asset class selection decision”). Of the Lusardi-Mitchell Big 5 general financial liter- acy questions, the most directly relevant to the context of mutual fund investments is a true- false question that asks: “buying a single company’s stock usually provides a safer return than a stock mutual fund?” This question seems directly related to the asset class selection decision and perhaps the participation decision, but less directly related to the asset selection decision in which the investor evaluates the merits of features of funds. To make a reasona- ble asset selection choice, the investor may need to be aware of facts such as: that B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 33 differences in fees exist; that the expense ratio does not necessarily reflect all fees such as commissions or loads; that fees compound over time much like interest; and where to find information about fees. Moreover, the investor is often only presented with a single fund, and may need to evaluate such a fund in reference to an unknown set of alternatives; if they are presented with Fund B, they may need to engage in costly search to identify a better al- ternative (as in Hortaçsu & Syverson, 2004). A lack of knowledge about the potential to find alternatives may forestall search, possibly because they do not know that cheaper, nearly identical alternatives exist. For securities regulators, the asset selection decision is arguably the most relevant to policy levers given that disclosures tend to contain information relevant to investment selection and management decisions rather than participation decisions. These concerns motivated us to develop an index that more specifically focuses on technical knowledge geared towards asset selection decisions that are most relevant in the policy and research contexts we study. Technical knowledge of investment products would seem a prereq- uisite to good investor decision making—after all, if an investor does not understand fees, where they may be hidden, or how to identify them in the disclosures, they ultimately may not even know the choice dimensions on which they should be optimizing. The focus on technical knowl- edge recognizes that disclosures are typically written by securities attorneys with specialized knowledge of the subject matter, and are often intended for multiple audiences: from mutual fund experts to ordinary investors with limited investment experience. Over time, much criti- cism has been levied at disclosures that are not informative for retail investors.6 Our broad view is that general financial literacy may be informative about respondent inclusion of mutual funds in their portfolios, but believe our measure of knowledge may pro- vide additional insight on the respondent’s ability to distinguish between products in the marketplace. It is households’ propensity for making technical mistakes in mutual fund choice decisions that we attempt to assess in our measure of knowledge; arguably, such pro- pensities should be of interest to policymakers and researchers interested in promoting better investor decision making. Ultimately, the relative value of any measure of knowledge in a particular context is an empirical question that we study in this article; here, we provide extensive evidence that demonstrates the utility of our knowledge index. Regrettably, the picture we paint about mutual fund knowledge in the population is some- what bleak. Overall knowledge scores are quite low with many respondents performing far worse than if they had guessed randomly in responding to the questions. These results are disheartening: most of the concepts that form a potential basis for informed decision making, and have been mandated for inclusion in disclosures, are not broadly understood. Our find- ings call into question the sufficiency of existing disclosures as a vehicle for providing deci- sion-relevant information to investors, and also raise questions about the adequacy of relying on disclosure alone as such a prominent method of promoting investor protection. The results also challenge the notion that financial education on its own has been sufficient to equip most individuals with the knowledge and skills needed to successfully manage their own investments: either that education has been insufficiently widespread, or its efficacy may be limited; whatever the reason, we do not find a sufficient level of pass-through from education to decision-relevant knowledge at the population-level. These results are particu- larly troubling especially in light of the importance of mutual funds to education and savings (Scholl & Hung, 2018). Moreover, while fees are widely considered among the most 34 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 important and controllable feature on which to make choices, see, for example, (Barber et al., 2005; Choi et al., 2010; Carhart, 1997; SEC Office of Investor Education and Advocacy 2016), virtually every subgroup we have examined has demonstrated extremely poor performance on fee questions.6 Our analysis demonstrates the potential of our knowledge index to serve as a measurement tool to explain a host of household financial concerns including investment partici- pation (i.e., ownership of investment accounts and/or financial securities), financial well-being, and fee computation skill. Other emergent work has also demonstrated the utility of the measure in other contexts (e.g., Chin, Scholl, & VanEpps, 2021; Scholl, 2020). This article proceeds as follows: Section 2 provides a comprehensive review of relevant literature; Section 3 describes the individual items and methodology; Section 4 provides a synopsis of item and cumulative results; Section 5 discusses index development and valida- tion; Section 6 concludes. 2. Literature Our article speaks to the literature on financial literacy, and to a lesser extent financial education, policy work on the role of financial capability and education in promoting better investment outcomes, and broader literature on financial decision making. Financial literacy has been associated with a wide array of economic outcomes such as debt management, wealth accumulation, financial vulnerability, and a host of other economic outcomes in the United States and other countries (Lusardi, 2008, 2012, 2019; Lusardi & Mitchell, 2014; Lusardi & Tufano, 2009, 2015; Lusardi, Mitchell, & Curto, 2014; van Rooij, Lusardi, & Alessie, 2011). In these studies, literacy is measured by the number of correct responses to a set of survey questions, with by far the most widely accepted being the standard set devel- oped as the Lusardi-Mitchell “e” or the “Big 5” (Lusardi & Mitchell, 2014, 2011; Lusardi & Mitchell, 2008). Other work has extended these standard questions (e.g., Lin et al., 2019; Lusardi, 2008), while other approaches to measurement of financial literacy are surveyed in Elan (2011). These studies overwhelmingly find that higher levels of financial literacy are associated with more favorable financial and economic outcomes (while also providing im- portant tools for measurement of knowledge within the population). Rather than using sur- vey measures, Calvet, Campbell, and Sodini (2007) measure the related concept of financial sophistication, backing out an index of sophistication from identifiable mistakes in observed household portfolio choices using Swedish administrative data, although that approach requires data that is largely unavailable for most populations. Meta-analyses that have altogether examined hundreds of studies on the topic have dem- onstrated mixed conclusions as to the importance of financial literacy and financial educa- tion programs (Fernandes et al., 2014; Kaiser & Menkhoff, 2017, 2020). The topics remain of sustained interest, with a recent special issue of the Economics of Education Review pro- viding a wealth of articles examining related issues in financial literacy and education (including Davoli & Rodrı́guez-Planas, 2020; Kaiser & Menkhoff, 2020; Lusardi et al., 2020; Urban et al., 2020). Hastings, Madrian, and Skimmyhorn (2013) outline a number of issues and raise a number of outstanding questions related to financial literacy, including the B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 35 very goals related to research and education. Beshears et al. (2018) provide an extensive review of this literature in behavioral household finance. They conclude that while the litera- ture has demonstrated the potential effectiveness of financial education and information interventions, they express some skepticism in relation to cost-effectiveness. While these are extensive contributions to the debate on topics of financial education, it is important to note that we do not take a perspective in this article on financial education per se nor its efficacy: we view our article as focusing on measurement that can help explain behavior in certain de- cision and policy contexts. While mutual fund products are extremely important to household investment portfolios (Scholl & Hung, 2018), specific knowledge of mutual fund features has apparently received far less attention than overall financial literacy. Muller and Turner (2021) examine knowl- edge in the context of the “high-fee puzzle,” or the selection of strictly dominated funds. They find that while three quarters of their sample correctly answer common financial liter- acy questions, only a third could correctly answer questions related to quantifying fees. Kahraman (2021) examines investor mistakes in the context of purchasing inappropriate (and more expensive than necessary) share classes for mutual funds, leading to real conse- quences for investors. The author presents evidence to suggest that the selection of these in- ferior share classes is a form of exploitation of investors by professionals. In addition, the author examines fee-flow sensitivity and holding periods to test whether fund flows suggest rational or naı̈ve purchase of these funds, concluding that these are naı̈ve purchases. Other recent work has examined additional barriers to investment decision making, in the context of the language used to describe funds. Chin, Scholl, and VanEpps (2021) study lin- guistic barriers to understanding mutual fund fees. They conduct two studies that suggest that terminology often used is unintuitive for respondents, and that a simple set of alternative terms they test leads respondents to higher rates of identifying the true underlying fee con- cept. Scholl, Silverman, and Enriquez (2020, 2021), examine prospectus readability and other textual features using natural language processing and machine learning techniques, and relate these features to ex-post fund performance. Readability calculated using structural features of prospectus sentences; this concept is distinct from comprehension of the underly- ing concepts, which could require expert knowledge. One descriptive fact that they docu- ment is that readability of fund disclosures is extremely low. Less than one percentage of fund summary prospectus documents are as readable as a U.S.A. Today article, while roughly three-quarters are less readable than the U.S. tax code. The majority of these docu- ments are at college reading levels and above. deHaan et al. (2021) provide evidence that highlights the potential for intentional obfuscation in terms of narrative complexity and the structural complexity of the securities instrument. The only study of which we are aware that specifically attempts to profile specific knowl- edge of mutual funds products is Alexander, Jones, and Nigro (1998) that was conducted during a very different investing environment. That article focuses on the sources of infor- mation that investors use and the differences in knowledge based on the purchase channels investors use. The authors concluded based on survey results conducted a quarter of a cen- tury ago that there is much room for improvement in investor’s knowledge levels; they also survey an earlier literature that documents some common misconceptions of investors such as that mutual funds sold through a bank are backed by FDIC insurance. Our work builds off 36 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 some of the key features identified in Alexander, Jones, and Nigro (1998), while delving into a broader set of mutual fund features, and formalizing an index. Moreover, our knowl- edge index design and our survey methodology are quite distinct in several important dimen- sions. Our sample is a general population sample, which includes both individuals residing in mutual fund owning households as well as those that reside in households that do not own mutual funds that were the focus of Alexander et al. (1998).8 This sample distinction allows for us to document initial observations related to our interest in participation in the market (although a more detailed study of participation barriers in connection to knowledge deserves a separate treatment). Our intention is that our work is informative in the context of literature on investor choice and decision making, and lays the groundwork for further decomposing aspects of decision making. A relatively large body of literature has emerged along these lines in recent decades relating largely to deviations from rationality or a lack of information by investors (e.g., see observational studies by Barber et al. [2005], Elton et al. [2011]; as well as work using behav- ioral experiments such as Beshears et al. [2011], Choi et al. [2010]; and also the extensive review in Beshears et al. [2011]). In a related study, Müller and Weber (2010) construct a finan- cial literacy measure, and look at participation and choice decisions in selected mutual fund markets. They note there is a positive relationship between their literacy measure and the likeli- hood of investing in active funds (that they argue are worse than passive management alterna- tives); nevertheless, despite the fact that literacy matters, it alone cannot explain the historical growth of active management. Overall, they find only weak evidence that investors that per- formed well on their measure had superior fund selection skills. Scholl (2020) examines deci- sion making in the presence of choice set complexity in a large-scale experiment in which subjects complete an allocation problem from a menu of S&P 500 index mutual funds. That study directly uses an early version of the mutual fund knowledge index we examine here. Scores on our index demonstrated strong separation and well ordering of subjects in terms of their overall performance on the investment task (where fee minimization is a strictly dominat- ing strategy). Other forthcoming behavioral research using allocation experiments by Scholl and coauthors also demonstrate the utility of our mutual fund knowledge index in other contexts, such as in the classification of investor types described in Chin, Scholl, and VanEpps (2021). The importance of the knowledge we test is highlighted in Bhattacharya et al. (2017), who decomposed investor decision mistakes with respect to ETF investments into “poor timing” and “poor selection”—the latter accounting for 1.69% loss per annum in investor returns; our index directly tests knowledge that could plausibly help investors avoid such selection mistakes. 3. Method 3.1. Data collection Data from 3,444 respondents were collected over three waves of data collection (over a 10-month period) using the AmeriSpeak Panel (AS), a probability-based, nationally repre- sentative U.S. panel.9 Respondents were incentivized for participation in each wave of the B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 37 T ab le 1 D em o g ra p h ic ch ar ac te ri st ic s, b y g ro u p D em o g ra p h ic ch ar ac te ri st ic s, b y g ro u p P an el A : T o ta l P an el B : G en er al fi n an ci al li te ra cy P an el C : V al id at io n v ar ia b le b re ak o u ts In d ex it em P o p to ta l L o w li te ra cy H ig h li te ra cy In v es to rs C al cu la te d fe e q u es ti o n co rr ec tl y F W B Q 3 + G en d er (m al e) 5 5 .7 5 % 3 3 .7 8 % 6 4 .4 2 % 5 6 .2 5 % 6 6 .0 8 % 5 8 .5 4 % A g e 1 8 -2 9 4 .2 7 % 6 .7 6 % 4 .0 0 % 4 .2 0 % 4 .2 4 % 3 .3 3 % 3 0 -4 4 2 5 .2 6 % 3 5 .1 4 % 2 2 .2 3 % 2 5 .4 9 % 2 7 .1 8 % 1 1 .9 2 % 4 5 -5 9 2 9 .3 8 % 2 8 .3 8 % 2 8 .7 9 % 2 9 .2 9 % 2 7 .4 3 % 2 0 .6 2 % 6 0 + 4 1 .0 9 % 2 9 .7 3 % 4 4 .9 8 % 4 1 .0 2 % 4 1 .1 5 % 6 4 .1 2 % R ac e/ et h n ic it y W h it e n o n -H is p an ic 8 3 .4 2 % 6 3 .5 1 % 8 5 .4 0 % 8 3 .8 7 % 8 3 .4 2 % 8 8 .4 0 % A fr ic an A m er ic an n o n -H is p an ic 3 .7 5 % 1 2 .1 6 % 2 .7 0 % 3 .7 3 % 3 .3 7 % 2 .9 0 % H is p an ic 4 .7 6 % 1 2 .1 6 % 3 .8 6 % 4 .6 3 % 3 .2 4 % 3 .1 1 % O th er 8 .0 7 % 1 2 .1 6 % 8 .0 5 % 7 .7 6 % 9 .9 8 % 5 .5 9 % E d u ca ti o n N o h ig h sc h o o l d ip lo m a 1 .1 0 % 5 .4 1 % 0 .3 3 % 0 .7 7 % 0 .5 0 % 0 .4 3 % H ig h sc h o o l g ra d u at e o r eq u iv al en t 7 .9 3 % 1 7 .5 7 % 4 .3 3 % 6 .5 6 % 3 .9 9 % 4 .9 4 % S o m e co ll eg e 2 7 .1 8 % 4 4 .5 9 % 2 2 .4 2 % 2 5 .4 2 % 1 9 .8 3 % 2 1 .9 1 % B A o r ab o v e 6 3 .7 9 % 3 2 .4 3 % 7 2 .9 3 % 6 7 .2 4 % 7 5 .6 9 % 7 2 .7 2 % In co m e L es s th an $ 3 5 ,0 0 0 1 6 .0 6 % 3 3 .7 8 % 1 1 .1 2 % 1 1 .2 0 % 1 0 .3 5 % 5 .8 0 % $ 3 5 0 0 0 -$ 5 9 ,9 9 9 1 7 .5 7 % 2 2 .9 7 % 1 6 .1 9 % 1 6 .8 3 % 1 4 .4 6 % 1 1 .2 8 % $ 6 0 ,0 0 0 -$ 9 9 ,9 9 9 2 9 .8 8 % 2 0 .2 7 % 3 0 .5 6 % 3 1 .5 9 % 2 9 .5 5 % 3 2 .6 5 % $ 1 0 0 ,0 0 0 o r m o re 3 6 .5 0 % 2 2 .9 7 % 4 2 .1 4 % 4 0 .3 9 % 4 5 .6 4 % 5 0 .2 7 % N et w o rt h In d eb t 1 2 .2 2 % 2 2 .9 7 % 8 .8 8 % 9 .8 0 % 8 .1 0 % 2 .5 8 % Z er o 9 .7 0 % 2 9 .7 3 % 6 .3 7 % 8 .2 3 % 5 .6 1 % 2 .2 6 % G re at er th an ze ro 7 8 .0 8 % 4 7 .3 0 % 8 4 .7 4 % 8 1 .9 7 % 8 6 .2 8 % 9 5 .1 7 % F ee ca lc u la ti o n (C o rr ec t) 2 3 .2 9 % 1 3 .5 1 % 2 9 .7 2 % 2 4 .8 3 % 1 0 0 .0 0 % 2 7 .3 9 % F in an ci al W el l- b ei n g (Q 3 + ) 2 7 .0 3 % 1 4 .8 6 % 3 1 .9 1 % 2 9 .5 2 % 3 1 .8 0 % 1 0 0 .0 0 % F in an ci al li te ra cy sc o re (M ) 2 .4 8 2 .5 3 3 .0 0 2 .5 3 2 .7 3 2 .6 6 N 3 ,4 4 4 7 4 2 ,1 5 0 3 ,0 0 1 8 0 2 9 3 1 N o te s: “L o w li te ra cy ” g ro u p an sw er ed n o L u sa rd i- M it ch el l B ig 3 q u es ti o n s co rr ec tl y , “h ig h li te ra cy ” g ro u p an sw er ed th re e q u es ti o n s co rr ec tl y . F W B Q 3 + ar e re sp o n d en ts th at w er e in th e th ir d an d fo u rt h q u ar ti le o f F W B . F W B = C F P B ’s F in an ci al W el l- B ei n g . 38 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 study via points redeemable for consumer goods, gift card, or cash of approximately $10. Average survey length for each wave was 21-26 min, although our knowledge index only required a few minutes to complete. Several aspects of the AS panel contribute to data qual- ity when investigating household finances. Panelists are recruited with an enhanced address- based sampling frame that increases coverage of U.S. households to over 97%, increasing rural enumeration. Internet-phone mixed-mode data collection accommodates households without internet access.10 Table 1 presents the demographics of the sample as a whole, and by subcategories used in index validation. By intention, we oversampled likely mutual fund investors in our initial recruitment; as such, the sample reported here is older and more edu- cated than the U.S. population.11 3.2. Mutual fund literacy item identification We developed a set of true/false items based on a careful review of existing work in the areas of general financial literacy (e.g., Lusardi, 2008; Lusardi & Mitchell, 2014), investment literacy (e.g., Forbes & Kara, 2010), and mutual fund investment knowledge (specifically, Alexander, Jones, & Nigro, 1998). Our question areas were designed to test respondent technical knowledge on key features of mutual funds. More specifically, they were designed to elicit technical knowl- edge of key choice features (most notably, risks and fees) that we identified as helpful to an in- vestor in the context of mutual fund selection problems. We believe that individuals lacking knowledge of these attributes would be impaired in investment decision-making situations. We favored this technical knowledge approach over a more generalized set of economic concepts that are typically found in financial literacy batteries. Once identified, items were then refined with extensive expert input from individuals with highly specialized knowledge and experience regarding regulation, financial education, in- vestor advocacy, in the context of both the technical features of the funds and the regulatory tools that regulators apply to mutual fund products. Some of these individuals were inti- mately familiar with financial literacy issues and programs as they are viewed and imple- mented by regulators, while others had experience in disclosure review and/or the writing of disclosure rules and regulations. An investor advocate entity that promotes pro-investor poli- cies with regulators also provided an important perspective. We refined the questions so that they would identify and elicit knowledge gaps that would inhibit investors from utilizing the information contained in disclosures for decision making. We view the link to mutual fund disclosures, the primary method of information exchange on investment options, as extremely important. Our article focuses on technical knowledge that relates to feature concepts or applications that may inhibit investors from making use of disclosed information for informed decision making. For example, for an investor to pick a mutual fund that avoids a load fee: the individual may need to be aware that a sales charge exists; what services the load pays for and what is the typical range in such charges; know that it must be disclosed in disclosures; understand that the appropriate term is “load”; be able to locate it in disclosures; potentially evaluate the fee as part of a tradeoff vis-à-vis other fees; understand that no-load mutual fund investments exist; distinguish it from alternative “sales” charges (e.g., broker commissions)—that is, know that commissions are not the only B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 39 sales charge; and potentially to understand that loads can potentially be applied at both the time of purchase and the time of sale. Without these elements of knowledge—clearly linked to disclosed information and the investor’s interaction with disclosures—the investor may not be able to make an optimal choice. The overriding importance of this linkage between knowledge or literacy and disclosures has not received much attention in the academic liter- ature, but is highlighted by a 2012 SEC Staff report on financial literacy (U.S. Securities and Exchange Commission, 2012), which was undertaken pursuant to a mandate in the Dodd- Frank Act by Congress. Qualitative research was used extensively during the development process. An initial round of 19 interviews provided early insights that helped us to identify broad deficiencies in misconceptions. These interviews were centered around completion of a specific mutual fund choice task, and revealed large deficiencies in knowledge for certain participants. Later, once themes and the initial structure of question items were identified, we tested them with extensive cognitive interviews in subsequent rounds with a total of 23 participants drawn from a nationally representative probability sample. Such testing helped to refine the phrasing of questions and fielding protocol of the questions (in particular, the need for ran- domization of placement on a survey instrument). A larger battery of questions was origi- nally considered, including both multiple choice questions and true-false questions on technical features of funds. Additional questions increased difficulty of the overall assess- ment considerably, and appeared to make interviewees—particularly those with less experi- ence in investment—more reluctant to venture answers to the questions at all. Moreover, While these questions added further richness to our perspectives of respondents, in the end, we determined that a focused battery centering on the true-false questions provided suffi- cient comfort to the interviewees, sufficient well-ordering in terms of sophistication, and that we had a sufficient number of questions to distinguish between respondents based on sophistication. The interviews also provided insight into dealing with incorrect answers ver- sus question skips.12 One particular debate that cognitive testing sparked among members of the research team was related to the use of terminology. Our testing revealed that some indi- viduals were so unfamiliar with mutual funds and ETFs that they did not understand the ter- minology used in our questions such as “loads” or the “expense ratio.” While it can be argued that this unfamiliarity with the terminology creates barriers to answering the ques- tions (as per Chin, Scholl, & VanEpps, 2021), these barriers directly mimic barriers that individuals would face in seeking and choosing among mutual fund investments and are related to the specific knowledge each question is testing. After all, in reading investment disclosures, investors have to do their own translation of technical language. As such, we deemed that preserving such terminology was important in the context of assessing a respondent’s overall knowledge. Naturally, it is possible that individuals that have engaged with the market in terms of key investment decisions in the past will have been more moti- vated to understand terminology before making a decision. Yet, this learning process would consequentially be captured in our stock measurements of overall investor knowledge in the population—and would analytically present itself in the difference in knowledge levels between investors and non-investors. In addition to these steps in development, we have utilized the battery in experiment and test- ing studies with thousands of participants (e.g., Scholl, 2020). The battery has overall been 40 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 effective at rank ordering investor sophistication, as well as the propensity of individuals to ex- hibit a number of decision making and comprehension mistakes in investment settings (e.g., naı̈ve diversification, susceptibility to complexity, and failures to avoid fees). Our final battery identified 11 items in four key areas: market alternatives, risk, performance history, and fees. Questions were true/false, with a “don’t know” response option. These questions (correct answers in parentheses) are as follows, with labels assigned to each question for ease of reference: Marketplace alternatives category: 1. Financial markets offer thousands of different mutual funds to investors. (TRUE) (Label: market options) Risk category: 2. Mutual funds pay a guaranteed rate of return. (FALSE) (“guaranteed return”) 3. It is possible to lose money in a stock mutual fund. (TRUE) (“risk stockfund”) 4. It is possible to lose money in a bond mutual fund. (TRUE) (“risk bondfund”) 5. If a mutual fund is registered with the Securities and Exchange Commission (the SEC) or state securities regulators, you cannot lose money. (FALSE) (“risk regulation”)13 Performance history category: 6. A good predictor of the future performance of a mutual fund is its past performance. (FALSE) (“performance history”)Fee category: 7. A no-load mutual fund charges yearly expenses. (TRUE) (“yearly expenses”) 8. A load fee is charged only when the fund is initially purchased. (FALSE) (“load”) 9. Fees and expenses for the mutual fund industry are capped at a maximum level by reg- ulatory authorities. (FALSE) (“fee cap”) 10. Fund fees are required to be reported in the fund’s prospectus document. (TRUE) (“prospectus fees”) 11. The fees or expenses charged by the mutual fund company in a given year can be approximated by multiplying the fund’s net expense ratio by the investment gains for the year. (FALSE) (“fee basis”) In addition to these four primary categories, questions risk regulation and fee cap implic- itly ask respondents for assumptions about regulatory protections. Question prospectus fees connects to disclosure requirements (and the respondent’s familiarity with the prospectus document from which much of a fund’s decision-relevant information can be gleaned). Question performance history also has a direct link to the standard disclosure, sometimes referred to as the mutual fund warning label that is required by the Securities and Exchange Commission on certain performance presentations. To minimize order effects, we randomized the presentation order of questions on the sur- vey. As much as possible, we endeavored to develop questions that had objectively correct and incorrect answers rather than ones that might be considered situationally dependent. Although the questions have varying degrees of difficulty, few, if any of the questions, can be considered “trick questions.” B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 41 T ab le 2 It em le v el re su lt s, b y g ro u p P er ce n ta g e co rr ec t, b y g ro u p P an el A : T o ta l P an el B : G en er al fi n an ci al li te ra cy P an el C : In v es to r st at u s P o p to ta l S E L o w li te ra cy S E H ig h li te ra cy S E In v es to r S E N o n -i n v es to r S E F in an ci al m ar k et s o ff er th o u sa n d s o f d if fe re n t m u tu al fu n d s to in v es to rs . 7 1 .0 5 0 .7 7 5 6 .7 6 5 .8 0 8 0 .6 0 0 .8 5 7 4 .1 1 0 .8 0 5 0 .3 4 2 .3 8 M u tu al fu n d s p ay a g u ar an te ed ra te o f re tu rn . 6 4 .2 3 0 .8 2 2 1 .6 2 4 .8 2 7 9 .1 6 0 .8 8 6 8 .3 4 0 .8 5 3 6 .3 4 2 .2 9 It is p o ss ib le to lo se m o n ey in a st o ck m u tu al fu n d . 8 1 .2 7 0 .6 6 6 0 .8 1 5 .7 1 8 9 .0 7 0 .6 7 8 3 .5 1 0 .6 8 6 6 .1 4 2 .2 5 It is p o ss ib le to lo se m o n ey in a b o n d m u tu al fu n d . 5 5 .1 1 0 .8 5 3 6 .4 9 5 .6 3 6 4 .2 8 1 .0 3 5 7 .7 1 0 .9 0 3 7 .4 7 2 .3 0 If a m u tu al fu n d is re g is te re d w it h th e S E C o r st at e se cu ri ti es re g u la to rs , y o u ca n n o t lo se m o n ey . 6 7 .7 1 0 .8 0 2 7 .0 3 5 .2 0 8 1 .0 2 0 .8 5 7 0 .8 4 0 .8 3 4 6 .5 0 2 .3 7 A g o o d p re d ic to r o f th e fu tu re p er fo rm an ce o f a m u tu al fu n d is it s p as t p er fo rm an ce . 3 2 .9 3 0 .8 0 1 7 .5 7 4 .4 5 4 0 .4 2 1 .0 6 3 4 .9 6 0 .8 7 1 9 .1 9 1 .8 7 A n o -l o ad m u tu al fu n d ch ar g es y ea rl y ex p en se s. 2 4 .2 2 0 .7 3 1 6 .2 2 4 .3 1 3 0 .4 2 0 .9 9 2 5 .7 6 0 .8 0 1 3 .7 7 1 .6 4 A lo ad fe e is ch ar g ed o n ly w h en th e fu n d is in it ia ll y p u rc h as ed . 1 6 .4 6 0 .6 3 5 .4 1 2 .6 5 2 0 .9 8 0 .8 8 1 7 .7 9 0 .7 0 7 .4 5 1 .2 5 F ee s an d ex p en se s fo r th e m u tu al fu n d in d u s- tr y ar e ca p p ed at a m ax im u m le v el b y re g - u la to ry au th o ri ti es . 2 2 .8 2 0 .7 2 9 .4 6 3 .4 3 3 0 .3 7 0 .9 9 2 4 .6 6 0 .7 9 1 0 .3 8 1 .4 5 F u n d fe es ar e re q u ir ed to b e re p o rt ed in th e fu n d ’s p ro sp ec tu s d o cu m en t. 5 9 .9 3 0 .8 4 3 7 .8 4 5 .6 8 7 2 .0 9 0 .9 7 6 3 .0 8 0 .8 8 3 8 .6 0 2 .3 2 T h e fe es o r ex p en se s ch ar g ed b y th e m u tu al fu n d co m p an y in a g iv en y ea r ca n b e ap p ro x im at ed b y m u lt ip ly in g th e fu n d ’s n et ex p en se ra ti o b y th e in v es tm en t g ai n s fo r th e y ea r. 1 4 .3 7 0 .6 0 1 .3 5 1 .3 5 1 8 .8 8 0 .8 4 1 5 .5 3 0 .6 6 6 .5 5 1 .1 8 M ea n sc o re (a v er ag e to ta l co rr ec t) 5 .1 2 .9 6 .1 5 .4 3 .3 N 3 4 4 4 7 4 2 1 5 0 3 0 0 1 4 4 3 42 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 4. Results 4.1. Individual question responses Table 2 provides the proportion of the sample who responded correctly to each item in the mutual fund index. Considering that most questions for this survey were developed with the intention of reflecting basic properties of mutual funds that should be considered when making investment decisions, the results overall are not encouraging in terms of respondent knowledge. Of the 11 questions we developed, only six were each answered correctly by more than half of respondents. The remaining five questions each had only a third of respondents, or fewer, responding correctly. The marketplace options question aims to determine if respondents are aware that there are many different alternative investment options available to them (and by implication, if they are not happy, they can shop around). Encouragingly, over 71 percent of respondents were aware that financial markets offer thousands of different mutual fund options. Of course, this does not mean that the respondents believe they have the skills to successfully navigate such a diverse choice environment, and in fact, it is conceivable that investors and non-investors may be para- lyzed by choice (e.g., see Carvalho & Silverman, 2019, and the somewhat related and Agnew & Szykman, 2005). Concern was reflected in cognitive testing conducted during the refinement of survey questions, where several respondents indicated that choosing funds felt overwhelming (e. g., when making choices related to their employer-sponsored retirement plan). However, these responses at least suggest a realization that alternatives exist and it may be worth additional search in the marketplace if their satisfaction is low with their current investment mix. In terms of risk, 36% of respondents thought that mutual funds pay a guaranteed rate of return. Almost half thought that it is not possible to lose money in a bond mutual fund (45%), although more than three-quarters (81%) did recognize that one could lose money in a stock mutual fund. In addition, one-third (32%) thought that mutual funds that are regis- tered with the SEC or a state regulator cannot lose money. Overall, these results suggest very little understanding of mutual fund risk. Despite the warning label offered on mutual fund product documents, nearly two-thirds (67%) indicated that past performance is a good predictor of future performance.14 The extant literature is not supportive of this view (e.g., Brown & Goetzmann, 1995; Carhart, 1997; Goetzmann & Ibbotson, 1994; Malkiel, 1995).15 Fee questions were constructed to be slightly more technical than risk questions, with cor- rect responses that may require knowledge that is more specialized to mutual fund products; at the same time, these questions are germane in the context of mutual fund choice problems given the importance of fees in determining net returns. Only a quarter (24%) of respondents correctly stated that no-load mutual funds charge yearly expenses (as would be reflected in the fund’s expense ratio), perhaps indicating a lack of understanding of the term load, which refers to a sales charge. Only 16% correctly identified that a load is not confined to the point of purchase (“front-load”), which sales loads (“back-loads”) exist. Seventy-eight percent erroneously believed that fees are capped at a maximum level by regulatory authorities. Three-fifths (60%) indicated that fund fees need to be reported in the fund’s prospectus. B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 43 Perhaps most disheartening is that only 14% correctly identified our conceptual fee compu- tation question as false. The fee computation question was intended to identify if respond- ents understood that fees are computed based on total account balance rather than on the basis of investment returns. This observation arose in cognitive interviews that revealed that some individuals believed that the fee basis is the much lower level (investment gain) than it actually is in practice (total balance), and the possibility that some investors believe that fees are not accrued if the fund has negative performance in a given period. Respondents over- whelmingly indicated that they implicitly believe that mutual fund fees are much lower than they really are, perhaps giving insight to the Bhattacharya et al. (2017) results. 4.2. Cumulative index scores Summing correct responses provides a composite score of the extent of a respondent’s knowledge about mutual funds, as well as a general perspective on aggregate knowledge of the public. Fig. 1 presents the distribution of index performance as the sum of correct responses. Fig. 2 presents the cumulative distribution of correct responses. Blue vertical lines denote the expected value from coin tossing true-false responses (5.5). Overall performance in the literacy index is poor. The average respondent score was 5.1, with a median score of 5.0 (Table 2a, Panel A). In Fig. 1, 52.2% of the sample answered less than six questions correctly; 13.2% answered less than two questions correctly. Only 11.3% answered at least nine questions correctly. Average results in particular groups roughly align with expectations, but highlight addi- tional deficiencies. In Panel B, average scores for those with the highest score on general fi- nancial literacy (all three “big 3” questions answered correct) and low general financial literacy (no “big 3” questions answered correctly), align with expectations to a certain degree. The few respondents (n = 74) that failed to correctly answer any of the generalized Fig. 1. Mutual fund knowledge score distribution. 44 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 financial literacy questions correctly, answered less than three mutual fund knowledge ques- tions correctly, on average. High financial literacy score respondents (n = 2,150), performed significantly better, with an average of 6.1 questions correct (two-sample t test t-statistic of difference in means is 10.5, with a p-value of 0.00). Yet, the average score of 6.1 questions correct is not impressive. Note that the correlation coefficient between our knowledge score and general financial literacy in the sample is moderate at 0.44. Fig. 2. Cumulative distribution function of scores Note: Cumulative number of respondents that scored less than or equal to a given score value. Fig. 3. Factor loadings in a two-factor solution. B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 45 Our survey collected both investor and non-investor responses. On one hand, it may be argued that non-investors are less consequential for determining overall knowledge and excluded from analysis, given that they may not have experience with these products. On the other hand, we view knowledge as a potentially important barrier for participation deci- sions, so application of the index to this subpopulation is of interest. Nevertheless, the ques- tion remains: do non-investors drive the results reported above? The answer from Table 2a (Panel C) is clearly no. Non-investors make up only about 13% of our respondents. While their scores on average are much lower than those for investors (3.3 vs. 5.4; 2-sample t test t-value: 15.2, p < .001), this has little effect on the average score previously reported; in short, investors do poorly enough on their own. Similarly, individual item responses for investors are as much as twice as accurate as those for non-investors, but performance on some questions such as fee basis and load were extremely poor even for the more experi- enced group. About three quarters of the more experienced group answered these questions wrong as compared with nearly ninety percentage of the non-investor group. As with high fi- nancial literacy respondents, investors did modestly well on risk questions, but overall tended to do poorly on fee questions. Fig. 4. Fee calculation question. 46 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 4.3. Index development While our 11 question battery was designed to test specific technical knowledge of key features of mutual funds, it is conceivable that the questions are really capturing a smaller set of underlying latent aspects of respondent knowledge. This might make some questions redundant. After all, if a respondent does not know that mutual funds are risky financial investments, they may answer both the risk stockfund and risk bondfund questions incor- rectly so that one of these questions might be eliminated. Following DeVellis (2016), to identify latent components, we conducted factor analysis with the 11 individual items (presented in Fig. 1). To determine the number of factors, or latent variables, present in the data, we investigated both the eigenvalues of factors identi- fied. Following the standard method identified in Kaiser (1960, 1970), factors with eigenval- ues greater than one were retained for analysis. Our analysis identified two factors with eigenvalues over 1: one corresponding to the market alternatives and risk categories, and a second corresponding to the performance history and fees categories (see Figure 3). Performance history loaded on to the fees factor, although this was the weakest loading of any item. We calculated item-total correlations for each item and the total index score and each item with its corresponding total factor score (presented in Table 3). Item-total correlations suggested strong relationships between each of the individual items and the overall index score, with the correlations ranging between 0.44 and .70. The correlation between individ- ual items and their respective factors is quite strong, with a range between 0.59 and .76. These results argue in favor of preserving all 11 items in the index.16 4.4. Index validation To demonstrate construct validity, we use descriptive regressions to examine the relation- ship between the index and selected outcome measures to gain more perspective on the ex- planatory power of the index (DeVellis, 2016).17 Results are presented in Table 4. Each outcome we consider has a widening sphere of influence: as a measure of direct application, Table 3 Item-total and item-factor total correlations Item Index total correlation Factor 1 correlation (market alternatives) Factor 2 correlation (fees) 2 0.7036 0.7553 3 0.6259 0.7182 4 0.6235 0.6898 5 0.6200 0.6442 9 0.6693 0.7023 10 0.7054 0.7686 1 0.5086 0.6378 6 0.5235 0.6377 7 0.4504 0.5978 8 0.5316 0.6663 11 0.4410 0.6020 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 47 we modeled response to a fee calculation skill question (column 1) (see Figure 4); we exam- ined the value of the index in explaining ownership of any financial investments (“Investors”; column 2); as our broadest outcome of interest, we examine the relationship between the index and the CFPB’s Financial Well-Being (FWB) metric (see Consumer Financial Protection Bureau, 2017; column 3). For ease of demonstrating the complementar- ity between overall financial literacy and mutual fund knowledge, the sum of correct ques- tions for each were normalized. For ease of exposition, outcomes were modeled using a linear probability model (LPM) or ordinary least squares (OLS), as appropriate.18 Covariates included were: gender, age, education, income, race/ethnicity, net worth, and general financial literacy (see Table 5). 4.5. Fee calculation skill We presented survey respondents with information showing a hypothetical fund fee pre- sentation, and response options that offered a total fee calculation with a rationale for each response. Respondents were asked to identify the response option that correctly approxi- mated the amount of fees paid to the fund’s management company in a year. We developed this question to eliminate challenges associated with numerical ability, while at the same Table 4 Descriptive regression estimates Fee calculation (1) Investor (2) FWB (3) Mutual fund knowledge (SD) 0.104*** (0.008) 0.053*** (0.006) 0.753*** (0.238) Financial literacy (SD) 0.021** (0.009) 0.001 (0.007) 0.497* (0.264) Female �0.043*** (0.015) 0.022** (0.011) �0.554 (0.416) Age 30–44 0.008 (0.036) �0.037 (0.027) �4.023*** (1.034) Age 45–59 �0.046 (0.036) �0.071*** (0.027) �4.747*** (1.039) Age 60+ �0.068* (0.036) �0.096*** (0.027) 3.060*** (1.026) High school graduate or equivalent �0.038 (0.070) 0.064 (0.052) �4.132** (1.995) Some college �0.027 (0.067) 0.089* (0.050) �3.582* (1.921) BA or above 0.017 (0.067) 0.124** (0.050) �1.886 (1.926) Income (USD 35,000–59,999) �0.004 (0.024) 0.192*** (0.018) 2.884*** (0.688) Income (USD 60,000–99,999) 0.012 (0.022) 0.259*** (0.017) 6.591*** (0.632) Income (USD 100,000+) 0.032 (0.022) 0.273*** (0.017) 8.910*** (0.638) Black 0.021 (0.037) 0.046* (0.028) 1.372 (1.054) Hispanic �0.068** (0.033) �0.014 (0.024) �0.729 (0.931) Race other 0.030 (0.026) �0.053*** (0.019) �2.378*** (0.733) Net Worth (breakeven) �0.008 (0.030) 0.030 (0.022) 3.056*** (0.850) Net Worth (positive) 0.036 (0.023) 0.138*** (0.017) 9.510*** (0.647) Constant 0.233*** (0.076) 0.499*** (0.056) 47.412*** (2.158) Observations 3,444 3,444 3,444 R2 0.100 0.198 0.294 Adjusted R2 0.096 0.194 0.291 Residual SE (df = 3,426) 0.402 0.301 11.490 F Statistic (df = 17; 3,426) 22.438*** 49.606*** 83.951*** Note: FWB = CFPB’s Financial Well-Being. *p < 0.1, **p < 0.05, ***p < 0.01. 48 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 time requiring knowledge of fee calculation and approximate valuation.19 Only a single option was correct. This question proximally relates to optimal decision making in mutual fund investment contexts since fees directly affect net fund returns (e.g., see Elton et al., 1993; Elton & Gruber, 2013; Elton et al., 2011; Gruber, 1996). Nevertheless, this question has proved challenging for most respondents; in initial trials, we found that only 20% of sur- vey participants correctly answered this question (Scholl & Fontes, 2019), and in the current survey only 21.6% answered correctly. Our model suggests that higher scores on our index are associated with correct calculation of the mutual fund fee. A 1 SD increase in mutual fund knowledge scores corresponds to an increase in the likelihood of correctly answering the question by 10.4 percentage points (Table 4, column 1). The complementary between our knowledge index and general finan- cial literacy is highlighted by the fact that the coefficient on our knowledge score is strong and significant when general financial literacy and other covariates are included in the regression model. In terms of relative explanatory impact, the coefficient on mutual fund knowledge was roughly five times that of general financial literacy (0.02). The adjusted R2 Table 5 Specification of co-variates Co-variate specification Gender (male, or not) Age 18–29 (omitted) 30–44 45–59 60+ Race/ethnicity White non-Hispanic (omitted) African American non-Hispanic Hispanic Other Education No high school diploma (omitted) HS graduate or equivalent Some college BA or above Income Less than $35,000 (omitted) $35000–$59,999 $60,000–$99,999 $100,000 or more Net worth In debt (omitted) Zero Greater than zero Fee calculation (correct, or not) Financial well-being Top 40% of scores Financial literacy score (0–3) Low literacy = 0 correct High literacy = 3 correct B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 49 for the regression is 0.096, and analysis of variance (ANOVA) analysis suggests that 8.9% of the total sum of squares is explained by mutual fund knowledge, relative to 0.004% explained by variation in general financial literacy (mean squared error of 49.8 vs. 2.3; F- statistic of 308.45 and 14.45, respectively). These results suggest that the mutual fund knowledge index rank orders respondents by fee calculation skill proficiency, which is highly important in investment decision making. While general financial literacy remains an important factor in predicting a correct response to the fee calculation question, the much larger relationship with mutual fund literacy sup- ports the idea that a specific measure of knowledge related to mutual funds is more relevant to modeling decision making. Furthermore, controlling for other demographic characteris- tics, only with a perfect score on our knowledge index is an individual more likely to cor- rectly answer our fee calculation question than to get it wrong—only 1.3% of respondents scored this high. 4.6. Investor status (investor participation) If our index is appropriately measuring mutual fund knowledge, we expect to find that higher scores are associated with a higher likelihood of owning financial securities invest- ments. As per Scholl and Hung (2018), mutual funds overwhelmingly dominate the compo- sition of investors’ investment holdings. Mutual fund owners are likely to have more experience with the products, resulting in higher knowledge scores, while those with less knowledge may be disinclined to purchase mutual funds. As presented in Table 4 (column 2), a 1 SD increase in our knowledge score is related to a 5.3 percentage point increase in the probability of owning financial investments (p-value < 0.001). Surprisingly, unlike our descriptive model predicting for the fee calculation question, general financial literacy did not explain variation in investor participation. The R2 for the regression is strong at 0.198. ANOVA results suggest mutual fund knowledge alone explains approximately 7.5% of the variation in participation. These findings are somewhat surprising in that general financial literacy has been used to explain broad financial outcomes in a variety of contexts and our measure of participation is not simply a measure of ownership of mutual funds or exchange traded funds, but rather of financial investments overall. Results support the idea that while general financial literacy is important in measuring many financial behaviors, a more targeted measure of mutual fund literacy may be important in understanding investment behavior. 4.7. Financial well-being Using the CFPB’s FWB score allows us to investigate the relationship between the mutual fund knowledge index and a much broader measure of overall financial wellness. The link- age between mutual fund knowledge and financial well-being is less direct than our prior two outcomes. As in the case of a household’s overall net worth, financial investments may only represent a portion of a household’s overall financial well-being. Rent and mortgages, debt, a family’s employment situation and life circumstances all arguably play a larger role 50 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 than financial investments for most families; Consumer Financial Protection Bureau (2017) notes, in particular, that liquid savings provided the biggest differentiation between respond- ents with different levels of FWB. Nevertheless, our knowledge measures may be a better proxy than other measures for latent knowledge components that are important to overall fi- nancial health. We found a positive relationship between the index score and increased financial well- being. Table 4 column 3 reports that a 1 SD increase in our mutual fund knowledge index equates to an increase in FWB of 0.753, and the coefficient is highly significant despite the presence of several other potentially important covariates such as general financial literacy, age, income, and net worth. The point estimate on mutual fund knowledge is somewhat modest given the standard deviation of FWB in our sample is 13.6, but the R2 for this model is overall 0.29. ANOVA results indicate that the knowledge index alone explains 8.7% of overall variation in FWB, with a mean sum of squares 2.3 times higher than the next most consequential covariate (net worth). Of note, the coefficient on general financial literacy was not quite significant at the 95% confidence level (p = .06) in our regression. 4.8. Conclusion and discussion We developed and deployed an 11-question index of mutual fund knowledge questions that is easy to deliver and has relatively low respondent burden, and fielded the module with a large, nationally representative, address-based probability sample to obtain credible popu- lation estimates of mutual fund knowledge. We developed the questions to reflect varying degrees of difficulty in mutual fund subject matter; all questions represented important choice-relevant topics in mutual fund selection and features that regulatory bodies routinely ascribe as important features for the investing public to consider when selecting investments. We refined the index with qualitative interviewing and extensive expert input, factor analy- sis and descriptive regression validation. While our results indicate the index is helpful in explaining important overall financial well-being, investor participation, and the highly important fee calculation skill, respondent performance on this battery is worrisome. A substantial fraction of respondents were no more accurate in their responses than if they had guessed at random and many respondents were unable to accurately answer a single true-false question. Our estimates suggest that only the top 11.4 respondents could achieve a score of nine or higher, which we broadly con- sider a high level of knowledge. Only the top 1.3% of respondents with a perfect score were more likely to correctly answer our fee calculation question than to get it wrong. In the con- text of the secular shift from defined benefit to defined contribution retirement plans in the United States, mutual funds in principle offer cheap diversification opportunities for most investors. Troublingly, we find that about eighty percentage of our respondents probably do not understand enough about mutual funds to make informed choices. In the context of regulatory and disclosure efficacy, our index provides context to the real- ities of the regulatory environment. The regulatory environment puts the onus of investment selection to investors; our battery of questions tests knowledge we identified as crucial to avoiding poor investment selection from the pool of available mutual funds and ETFs. As B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 51 discussed, poor selection within this class of investments can have severe consequences for investors. Even for subgroups one would a priori expect to perform better (e.g., investors and high general financial literacy individuals), knowledge of key fund characteristics is not very ro- bust. While these higher literacy and experience groups performed reasonably well on our market options and risk questions, they performed extremely poorly on questions about fees and the relationship of past and future performance. The limited understanding of fees in particular is perhaps the most worrisome finding in that fees and expenses are widely viewed as perhaps the single most important aspect of the investor’s investment decision. It also suggests that financial intermediaries, the educational system, and regulators are not doing enough to prepare people to make decisions crucial to their own well-being; or alternatively, it could indicate that the investment marketplace itself is simply too complex for broad seg- ments of the population to navigate successfully. Other research, including Scholl (2020) and a related set of experiments, provide evidence that more sophisticated investors have higher mutual fund knowledge (see Chin, Scholl, & VanEpps, 2021). General financial literacy correlates with higher mutual fund knowledge (r = 0.44), but nearly one-third of high financial literacy individuals still did worse on the question battery than the expected value from answering randomly. This highlights the fact that our knowledge index can serve to supplement general financial literacy measures in selected contexts researchers in understanding decision-making pathologies, and policy- makers in terms of assessing population vulnerabilities and checking assumptions about baseline investor knowledge. We believe financial regulatory authorities’ disclosure objec- tives relate to disclosures that help investors make informed investment decisions in invest- ment contexts. Our results suggest that in the context of inhibiting informed decision making by investors, the availability of information may be less important than investors’ (in)ability to interpret it. Notes 1 This article will discuss knowledge of properties largely common to mutual funds and Exchange Traded Funds (ETFs). Henceforth; we will simply refer to these as “mutual funds.” While there are technical differences between the two (e.g., how they are traded), these differences are of little consequence to our context here). 2 Unless otherwise noted, facts described here are documented in (Scholl & Hung, 2018), including facts about ownership of mutual funds and ETFs, account types, and the prevalence of funds in educational and retirement accounts. 3 Direct communication with investors is only one purpose for disclosures, and disclo- sures are vital to helping markets to operate in other ways. For example, they also communicate critical information to financial professionals who may serve as inter- mediaries for the investors. 4 Funds fall into the purview of other regulators in certain contexts. For example, the Department of Labor has jurisdiction over certain types of accounts that typically are comprised of mutual funds and exchange traded funds. 52 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 5 Gruber (1996), Elton et al. (1993), Elton and Gruber (2013), Elton et al. (2011). 6 See, for example, the recommendation of the SEC’s Investor Advisory Committee on fee disclosures: https://www.sec.gov/spotlight/investor-advisory-committee- 2012/recommendation-mf-fee-disclosure-041916.pdf. 7 Scholl and Fontes (2019, 2020) provide subgroup analysis in greater depth. 8 Alexander, Jones, and Nigro (1998) only sample individuals in mutual fund owning households. 9 Technical documentation for AmeriSpeak is available at: https://amerispeak.norc.org/ Documents/Research/AmeriSpeak%20Technical%20Overview%202019%2002%2018. pdf. A list of publications using AmeriSpeak data can be found at: https://amerispeak. norc.org/research/Pages/default.aspx. 10 NORC’s National Frame is used for the AmeriSpeak Panel, as well as other federal surveys including the Survey of Consumer Finances and the General Social Survey. 11 Results presented in this article do not use survey weights that would bring these demographics more in line with the U.S. population. 12 The index presented in this article treats skipped questions and incorrect answers as incorrect answers. In previous work, we also constructed a penalty-adjusted version of the score, which penalized incorrect answers. The two scores are highly corre- lated, with the penalty-adjusted scores providing more separation between sub- groups, but are overall highly correlated. Many of the additional questions that were proposed are discussed in Scholl and Fontes (2019). 13 As of 1996, states do not technically register mutual funds, but they do collect cer- tain fees associated with mutual fund filings. The intent of this question was to mea- sure whether investors believe that the regulatory environment prevents the loss of money, not to test their knowledge of the specific responsibilities of individual regu- lators or the division of state/federal roles. This question phrasing was adopted because it was deemed possible that individuals might not know how to distinguish between state and federal powers and roles as they have evolved over time. Testing did not reveal any particular focus on the first clause in the question (example regu- lators), respondents’ attention appeared to have been placed on the second clause (whether or not you “cannot lose money”). 14 “Past performance does not guarantee future results.” 15 This result may suggest that mutual fund warning labels are ambiguous, ineffective, ill placed or not understood, but it could also reflect the fact that investors struggle to identify specific characteristics that help determine how a mutual fund will per- form and are left to contemplate past performance when no other alternative discern- ment features present themselves. We leave interpretation to additional research. 16 “Don’t know” responses are treated as incorrect responses. We used the oblimin rotation (see Clarkson & Jennrich, 1988). Correlation between the two factors was �0.55. 17 See also Joint Research Centre-European Commission (2008) and Consumer Financial Protection Bureau (2017). B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 53 18 LPM estimates provided for convenience. We also estimated models using logistic regression (not reported); these were nearly identical when marginal probabilities were calculated. 19 Additional specifications on the fee calculation question (not presented) controlled for numeracy and survey design effects, but were virtually identical to those reported. We preferred to keep specifications largely consistent across our three out- come variables. References Agnew, J. R., & Szykman, L. R. (2005). Asset allocation and information overload: The influence of information display, asset choice, and investor experience. Journal of Behavioral Finance, 6(2), 57–70. Alexander, G. J., Jones, J. D., & Nigro, P. J. (1998). Mutual fund shareholders: Characteristics, investor knowl- edge, and sources of information. Financial Services Review, 7, 301–316. Barber, B. M., Odean, T., & Zheng, L. (2005). Out of sight, out of mind: The effects of expenses on mutual fund flows. The Journal of Business, 78, 2095–2120. Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2018). Behavioral household finance. In D. Bernheim, D. Laibson, & S. DellaVigna (Eds.), Handbook of Behavioral Economics: Applications and Foundations 1 (Vol. 1, pp. 177–276). Amsterdam: Elsevier. Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2011). How does simplified disclosure affect individuals’ mutual fund choices? In D. A. Wise (Ed.), Explorations in the Economics of Aging (pp. 75–96). Chicago, IL: University of Chicago. Bhattacharya, U., Loos, B., Meyer, S., & Hackethal, A. (2017). Abusing ETFs. Review of Finance, 21, 1217– 1250. Brown, S. J., & Goetzmann, W. N. (1995). Performance persistence. The Journal of Finance, 50, 679–698. Calvet, L. E., Campbell, J. Y., & Sodini, P. (2007). Down or out: Assessing the welfare costs of household invest- ment mistakes. Journal of Political Economy, 115, 707–747. Carhart, M. M. (1997). On persistence in mutual fund performance. The Journal of Finance, 52, 57–82. Carvalho, L., & Silverman, D. (2019). Complexity and sophistication. NBER Working Papers 26036. Cambridge, MA: National Bureau of Economic Research, Inc. Chin, A., Scholl, B., & VanEpps, E. (2021). Jargon in fund fee disclosures. Office of the Investor Advocate Working Paper, 2021–2001. Washington, DC: Office of the investor Advocate. Choi, J. J., Laibson, D., & Madrian, B. C. (2010). Why does the law of one price fail? An experiment on index mutual funds. The Review of Financial Studies, 23, 1405–1432. Clarkson, D. B., & Jennrich, R. I. (1988). Quartic rotation criteria and algorithms. Psychometrika, 53, 251–259. Consumer Financial Protection Bureau (CFPB). (2017). Financial Well-Being Scale: Scale Development Technical Report. Available at https://files.consumerfinance.gov/f/documents/201705_cfpb_financial-well- being-scale-technical-report.pdf deHaan, E., Song, Y., Xie, C., & Zhu, C. (2021). Obfuscation in mutual funds. SSRN Electronic Journal, 72(2/3), 2010. Davoli, M., & Rodrı́guez-Planas, N. (2020). Culture and adult financial literacy: Evidence from the United States. Economics of Education Review, 78, 102013. DeVellis, R. F. (2016). Scale Development: Theory and Applications (Vol. 26). Thousand Oaks, CA: Sage publications. Elan, S. (2011). Financial Literacy Among Retail Investors in the United States. Washington, DC: Library of Congress Federal Research Division. Elton, E. J., & Gruber, M. J. (2013). Mutual funds. In G. M. Constantinides, M. Harris, & R. M. Stulz (Ed.), Handbook of the Economics of Finance (Vol. 2, pp. 1011–1061). Amsterdam: Elsevier. 54 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 Elton, E. J., Gruber, M. J., & Busse, J. A. (2011). Are investors rational? Choices among index funds. The Journal of Finance, 59, 261–288. Elton, E. J., Gruber, M. J., Das, S., & Hlavka, M. (1993). Efficiency with costly information: A reinterpretation of evidence from managed portfolios. Review of Financial Studies, 6, 1–22. Fernandes, D., Lynch, J. G. , Jr., , & Netemeyer, R. G. (2014). Financial literacy, financial education, and down- stream financial behaviors.Management Science, 60, 1861–1883. Fontes, A., Jain, A., Zhu, M., & Dwan, K. (2019). Collecting household financial data online: A comparison of the Financial Well-being and Literacy survey with federal benchmarks. Presented at the CIPHER Conference. Washington, DC. Forbes, J., & Kara, S. M. (2010). Confidence mediates how investment knowledge influences investing self-efficacy. Journal of Economic Psychology, 31, 435–443. Goetzmann, W. N., & Ibbotson, R. G. (1994). Do winners repeat? The Journal of Portfolio Management, 20, 9–18. Hastings, J. S., Madrian, B. C., & Skimmyhorn, W. L. (2013). Financial literacy, financial education, and eco- nomic outcomes. Annual Review of Economics, 5, 347–373.e. [PMC] Hortaçsu, A., & Syverson, C. (2004). Product differentiation, search costs, and competition in the mutual fund industry: A case study of S&P 500 index funds. The Quarterly Journal of Economics, 119, 403–456. ICI. (2010). ICI Research Report: Profile of Mutual Fund Shareholders, 2010. Washington, DC: Investment Company Institute. ICI. (2018). ICI Research Report: profile of Mutual Fund Shareholders, 2018. Washington, DC: Investment Company Institute. Joint Research Centre-European Commission. (2008). Handbook on Constructing Composite Indicators: Methodology and User Guide. Paris: OECD publishing. Kahraman, B. (2021). Fee complexity and investor mistakes in retail financial markets. Quarterly Journal of Finance, 11(1), 2150004. Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151. Kaiser, H. F. (1970). A second-generation Little Jiffy. Psychometrika, 35, 401–415. Kaiser, T., & Menkhoff, L. (2017). Does financial education impact financial literacy and financial behavior, and if so, when? The World Bank Economic Review, 31, 611–630. Kaiser, T., & Menkhoff, L. (2020). Financial education in schools: A meta-analysis of experimental studies. Economics of Education Review, 78, 101930. Lin, J. T., Bumcrot, C., Ulicny, T., Mottola, G., Walsh, G., Ganem, R., Kieffer, C., & Lusardi, A. (2019). The State of U. S. Financial Capability: The 2018 National Financial Capability Study.Washington, DC: FINRA. Lusardi, A. (2008). Financial Literacy: An Essential Tool for Informed Consumer Choice? Cambridge, MA: National Bureau of Economic Research. Lusardi, A. (2011). Financial literacy and planning: Implications for retirement well-being. In O. S. Mitchell & A. Lusardi (Eds.), Financial Literacy: Implications for Retirement Security and the Financial Marketplace (pp. 17–39). Cambridge, MA: Oxford University Press. Lusardi, A. (2012). Numeracy, Financial Literacy, and Financial Decision Making. Cambridge, MA: National Bureau of Economic Research. Lusardi, A. (2019). Financial literacy and the need for financial education: Evidence and implications. Swiss Journal of Economics and Statistics, 155, 1.e. Lusardi, A., Michaud, P. C., & Mitchell, O. S. (2020). Assessing the impact of financial education programs: A quantitative model. Economics of Education Review, 78, 101899. Lusardi, A., & Mitchell, O. S. (2014). The economic importance of financial literacy: Theory and evidence. Journal of Economic Literature, 52, 5–44. Lusardi, A., & Mitchell, O. S. (2008). Planning and financial literacy: How do women fare? American Economic Review, 98, 413–417. Lusardi, A., Mitchell, O. S., & Curto, V. (2014). Financial literacy and financial sophistication in the older popu- lation. Journal of Pension Economics & Finance, 13, 347–366. B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56 55 Lusardi, A., & Tufano, P. (2009). Debt literacy, financial experiences, and overindebtedness. NBER Working Papers 14808. Washington, DC: National Bureau of Economic Research, Inc. Lusardi, A., & Tufano, P. (2015). Debt literacy, financial experiences, and overindebtedness. Journal of Pension Economics and Finance, 14, 332–368. Malkiel, B. G. (1995). Returns from investing in equity mutual funds 1971 to 1991. The Journal of Finance, 50, 549–572. Muller, L. A., & Turner, J. A. (2021). Financial literacy, the “High-fee puzzle,” and knowledge about the impor- tance of fees. The Journal of Retirement, 8, 29–38. Müller, S., & Weber, M. (2010). Financial literacy and mutual fund investments: who buys actively managed funds? Schmalenbach Business Review, 62, 126–153. Scholl, B. (2020). Choice Set Complexity. Working Paper. Washington, DC: SEC Office of the Investor Advocate. Scholl, B., & Fontes, A. (2019). What Does the Public Know about Mutual Funds? Working Paper. Washington, DC: SEC Office of the Investor Advocate. Scholl, B., & Hung, A. (2018). The Retail Market for Investment Advice. Washington, DC: Office of the Investor Advocate. Available at https://www.sec.gov/files/retail-market-for-investment-advice.pdf Scholl, B., Silverman, D., & Enriquez, M. (2020). Text Complexity and Performance. Santa Monica, CA: Rand BeFi Presentation. Scholl, B., Silverman, D., & Enriquez, M. (2021). Disclosure Complexity and Fund Performance. Working Paper. Washington, DC: SEC Office of the Investor Advocate. Urban, C., Schmeiser, M., Collins, J. M., & Brown, A. (2020). The effects of high school personal financial edu- cation policies on financial behavior. Economics of Education Review, 78, 10178. U. S. Securities & Exchange Commission. (2012). Study Regarding Financial Literacy Among Investors. Washington, DC: US Security and Exchange Commission. Available at https://www.sec.gov/news/studies/ 2012/917-financial-literacy-study-part1.pdf van Rooij, M., Lusardi, A., & Alessie, R. (2011). Financial literacy and stock market participation. Journal of Financial Economics, 101, 449–472. 56 B. Scholl and A. Fontes / Financial Services Review 30 (2022) 31–56