Hüffmeier & Krumm (2018).MP2018.842 Meta-Psychology, 2018, vol 2, MP.2018.842, https://doi.org/10.15626/MP.2018.842 Article type: Commentary Published under the CC-BY4.0 license Preregistration: N/A Preprint: https://doi.org/10.17605/osf.io/szx45 Data, Code and Materials: https://doi.org/10.17605/osf.io/yt2dq Edited by: Rickard Carlsson Reviewed by: Andreas Ivarsson & Ulrich Schimmack Peer review report: https://doi.org/10.17605/osf.io/dzw8c Editorial history: https://doi.org/10.17605/osf.io/rq4fv No Myth far and wide: Relay Swimming is Faster than Individual Swimming and the Conclusion of Skorski et al. (2016) is Unfounded Joachim Hüffmeier Technische Universität Dortmund Stefan Krumm Freie Universität Berlin Skorski, Extebarria, and Thompson (2016) aim at our article on relay swimmers (Hüffmeier, Krumm, Kanthak, & Hertel, 2012). We have shown that professional freestyle swimmers at relay positions 2 to 4 swam faster in the relay than in the individual competition if they had a high chance to win a relay medal. After applying a reaction-time correction that controls for different starting procedures in relay and individual competitions, Skorski et al. (2016) con- clude that swimmers in relays do not swim faster. At first sight, their results appear to show this very pattern. However, we argue that the authors’ findings and conclusion—that our find- ing is a myth—are not warranted. First, we have also controlled for quicker reaction times in the relay competition. Our correction has been based on the swimmers’ own reaction time data rather than on a constant reaction time estimate and is, thus, more precise than theirs. Second, Skorski et al. treat data from international and national competitions equally although national relay competitions are less attractive for the swimmers than national individual competitions. This difference likely biases their data towards slower relay times. Third, the authors select a small and arbitrary sample without explicit power considerations or a clear stopping rule. Fourth, they unfavorably aggregate their data. We conclude that the reported results are most likely due to the methodological choices by Skorski et al. and do not invalidate our findings. Keywords: teams; groups; performance gains; effort gains; relay swimming; reaction-time cor- rection; exchange-block times Meta-Psychology, 2018, vol 2, MP.2018.842, https://doi.org/10.15626/MP.2018.842 Article type: Commentary Published under the CC-BY4.0 license Preregistration: N/A Preprint: https://doi.org/10.17605/osf.io/szx45 Data, Code and Materials: https://doi.org/10.17605/osf.io/yt2dq Edited by: Rickard Carlsson Reviewed by: Andreas Ivarsson & Ulrich Schimmack Peer review report: https://doi.org/10.17605/osf.io/dzw8c Editorial history: https://doi.org/10.17605/osf.io/rq4fv In their recent study, Skorski, Extebarria, and Thompson (2016) set out to show that our findings (Hüffmeier, Krumm, Kanthak, & Hertel, 2012) are un- founded or, in their words, “a myth” (cf. the title of their article). In the current commentary, we intend to illus- trate (i) that the reported findings in the article by Skor- ski et al. (2016) most likely result from unwarranted methodological choices and (ii) that their assessment of our findings is, thus, not valid. To do so, we first portray our pertinent studies. We focus on our study proce- dures, the methodological choices we made, our main findings, our explanation as well as possible alternative explanations for these findings, and we also discuss the robustness of our findings in relation to different meth- odological choices. We then describe the procedure and findings of Skorski et al. (2016). Finally, we point out four central shortcomings in the Skorski et al. article and conclude that the results reported by Skorski et al. cannot effectively question our results. Our Studies Main goal of our studies. Our article (Hüffmeier et al., 2012) is part of a series of published (Hüffmeier et al., 2017; Hüffmeier & Her- tel, 2011; Hüffmeier, Kanthak, & Hertel, 2013) and cur- rently not yet published studies (Schleu, Mojzisch, & Hüffmeier, 2018). They build on a long research tradi- tion in (social) psychology that investigated—mostly in laboratory experiments—how working as part of a group affects the members’ motivation and perfor- mance (see Karau & Williams, 1993; Weber & Hertel, 2007, for overviews). In their meta-analysis, Weber and Hertel (2007) have shown that group work can result in effort gains in groups (i.e., higher effort during group as compared to individual work).1 They have further found that perceived social indispensability (i.e., the perception that the own contribution to the group per- formance is critical) is an important driver of such effort gains. Our studies investigated whether and under which conditions group work results in such indispen- sability-based effort gains in groups outside the research 1 Prior research used the term “motivation gains in groups”. However, motivation encompasses the direction, intensity, and persistence of behavior. Pertinent small group research such as ours typically does not study the direction, but the laboratory (see Hüffmeier & Hertel, in press, for an over- view of the swimming studies). Methodological approach of our studies. Most of our pertinent studies were situated in a pro- fessional swimming context and compared swimming performances from the individual and relay competi- tions of the same event (e.g., the 2008 Olympic Games; in a recent study, we investigated performances in a track and field context, cf. Schleu et al., 2018). Due to the high standardization of performances, professional swimming is an apt context to study the motivational impact of group versus individual work. In fact, only the prescribed starting procedures differentiate between the work that swimmers do in the individual and relay competitions. The starting rules for official swimming competi- tions (cf. the rules of the world swimming federation [FINA]) prescribe the respective starting procedures:2 Swimmers at the first relay position and swimmers in the individual competition have to follow identical starting rules. These swimmers perform a “flat start”, meaning that they cannot reliably anticipate their acoustic starting signal and are not allowed to move be- fore this starting signal sounds. By contrast, in the relay competition, swimmers at relay positions 2 to 4 perform a “flying start”. That is, they can reliably anticipate their “starting signal” (i.e., their predecessor touching the pool wall). And they are allowed to move before this starting signal as long as at least one foot still touches the starting block when their predecessor touches the wall. Obviously, these differences lead to quicker start- ing or reaction times in the relay as compared to the individual competitions for swimmers at relay positions 2 to 4. Consequently, the resulting reaction-time ad- vantages in relays of about 0.422 to 0.524 sec (depend- ing on the swimmers’ gender, relay position, and swim- ming distance, see Hüffmeier et al., 2017; see also Table 1) have to be controlled to assess the pure swimming time difference between relay and individual swim- ming. Our studies shared the following approach: First, we sampled data of swimmers taking part in both the indi- vidual and relay competitions in the same swimming intensity and persistence of behavior (i.e., the two compo- nents of effort). We, thus, prefer the term “effort gains in groups”. 2 See the FINA homepage: http://www.fina.org/content/fina-rules NO MYTH FAR AND WIDE: RELAY SWIMMING IS FASTER THAN INDIVIDUAL SWIMMING AND THE CONCLUSION OF SKORSKI ET AL. (2016) IS UNFOUNDED 3 discipline at the same professional and international events (e.g., 100 meter freestyle competitions at the 2008 Olympic Games).3 We analyzed data from na- tional competitions in only one article (Hüffmeier et al., 2017). Second, with one exception (Hüffmeier & Her- tel, 2011), we made sure that the included data came from comparable stages in both competitions (e.g., final races from both the individual and relay competitions). Third, in most studies, we only included a swimmer if his/her reaction times from both competitions were available (Hüffmeier et al., 2013, 2012; Hüffmeier & Hertel, 2011). As an alternative approach, we con- ducted a pilot study to derive estimates of the reaction- time advantage separately for each relay position (Hüff- meier et al., 2017; see also Table 1). We used both types of data (i.e., original reaction-time data or estimates) to correct for the reaction-time advantages separately for each relay position (i.e., by subtracting the respective 3 The resulting data are not distributed equally across relay positions because normally only two swimmers from one country can qualify for individual competitions at the prestigious international championships and these two reaction times from the swimming times for the individ- ual and relay competition). Fourth, we either included all data that was availa- ble at the time of a study (e.g., Study 1 in Hüffmeier et al., 2017; Hüffmeier et al., 2013; 2012) or we con- ducted a power analysis to determine the necessary sample size to detect the expected effects (Study 2 in Hüffmeier et al., 2017). Fifth, in accordance with our finding that indispensability perceptions increase across the positions of the relay (Hüffmeier & Hertel, 2011), we expected a parallel increase in effort across the re- lay. Thus, we conducted analyses testing a correspond- ing linear contrast as well as separate analyses for each relay position rather than aggregating the swimming performance data across relay positions. Sixth, we in- terpreted the observed performance differences be- tween the individual and relay competition as differ- ences in the swimmers’ effort levels. strongest national swimmers are typically assigned to the first and fourth relay position (see, for instance, Table 2 in Neugart & Richiardi, 2013, or Table 1 below). Relay position Skorski, Extebarria, & Thompson (2016)1 Hüffmeier et al. (2017), Study 12 Hüffmeier et al. (2017) Study 23 Position 1 -- 0.019 s (95% CI: -0.07; 0.045) -0.006 s (95% CI: -0.010; -0.002) Position 2 0.48 s 0.505 s (95% CI: 0.442; 0.568) 0.465 s (95% CI: 0.448; 0.482) Position 3 0.48 s 0.455 s (95% CI: 0.366; 0.544) 0.455 s (95% CI: 0.431; 0.479) Position 4 0.48 s 0.524 s (95% CI: 0.445; 0.603) 0.463 s (95% CI: 0.447; 0.479) Note. 1 Skorski et al. (2016) applied a constant of 0.48 seconds to the swimming times of 100 meter relay swimmers at the second, third, and fourth relay position as a reaction time estimate. This constant was taken from a swimmer sample that was unrelated to their own sample (see Saavedra et al., 2014, for details). 2 In their Study 1, Hüffmeier et al. (2017) collected data to specifically estimate the reaction time advantages in relays sepa- rately for relay position, swimming distance, and gender. To illustrate the methodological differences as compared to Skorski et al. (2016), the data from men competing in the 100 meter freestyle competition is reported in Table 1. 3 In their Study 2, Hüffmeier et al. (2017) used a data set not specifically collected for this purpose to derive reaction time estimates. To illustrate the methodological differences as compared to Skorski et al. (2016), the data from men competing in the 100 meter freestyle competition is reported in Table 1. Table 1. Overview of the Reaction Time Estimates Applied in Skorski et al. (2016) and in the two Studies of Hüff- meier et al. (2017) for the Four Relay Positions (Reported in Seconds) HÜFFMEIER & KRUMM 4 Results of our studies. Across all five studies focusing on swimming perfor- mances, we found the expected linear increase in effort across the relay—in those conditions where we ex- pected it. The effect sizes for this linear increase were typically medium to large (Cohen, 1992; see also Table 2). In four of five studies (except for Study 1 in Hüff- meier et al., 2017), the effort level at the first relay po- sition was comparable to the expended effort in the in- dividual competition (i.e., no performance differences between the two competition types). Again, in four of five studies (except for Study 2 in Hüffmeier et al., 2017), we observed very small, but statistically signifi- cant effort gains in groups at relay position 2 (i.e., quicker swimming times in the relay as compared to the individual competition; see Table 2). We consistently found effort gains at position 3, which were largely comparable in size to those at position 2. At the fourth position, we again consistently found effort gains. These gains were more pronounced than at the prior positions in all but one study (Hüffmeier et al., 2013; see Table 2), but they were still small in size. Concerning potential moderators of these findings, we have initial evidence that type of swimming relay (e.g., freestyle vs. medley relays) may act as a modera- tor (i.e., we found no effort gains in medley relays, Hüff- meier et al., 2013). We have more reliable evidence that (i) the swimmers’ medal chances in the relay and (ii) the relative valence of the obtainable group outcomes may moderate these findings in the following way: Swimmers without chances to win a medal in the relay competition and swimmers competing for less attractive outcomes in the relay than in the individual competition did not exhibit effort gains in groups (Hüffmeier et al., 2017; 2012). By contrast, we did not find that swimmers’ gender, the swimming distance (100 vs. 200 meters) or the sports discipline (swimming vs. track and field relays) moderated the observed findings. In a first track and field study, we studied runners’ efforts (N = 397) in the individual and relay 400 meter running competitions of Effect Sizes 1 Study Total Sample Size Relevant Condition Sub-sam- ple Size Position 1 Position 2 Position 3 Position 4 Planned contrast 4 Hüffmeier et al. (2012) 199 High instru- mentality of group perfor- mance 2 151 -0.014 0.040 0.045 0.107 .74 Hüffmeier et al. (2017) – Study 1 302,576 High instru- mentality of group perfor- mance 2 and high valence competitions 3 928 0.010 0.003 0.007 0.089 .80 Table 2. Summary of Exemplary Previous Studies and Effect Sizes Indicating Increases in Effort Across the Relay and Effort Gains in Groups Note. To allow for a high comparability of the findings in the 2017 data set with the original findings in Hüffmeier et al. (2012), only 100 m free-style races were analyzed for this table. Effect sizes are Cohen’s d, using the standard deviation of individual swimming times. Negative effect sizes indicate performance losses. 1 Mean difference between individual and relay competition times (corrected by reaction times of different starting procedures) divided by the standard deviation of the individual times. 2 Good chance of winning a medal. 3 Olympic Games, World Championships, European Championships, Pan Pacific Games, Commonwealth Games, and Univer- siades. 4 Contrary to the analyses in the referenced papers, planned contrasts were analyzed within the relevant condition only. The t- value of the planned contrast was converted into Cohen’s d. NO MYTH FAR AND WIDE: RELAY SWIMMING IS FASTER THAN INDIVIDUAL SWIMMING AND THE CONCLUSION OF SKORSKI ET AL. (2016) IS UNFOUNDED 5 the same prestigious and international sports events (Schleu et al., 2018). Paralleling our findings from the swimming domain, we found a linear increase in self- reported effort and also in objective performance across the running relay. We again observed effort gains in groups for the athletes running at later positions in the relay. Explanation for our findings. To learn whether the perceived indispensability of the own contribution to the group performance (cf. We- ber & Hertel, 2007) could also be a relevant mediator explaining the effort gains of relay swimmers, we con- ducted a study with competitive adolescent swimmers (Hüffmeier & Hertel, 2011). Confirming our expecta- tions, they perceived an increasing indispensability of the own contribution to the group performance across the relay (i.e., they progressively perceived that an own bad performance could not be compensated by fellow relay swimmers). Because this increase in indispensa- bility across the relay parallels the consistently observed increase in effort across the relay, we believe that per- ceived indispensability may be an important mechanism underlying the effort gains among relay swimmers. Alternative explanations. In this section, we discuss four prominent alternative explanations for our main findings. First, the different starting procedures in the individual and relay condi- tion most likely do not represent a valid alternative ex- planation: This account cannot explain the consistently found linear increase in effort across the relay and, in particular, the most pronounced effort gains at the last relay position. Second, the order in which swimmers took part in their competitions (e.g., relay competition first vs. individual competition first) could be proposed as another explanation. In the studies where we could test this explanation, we did, however, not observe a significant moderation by competition order (Hüffmeier et al., 2017; 2012). Third, although prominent in the literature on effort gains in groups (Weber & Hertel, 2007), social compar- ison is a rather unlikely alternative explanation. Obvi- ously, the clearest possibilities for comparing one’s per- formance with those of others exist in the individual competition and for the first relay swimmers because all swimmers start simultaneously in these situations. However, first relay swimmers did not exhibit effort gains in any of our studies and the backlogs or leads accumulated over the course of the relay competitions impede such comparisons for the remaining relay posi- tions. These are, however, the positions where effort gains were observed. Fourth, group members’ relative strength was dis- cussed as an alternative explanation (cf. Osborn, Irwin, Skogsberg, & Feltz, 2012), meaning that effort gains were primarily expected among the weaker swimmers of a relay. This expectation is transferred from labora- tory research where the weaker group members often exhibit effort gains in typical laboratory tasks (cf. Weber & Hertel, 2007). We conducted a study (Study 2 in Hüffmeier et al., 2017) to put this explanation to an em- pirical test. To do so, we collected data on the relative strength of all relay members and ordered them accord- ingly in our analysis. The resulting linear contrast for relative member strength was, however, not significant and could therefore not explain the observed results. Robustness of our findings in relation to different methodological choices. Our extant results were largely unaffected by the type of analysis and the type of reaction-time correction we used. In most of our studies, we report analyses us- ing difference scores (i.e., we subtracted the relay from the individual swimming times). However, our results did not hinge on this type of analysis: We obtained equivalent results in all studies when conducting re- peated-measures analyses. Our results were also unaffected by the different types of data we applied to correct for the reaction-time advantages in the relay. First, we observed parallel find- ings when using the exact reaction-time data of the swimmers (e.g., Hüffmeier & Hertel, 2011; Hüffmeier et al., 2012) and when using reaction-time estimates (Hüffmeier et al., 2017). Second, different types of re- action-time estimates did also not affect our results: In our two most recent studies, we applied two sets of re- action-time corrections (i.e., from a pilot study designed to establish estimates and from a big data set not spe- cifically collected for this purpose; cf. Hüffmeier et al., 2017, and Table 1). We obtained comparable results with both sets of reaction time corrections. We are, however, well aware that our findings—es- pecially at relay positions 2 and 3—are probably sensi- tive to methodological choices because the related ef- fects are very small (see Table 2). For instance, if a uni- form and higher estimate is applied to each relay posi- tion to correct for the reaction-time advantages in relays (as, for instance, in Neugart & Richiardi, 2013) we probably would not observe effort gains at these posi- tions. Applying a uniform estimate to each relay posi- tion and choosing higher estimates are, however, HÜFFMEIER & KRUMM 6 doubtful methodological decisions (see Hüffmeier et al., 2017, for related data and a discussion; see also Table 1). Nevertheless, the effort gains at relay positions 2 and 3 are probably less robust than the effort gains at the fourth relay position and especially less robust than the overall linear increase in effort across the relay. Our study targeted by Skorski et al. (2016). In our study (Hüffmeier et al., 2012), we analyzed the swimming performances of N = 199 professional freestyle swimmers from about a decade of 100 meter competitions (i.e., Olympic Games from 1996 to 2008, World Championships from 1998 to 2011, and Euro- pean Championships from 2000 to 2010) and we cor- rected their swimming times from the individual and re- lay competition by using their own reaction times. Our analysis shows that swimmers at the first relay position swam equally fast in both competition types while swimmers at relay positions 2 to 4 swam faster in the relay as compared to the individual competition if they had a high chance to win a relay medal.4 These effort gains in groups increased from the second to the fourth relay position. The Skorski et al. (2016) Article Skorski et al. (2016) analyze the swimming perfor- mances of 166 male swimmers from 778 races. These races were either part of the 100 and 200 meter free- style competitions of the Olympic Games from 2000, 2004, and 2012 (n = 144 races) or of international and national Non-Olympic 100 and 200 meter competitions (n = 634 races). The authors frame their use of a reac- tion-time correction controlling for the quicker reac- tions times in the relay as compared to the individual competition as the main contribution of their research. The authors report that they do no longer find that the analyzed swimmers swim faster in relay events than in their individual races if they control for the existing reaction-time advantages in the relay competitions. As corrections, they add 0.48 sec to the swimming times of the 100 m relay competitions and 0.45 sec to the swim- ming times of the 200 m relay competitions. While the authors’ conclusion does not seem to be fully accurate (on page 412, the authors report that “there was a small positive performance effect after adjusting for reaction time” for the swimming times of swimmers at positions 4 This finding is not restricted to relay swimming with high medal chances as compared to individual swimming with freely varying medal chances. We have obtained similar 2 to 4 in Olympic 100 meter freestyle relays), we appre- ciate their efforts to critically assess our own findings. Upon closer inspection, however, it seems that the au- thors may have overlooked some aspects of our study and that they made several methodological decisions that taint their reported results and question their con- clusion. Clarification and Four Central Shortcomings of the Skorski et al. (2016) Article A clarification of the research literature: Most per- tinent studies applied a reaction-time correction. Skorski et al. present their use of a reaction-time cor- rection as a significant contribution. This creates the im- pression to readers that we did not apply a reaction- time correction in the targeted article (Hüffmeier et al., 2012). But of course, we and others have consistently done so in the targeted and other pertinent articles (e.g., Hüffmeier & Hertel, 2011; Hüffmeier et al., 2017; 2013; Neugart & Richiardi, 2013; Osborn et al., 2012). The first shortcoming: Our applied reaction-time correction is more precise than that of Skorski et al. (2016). The reaction-time correction in our 2012 article was more precise than the one reported by Skorski et al. (2016) because we relied on the swimmers’ own reac- tion time data: Thus, we did not use reaction-time esti- mates, but only included swimming data for a swimmer in our analysis if his/her exact reaction times were available for both the individual and relay competition. By contrast, Skorski et al. (2016) added two different and rather rough constants to their data from the 100 and 200 meter relay competitions (0.48 sec and 0.45 sec, respectively). These constants are not related to the swimmers of their sample, but were instead derived from a prior, unrelated study (cf. Saveedra, García-Her- moso, Escalante, Dominguez, Arellano, & Navarro, 2014). Our own recent research suggests that adding the same constant to each relay swimming time is an impre- cise approach because reaction times differ depending on the relay position (see Hüffmeier et al., 2017 and Table 1, for details). Thus, the authors did not only ig- nore that their use of a reaction-time correction was not findings when the chances of success in both the relay and individual competition varied freely (see Study 2 in Hüffmeier et al., 2017). NO MYTH FAR AND WIDE: RELAY SWIMMING IS FASTER THAN INDIVIDUAL SWIMMING AND THE CONCLUSION OF SKORSKI ET AL. (2016) IS UNFOUNDED 7 novel at all, but they also applied a less precise reaction- time correction than we did. It is, thus, unlikely that the absence of observed differences between swimming times from the relay and individual competitions is due to the mere use of reaction-time corrections as the au- thors claim. In contrast, their imprecise correction may possibly have obscured otherwise observable differ- ences. The second shortcoming: The sample in Skorski et al. (2016) with many more Non-Olympic than Olympic races systematically biases the results to- wards slower relay swimming times. Skorski et al. (2016) analyze a variety of competi- tions: Races from Olympic competitions (n = 144) as well as from international and national Non-Olympic competitions (n = 634), including national champion- ships. If the authors attempted to show that the results of our article are unfounded, this sampling approach is problematic because we focused our analysis exclu- sively on international championships (i.e., Olympic Games, World and European Championships; see Hüff- meier et al., 2012). In these championships, the relay and individual competitions are equally attractive for the professional swimmers—in both types of competitions they pursue the ultimate goal to win the race or at least to win a medal. The situation is different for national champion- ships: Here, the individual competition is much more at- tractive than the relay competition because swimmers can win and qualify for the upcoming international events in the individual competition (e.g., Olympic Games). By contrast, they can typically only win, but not qualify in the relay competition (i.e., relays at interna- tional events mostly consist of the best four national swimmers and not of the best relay from the national championship). In fact, many national swimming asso- ciations do not even consider relay performances from national championships as a relevant criterion for se- lecting swimmers for major international swimming competitions like the Olympic Games or World Cham- 5 For recent examples, see the nomination criteria for the 2017 World Championships by the German Swimming Association (www.dsv.de/schwimmen/nationalmannschaft/nominier- ungs-richtlinien/), the Austrian Swimming Association (http://www.osv.or.at/schwimmen-mit-open-water/quali- fikationsrichtlinien/) or the US National Swimming Associa- tion (https://www.usaswimming.org/resources-home/re- source-topic/resource-subtopic). pionships.5 We have found empirical evidence that ac- cords with the differing attractiveness of national indi- vidual and relay competitions in a re-analysis of a data set with more than 300,000 races (Hüffmeier et al., 2017; cf. Neugart & Richiardi, 2013). In this sample, professional athletes show the typical pattern of in- creasing effort gains in relays from position 2 to 4 for international competitions, while we even observe ef- fort losses in groups for national and local competitions (i.e., slower swimming times in the relay as compared to the individual competition; for a related hypothesis on the valence of group outcomes, see Karau, Markus, & Williams, 2000). In Skorski et al. (2016), a parallel difference be- tween Olympic and Non-Olympic competitions appears to be present for the 100 meter races (see their Figure 1). The authors even describe that “there was a small positive performance effect after adjusting for reaction time” (p. 412) for the swimming times of swimmers at positions 2 to 4 in Olympic 100 meter freestyle relays, which—nicely—replicates our findings from the 2012 study.6 The distribution of Olympic (n = 144) and Non- Olympic races (n = 634), thus, most likely biases the results of Skorski et al. (2016) towards effort losses dur- ing group work (i.e., slower swimming times in the re- lay than in the individual competition). The sample se- lected for the authors’ study appears not to be suited for showing that our findings may be unfounded, but rather analyzes another and a very specific population of relay competitions. The third shortcoming: Skorski et al. select an arbi- trary and rather small sample without providing a sufficient justification. Above, we have illustrated that only the Olympic competition data from the Skorski et al. article is suited for showing that our findings are unfounded. The sam- ple of Skorski et al. (2016) consists of n = 144 Olympic races and, thus, is rather small. It is not clear why the authors did not try to allocate a bigger sample (e.g., by including the data from the 1996 Olympics, which would also have been available on the website they used 6 Note, however, that the exact numbers reported in Skorski et al. (2016) for these data are wrong or at least unclear: Alt- hough the depicted reaction-time-corrected effect sizes in their Figure 1 for the Olympic 100 meter competitions are clearly positive and appear to have an effect size d of about 0.50-0.60, the graphically depicted values are described in the figure caption as being either very small and positive (0.05) or even strongly negative (-1.06). HÜFFMEIER & KRUMM 8 for their data collection [i.e., swimrankings.net]) and the authors do not provide a sufficient justification for the arbitrary selection of their sample.7 Assembling a certain sample size without a clear stopping rule (e.g., to collect all data that is available at the time of the study or to continue data collection until a stop criterion is reached, e.g., Frick, 1998) or without an a priori power analysis is problematic because it decreases the possibility to detect true effects (and increases the pos- sibility of coming up with false positive effects; cf. Co- hen, 1992). The small sample size may have contributed to the absence of significant differences between relay and in- dividual swimming performances—beyond the impre- cise applied reaction-time correction (cf. the first short- coming) and the predominant inclusion of national competition data (cf. the second shortcoming). With bigger samples and, thus, more statistical power, we have repeatedly found evidence for faster swimming times in the relay competition (see, for instance, the two studies in Hüffmeier et al., 2017). The fourth shortcoming: Skorski et al. aggregate data over systematically different relay positions and may thereby obscure otherwise observable dif- ferences between relay and individual swimming times. Skorski et al. (2016) aggregate the swimming times for swimmers at the relay positions 2 to 4 and only pro- vide statistical results for this aggregate (i.e., no sepa- rate results per relay position are given). This is prob- lematic because, in our studies, the size of the perfor- mance gains in the relay competition varied across po- sitions. This information was available when Skorski et al. (2016) wrote their article (cf. Hüffmeier & Hertel, 2011; Hüffmeier et al., 2012)—but, admittedly, it be- came most evident in our most recent article that was published only later (Hüffmeier et al., 2017). In the most recent article, we found only (very) small effort gains for positions 2 and 3, even in the con- dition that made effort gains in the relay most likely (i.e., high chances of winning a medal in the relay com- petition, high valence of the relay competition [e.g., at World championships]; see Hüffmeier et al., 2017). Specifically, we observed relay swimming times that were only 0.008 s (position 2) and 0.031 s (position 3) faster than the respective individual times—as com- pared to 0.202 s faster times at relay position 4. While 7 As far as we know, data availability in data bases starts with the 1996 Olympic Games. all these differences in swimming times were signifi- cantly different from zero in the Hüffmeier et al. (2017) study due to their large sample (n = 738 swimmers at positions 2 and 3 in 100 and 200 m races), studies that use smaller samples and aggregate swimming time dif- ferences across positions 2 to 4 might obscure otherwise observable faster swimming times especially at relay po- sition 4. Conclusion In our discussion of the Skorski et al. (2016) article, we have identified the need to clarify the existing liter- ature in one place and, more importantly, four short- comings: While Skorski et al. suggest that controlling for reaction-time advantages in relays is a novel contri- bution of their study it is in fact common research prac- tice. Turning to the first shortcoming, the authors ap- plied imprecise correction factors. Second, they in- cluded data from national competitions and—due to the relative unattractiveness of national relay competi- tions—most likely biased their findings towards slow re- lay swimming times (cf. Hüffmeier et al., 2017; see also Karau et al., 2000). This data is not well suited for an unbiased empirical test of the hypothesis that relay swimming is not faster than individual swimming. Third, Skorski et al. selected an arbitrary and compara- bly small sample, thereby reducing the chance to find significant differences. Fourth and finally, they only provide aggregated data analyses despite well-docu- mented differences between relay positions. They, thus, obscure possibly otherwise observable differences between swimming times from relay and in- dividual competitions especially at the last relay posi- tion. Given the multitude, weight, and association of these four shortcomings, we are convinced that the au- thors’ findings are non-conclusive in view of their re- search question. Their data therefore do not question the validity of our findings and their conclusion—that our findings are “a myth”—is necessarily unfounded. NO MYTH FAR AND WIDE: RELAY SWIMMING IS FASTER THAN INDIVIDUAL SWIMMING AND THE CONCLUSION OF SKORSKI ET AL. (2016) IS UNFOUNDED 9 Open Science Practices This article earned the Open Data badge for making the data available through the Open Science Framework project page at https://doi.org/10.17605/osf.io/rq4fv. An editorial assistant independently confirmed that the code reproduced the results presented in the article. The editorial history can also be accessed from the OSF project page. The nature of the research question meant that there were no relevant research materials. References Cohen, J. (1992). A power primer. Psychological Bulle- tin, 112, 155-159. doi:10.1037//0033-2909.112.1.155 Frick, R. W. (1998). A better stopping rule for conven- tional statistical tests. Behavior Research Methods, 30, 690-697. doi: 10.3758/BF03209488 Hüffmeier, J., Filusch, M., Mazei, J., Hertel, G., Mojzisch, A., & Krumm, S. (2017). On the bound- ary conditions of effort losses and effort gains in action teams. Journal of Applied Psychology, 102, 1673-1685. doi:10.1037/apl0000245 Hüffmeier, J., & Hertel, G. (2011). When the whole is more than the sum of its parts: Group motivation gains in the wild. Journal of Experimental Social Psychology, 47, 455-459. doi:10.1016/j.jesp.2010.12.004 Hüffmeier, J. & Hertel, G. (in press). Effort losses and effort gains in sports teams. In S. J. Karau (Ed.), Individual motivation within groups: Social loafing and motivation gains in work, academic, and sports teams. New York: Academic Press. Hüffmeier, J., Kanthak, J., & Hertel, G. (2013). Speci- ficity of partner feedback as moderator of group motivation gains in Olympic swimmers. Group Pro- cesses & Intergroup Relations, 16, 516-525. doi:10.1177/1368430212460894 Hüffmeier, J., Krumm, S., Kanthak, J., & Hertel, G. (2012). “Don't let the group down”: Facets of in- strumentality moderate the motivating effects of groups in a field experiment. European Journal of Social Psychology, 42, 533-538. doi:10.1002/ejsp.1875 Karau, S. J., Markus, M. J., & Williams, K. D. (2000). On the elusive search for motivation gains in groups: Insights from the Collective Effort Model. Zeitschrift für Sozialpsychologie, 31, 179-190. doi:10.1024//0044-3514.31.4.179 Neugart, M., & Richiardi, M. G. (2013). Sequential teamwork in competitive environments: Theory and evidence from swimming data. European Eco- nomic Review, 63, 186-205. doi:10.1016/j.euroe- corev.2013.07.006 Osborn, K. A., Irwin, B. C., Skogsberg, N. J., & Feltz, D. L. (2012). The Köhler effect: Motivation gains and losses in real sports groups. Sport, Exercise, and Per- formance Psychology, 1, 242-253. doi:10.1037/a0026887 Saavedra, J. M., García-Hermoso, A., Escalante, Y., Dominguez, A. M., Arellano, R., & Navarro, F. (2014). Relationship between exchange block time in swim starts and final performance in relay races in international championships. Journal of Sports Sciences, 32, 1783-1789. doi:10.1080/02640414.2014.920099 Schleu, J. E., Mojzisch, A., & Hüffmeier, J. (2018). Run! Run for the team: An analysis of effort gains in track and field relays. Manuscript in preparation. Skorski, S., Etxebarria, N., & Thompson, K. G. (2016). Breaking the myth that relay swimming is faster than individual swimming. International Journal of Sports Physiology and Performance, 11, 410-413. doi:10.1123/ijspp.2014-0577