Meta-Psychology, 2021, vol 5, MP.2020.2539
https://doi.org/10.15626/MP.2020.2539
Article type: Commentary
Published under the CC-BY4.0 license

Open data: Not applicable
Open materials: Not applicable

Open and reproducible analysis: Not applicable
Open reviews and editorial process: Yes

Preregistration: No

Edited by: Rickard Carlsson
Reviewed by: Brand, C.O., Martin, S.R.
Analysis reproduced by: Not applicable

All supplementary files can be accessed at OSF:
https://doi.org/10.17605/OSF.IO/JGXK7

Comparing Bayesian Posterior Passing with
Meta-analysis

Joshua Pritsker
Purdue University

Abstract
Brand, von der Post, Ounsley, and Morgan (2019) introduced Bayesian posterior passing as an alternative to tra-
ditional meta-analyses. In this commentary I relate their procedure to traditional meta-analysis, showing that pos-
terior passing is equivalent to fixed effects meta-analysis. To overcome the limitations of simple posterior passing,
I introduce improved posterior passing methods to account for heterogeneity and publication bias. Additionally,
practical limitations of posterior passing and the role that it can play in future research are discussed.

Keywords: bayesian updating, posterior passing, meta-analysis

Introduction

The ability to accumulate evidence across studies is
often said to be a great advantage of Bayesian infer-
ence. This point was recently discussed by Brand, von
der Post, Ounsley, and Morgan (2019), who suggested
that Bayesian posterior passing alleviates the need for
traditional meta-analysis. They performed a simulation
study comparing posterior passing to non-cumulative
analysis and combined analysis of the data from all
studies. However, they lacked a formal theoretical com-
parison of posterior passing to traditional methods of
meta-analysis. This commentary relates posterior pass-
ing to traditional meta-analyses, allowing for one to de-
termine the performance of posterior passing on the ba-
sis of how traditional meta-analytic methods are known
to perform. To avoid some of the pitfalls of posterior
passing, I suggest improved procedures that account
for heterogeneity and publication bias. I address Brand
et al.’s (2019) suggestion that posterior passing avoids
some of the problems of traditional meta-analyses, and
discuss practical limitations in using it as a replacement
for traditional meta-analysis.

Posterior Passing as Meta-analysis

As Brand et al. (2019) suggest that posterior passing
may replace traditional meta-analyses, one might won-
der how posterior passing relates to traditional meta-
analyses. Given that meta-analyses are typically based
on proxy statistics and their standard errors instead of
individual data points, one might consider posterior dis-
tributions generated the same way. We can do this by
using the likelihood of a statistic instead of the likeli-
hood of the full data. For instance, supposing that we
have a parameter θ that we want to make inferences
about, we might construct the likelihood function using
the sampling distribution of an estimate of θ. In stan-
dard meta-analysis, studies are typically summarized
by their estimates and standard errors. If an estimate
was derived by maximizing a likelihood that takes the
form of gaussian function over θ, the estimate and its
standard error fully summarize this likelihood function.
Even in cases where the likelihood function is not fully
described by its maximum likelihood estimate and stan-
dard error, using the estimate and standard error may
be viewed as a second-order asymptotic approximation
to the log-likelihood.

Taking this approach, with θ̂i being the estimate from

https://doi.org/10.15626/MP.2020.2539
https://doi.org/10.17605/OSF.IO/JGXK7


2

study i of θ, we can get the posterior after study i, de-
noted as fi (θ), by:

fi (θ) ∝ π (θ) f (xi | θ) (1a)

= fi−1 (θ) f
(
θ̂i | θ, SE

2
i

)
(1b)

Where π (·) is our prior distribution, xi refers to all
the information of study i, and SEi is the standard er-
ror for θ̂i. Typically, f

(
θ̂i | θ, SE

2
i

)
= φ

(
θ̂i | θ, SE

2
i

)
, where

φ
(
θ̂i | θ, SE

2
i

)
is the density at θ̂i under a normal distri-

bution with a mean of θ and variance of SE2i . Now, we
may expand (1) across studies:

f (θ) ∝ π0(θ)
k∏

i=1

φ
(
θ̂i | θ, SE

2
i

)
(2)

Where π0(θ) is the prior distribution used by the ini-
tial study. Hence, the posterior is proportional to the
initial prior times the product of the likelihoods of the
relevant studies. How does this compare to the poste-
rior distribution given by a traditional meta-analysis? In
a fixed-effects setting (3), they are identical, provided
that the meta-analysis uses the same prior as the ini-
tial study. The standard fixed effects model is that each
θ̂i follows a normal distribution with a universal mean
value of θ, and a variance of SE2i :

θ̂i ∼N
(
θ, SE2i

)
(3)

Then, the posterior density can be constructed by tak-
ing the product of the likelihoods of each estimate:

f f ixed (θ | x) ∝ π(θ)
k∏

i=1

φ
(
θ̂i | θ, SE

2
i

)
(4)

This equivalence also answers some of the questions
brought up by Brand et al. (2019), such as how pos-
terior passing would perform under publication bias.
Now, it is clear that we can use existing studies on tradi-
tional fixed-effects meta-analyses to answer these ques-
tions (e.g., Simonsohn et al., 2014).

In a random effects setting, we assume that each
study samples from a slightly different population, and
these populations have their own parameter values, re-
ferred to as µi, which vary around the true θ value:

µi ∼N
(
θ,τ2

)
(5a)

θ̂i ∼N
(
µi, SE

2
θ̂i

)
(5b)

Then, the likelihood for each estimate is given by
marginalizing out µi:

frandom (θ | x) ∝ π(θ,τ)
k∏

i=1

∫
µ

φ
(
θ̂i | µi, SE

2
i

)
(6a)

×φ
(
µi | θ,τ

2
)

dµi

= π(θ,τ)
k∏

i=1

φ
(
θ̂i | θ, SE

2
i +τ

2
)

(6b)

Where µi is the mean of the population that study
i comes from, τ2 is the variance across populations,
and the prior distribution is now over both θ and τ.
Hence, in a random-effects setting (4), posterior pass-
ing, like (3), will underestimate the posterior vari-
ance. This point was previously noted by Martin (2017).
A random-effects model is typically preferable, as the
fixed-effects assumption of no between-study variance
is implausible in most settings (Borenstein et al., 2010).
The use of the full-data posterior by Brand et al. (2019)
is inconsequential to this result. Hence, although poste-
rior passing will produce consistent point estimates, the
posterior variance may be underestimated.

How can we improve posterior passing?

Incorporating Random Effects

An obvious question to ask at this point is if we can
modify posterior passing to incorporate random effects.
A Bayesian solution is to update τ along with θ, mod-
eling their joint posterior distribution. Using statistic
likelihoods as in the previous section, we can derive the
joint posterior update by marginalizing out µ, just as in
a standard random-effects meta-analysis:

fi (θ,τ) ∝ π (θ,τ) f (xi | θ,τ) (7a)
= fi−1 (θ,τ) f (xi | θ,τ) (7b)

= fi−1 (θ,τ)
∫
µ

f (xi | µi) φ
(
µi | θ,τ

2
)

dµi (7c)

= fi−1 (θ,τ) φ
(
θ̂i | θ, SE

2
i +τ

2
)

(7d)

To get the marginal posterior distribution of θ, one
may integrate out τ, and vice versa. Notably, the pa-
rameter must be tracked across studies to gain evidence
about its value. However, even if previous studies hadn’t
used a random effects model, they can still be added
using their likelihood functions:

fi (θ,τ) ∝ π (θ,τ)φ
(
θ̂i−1 | θ, SE

2
i−1 +τ

2
)

(8)

×φ
(
θ̂i | θ, SE

2
i +τ

2
)



3

Where study i − 1 is the study that hadn’t used a ran-
dom effects model. In the case of the current study be-
ing the first on a topic, no evidence will be gained about
the value of τ, but incorporating it using only the infor-
mation from a subjective prior distribution will nonethe-
less give a more realistic view of the uncertainty of fi(θ).

Addressing publication bias

A second major issue with posterior passing is that
Brand et al. (2019) provide no way to address publi-
cation bias. Without adjustment, posterior passing will
perform identical to a fixed-effects meta-analysis that
completely ignores publication bias. Unadjusted meta-
analyses are known to perform poorly when publication
bias is substantiative, yielding potentially misleading re-
sults (Simonsohn et al., 2014). This problem can be
viewed as one of biased sampling, hence the likelihood
function is given by a weighted distribution as follows
(Pfeffermann et al., 1998):

f
(
xi | publishedi

)
=

Ei
[
p | xi

]
f (xi)

E
[
p
] (9)

=
Ei

[
p | xi

]∫
x

E
[
p | xi

]
f (xi) d xi

f (xi)

Where p are the probabilities of publication. Now,
we need to model E

[
p | xi

]
. Considering that studies are

typically given a dichotomous interpretation (McShane
& Gal, 2017), a realistic option is a simple step function:

E
[
p | xi

]
=

α si
(
θ̂i
)
= 0

β si
(
θ̂i
)
= 1

(10)

Where si(θi) is a function that gives the standard in-
terpretation of θ̂i in a pass/fail manner, such as its p-
value being below 0.05. However, it is unclear if di-
chotomization exists to the same extent in Bayesian
studies as it does in frequentist studies. In such a con-
text one may replace (10) with a smoother model, such
as a logistic one:

E
[
p | xi

]
= logistic

(
α+βsi

(
θ̂i
))

(11)

Where si
(
θ̂i
)

could represent a Bayes factor cutoff or
similar. For a review of other models that have been sug-
gested, see Sutton, Song, Gilbody, and Abrams (2000).

In any case, the posterior update is now as follows:

fi (θ,τ,α,β) ∝ π (θ,τ,α,β) f (xi | θ,τ,α,β) (12a)
∝ π (θ,τ,α,β) (12b)

×
E

[
p | xi,θ,τ,α,β

]
E

[
p | θ,τ,α,β

] f (xi | θ,τ)
= fi−1 (θ,τ,α,β) (12c)

×
E

[
p | xi,θ,τ,α,β

]
E

[
p | θ,τ,α,β

]
×

∫
µ

f (xi | µi) φ
(
µi | θ,τ

2
)

dµi

= fi−1 (θ,τ,α,β) (12d)

×
E

[
p | xi,θ,τ,α,β

]
E

[
p | θ,τ,α,β

]
×φ

(
θ̂i, | θ, SE

2
i +τ

2
)

As with τ, multiple studies are needed to identify α,
and β. Hence, inference in early studies will be highly
dependent on the prior distribution for these parame-
ters, which should not be uninformative. However, this
problem dissipates as evidence for these parameters ac-
cumulates.

Including Studies Outside of the Posterior Passing
Chain

In meta-analysis, extensive literature searches are
conducted to avoid systematically excluding any stud-
ies. However, it may not be obvious how one can in-
corporate studies outside of a posterior passing ‘chain,’
a sequence of studies where each study’s prior is equal
to the previous study’s posterior, into our prior distribu-
tion. The same problem occurs if two studies are done
simultaneously, creating a fork in the chain. Brand et
al. (2019) suggest that a normal prior with variance
representing our uncertainty and with a mean at the
estimate given by the study may be used. However, it
can be difficult to determine such a variance, and this
procedure only makes sense if we are the first study in
a chain aiming to get information from an unchained
study. A better answer is to include the study’s likeli-
hood function in our posterior derivation. When there
are simultaneous studies, simply take one’s posterior as
the prior and use the likelihood from the other as if it
were outside of the chain altogether. Switching back to
the simple fixed effects procedure for conciseness, this
gives us:

fi (θ) ∝ π (θ) f (xi−1 | θ) f (xi | θ) (13)

To include more studies, one simply needs to add
more f (xn | θ) functions. As a side note, the nature of



4

this function yields an obvious option for Bayesian-style
updating in a non-Bayesian framework:

Li (θ) ∝ f (x1, . . . , xi−1 | θ) f (xi | θ) (14)

This may simply be interpreted as the likelihood func-
tion for θ across all the included studies. Inferences can
then be made using standard frequentist sequential trial
methods (cf. Wetterslev et al., 2017).

Further comments and discussion

Is Posterior Passing a Practical Replacement for
Meta-analysis?

Brand et al. (2019) suggest that posterior passing can
replace traditional meta-analyses. Indeed, the improved
posterior passing procedures introduced in the previous
section can compete with traditional meta-analyses, but
is it practical? To match the quality of traditional meta-
analyses, one would have to meet the same conditions,
including extensive literature search and the inclusion
of all available studies. Furthermore, most studies are
currently done in a frequentist manner, and splits can
occur in a chain due to simultaneous studies, so in prac-
tice each study in the chain will have to do its own mini
meta-analysis. At that point, it might be preferable to
just do traditional meta-analyses.

Brand et al. (2019) also suggest that posterior pass-
ing can solve the problem of conflicting meta-analyses
by updating evidence in real time. Meta-analyses can
provide contradictory results due to a number of rea-
sons, such as differing inclusion criteria and using dif-
ferent methods. However, posterior passing only ap-
pears to be able to mitigate differences occurring by
meta-analyses occurring at different times rather than
by different methods. In this case though, one would
simply go with the most recent meta-analysis. How-
ever, when multiple meta-analyses disagree, they of-
ten have differences in statistical methodology or in-
clusion criteria. Replacing meta-analyses with posterior
passing would likely result in multiple posterior passing
chains to reflect this disagreement. Perhaps this might
be avoided if fields are sufficiently vigilant in prevent-
ing conflicting chains, but the same would be true of
traditional meta-analytic conflicts. In fact, one might
argue that for any criticism of a meta-analysis, one
could seemingly make an equivalent criticism of a poste-
rior passing chain. Instead of having disagreeing meta-
analyses, we would instead simply have a number of
individual studies in disagreement. It may be that pos-
terior passing could help mitigate disagreements of this
nature by the fact that inclusion criteria could change
with any study in the chain. However, having such fuzzy
inclusion criteria is clearly undesirable as it would lead

to results with an unclear interpretation. Hence, pos-
terior passing does not appear to avoid the problem of
conflicting meta-analyses in a desirable manner. The
same applies for all methodological limitations of stan-
dard meta-analyses, such as being impacted by publi-
cation bias, as posterior passing and meta-analysis are
mathematically equivalent.

Alternative Roles for Posterior Passing

Even if posterior passing cannot generally replace
traditional meta-analyses, it may nonetheless be use-
ful. With the improvements suggested above, posterior
passing can replace traditional meta-analysis in areas
where meta-analyses are unlikely to produce conflict-
ing results in the first place. An alternative to creating
posterior passing chains that still utilizes the posterior
passing mechanism is to use it in meta-analyses. This
doesn’t solve the issue of conflicting meta-analyses, but
has practical advantages. By using the posterior distri-
bution of the last similar meta-analysis as a prior distri-
bution, meta-analyses can be performed in chunks in-
stead of having to redo the entire meta-analysis with
each update. Similarly, instead of using posterior pass-
ing chains, studies can use posterior distributions from
meta-analyses as their priors to get accurate net effect
estimates within each study. This allows for broader
conclusions than would otherwise be warranted by the
study alone. A particularly relevant case for this is large-
scale replication projects, where the prior can be gotten
from the last meta-analysis. This yields a readily inter-
pretable lower-bound on the extent to which a field’s
view on a topic should shift as the result of the replica-
tion effort, by providing the change in posterior assum-
ing that previous studies had been conducted properly.
Hence, although posterior passing may have problems
as a replacement for meta-analysis, it can have utility
regardless.

Author Contact

The corresponding author may be contacted at
jpritsk@purdue.edu, ORCiD 0000-0001-9647-6684.

Conflict of Interest and Funding

No conflicts of interest declared.

Author Contributions

Pritsker is the sole author of this article.

Open Science Practices

This article is a commentary and had no data or ma-
terials to share, and it was not pre-registered. The en-

mailto:jpritsk@purdue.edu
https://orcid.org/0000-0001-9647-6684


5

tire editorial process, including the open reviews, are
published in the online supplement.

References

Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein,
H. R. (2010). A basic introduction to fixed-
effect and random-effects models for meta-
analysis. Research synthesis methods, 1, 97–111.
https://doi.org/10.1002/jrsm.12

Brand, C. O., Ounsley, J. P., van der Post, D. J., & Mor-
gan, T. J. H. (2019). Cumulative science via
bayesian posterior passing. Meta-psychology, 3.
https://doi.org/10.15626/MP.2017.840

Martin, S. (2017). Open peer review by Stephen Martin.
Meta-psychology: Decision Letter for Brand et al.
https://doi.org/10.17605/OSF.IO/C4WN8

McShane, B. B., & Gal, D. (2017). Statistical significance
and the dichotomization of evidence. Journal of
the American Statistical Association, 112, 885–
895. https://doi.org/10.1080/01621459.2017.
1289846

Pfeffermann, D., Krieger, A. M., & Rinott, Y. (1998).
Parametric distributions of complex survey data
under informative probability sampling. Statis-
tica Sinica, 8, 1087–1114.

Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014).
P-curve and effect size: Correcting for publica-
tion bias using only significant results. Perspec-
tives on Psychological Science, 9, 666–681. https:
//doi.org/10.1177/1745691614553988

Sutton, A. J., Song, F., Gilbody, M., S., & Abrams, K. R.
(2000). Modelling publication bias in meta-
analysis: A review. Statistical Methods in Med-
ical Research, 9, 421–445. https://doi.org/10.
1177/096228020000900503

Wetterslev, J., Jakobsen, J. C., & Gluud, C. (2017). Trial
sequential analysis in systematic reviews with
meta-analysis. BMC medical research methodol-
ogy, 17. https://doi.org/10.1186/s12874-017-
0315-7

https://doi.org/10.1002/jrsm.12
https://doi.org/10.15626/MP.2017.840
https://doi.org/10.17605/OSF.IO/C4WN8
https://doi.org/10.1080/01621459.2017.1289846
https://doi.org/10.1080/01621459.2017.1289846
https://www.jstor.org/stable/24306526
https://doi.org/10.1177/1745691614553988
https://doi.org/10.1177/1745691614553988
https://doi.org/10.1177/096228020000900503
https://doi.org/10.1177/096228020000900503
https://doi.org/10.1186/s12874-017-0315-7
https://doi.org/10.1186/s12874-017-0315-7

	Introduction
	Posterior Passing as Meta-analysis
	How can we improve posterior passing?
	Incorporating Random Effects
	Addressing publication bias
	Including Studies Outside of the Posterior Passing Chain

	Further comments and discussion
	Is Posterior Passing a Practical Replacement for Meta-analysis?
	Alternative Roles for Posterior Passing
	Author Contact
	Conflict of Interest and Funding
	Author Contributions
	Open Science Practices