Opinion

Quo Vadis CPS? Brief answers to big questions
Matthias Stadler1 and Samuel Greiff2
1Ludwig-Maximilians-University Munich. 2University of Luxembourg.

Research on complex problem solving (CPS) has reached
a stage where certain standards have been achieved,
whereas the future development is quite ambiguous. In
this situation, the editors of the Journal of Dynamic Deci-
sion Making asked a number of representative authors to
share their point of view with respect to seven questions
about the relevance of (complex) problem solving as a
research area, about the contribution of laboratory-based
CPS research to solving real life problems, about the roles
of knowledge, strategies, and intuition in CPS, and about
the existence of expertise in CPS.

Why should there continue to be problem
solving research (in addition to research
on memory, decision-making, motivation
etc.)?

The ability to solve problems (i.e., tasks for which no ap-parent solution is readily available) has, in our view,
become one of the quintessential abilities for both profes-
sional and personal life. By now, machines can complete
most repetitive tasks, leaving humans more time to focus
on creating new knowledge and applying this knowledge
to solve problems. While computers can help us overcome
some of our human limitations (e.g., externalize our mem-
ory or help with decision processes), ultimately we as hu-
mans need to define the problems we want to solve and
find ways to use the appropriate tools and strategies. Re-
search on how people approach problems, why they fail to
solve them, and how they can be supported to succeed in
the future, needs to be continued. However, the field is
also in need of either a clear delineation to other, often
overlapping, fields such as dynamic-decision making or of
stronger efforts to synthesize adjacent fields to see what
problem solving research can learn from fields such as de-
cision making and vice versa.

What are the connections between current
CPS research practice and real problems?
Where do you see potential for
development towards stronger relations?
Most of the current research on CPS focusses on complex
systems with only few variables or that, in some way, do not
fully resemble the complexity of the real world (Greiff, Fis-
cher, Stadler, & Wüstenberg, 2015). Despite the justified
criticism that reality is far more complex, this limitation in
contemporary assessment instruments might still be appro-
priate to represent “real-world” problem solving. If, for ex-

ample, your cat is sick it is certainly appropriate to identify
everything it ate (few variables) and then systematically
rule out potential causes of the illness. Obviously, some
problems are either too complex to fully understand the
influence of each individual variable, not stable enough to
actually specify any consistent rules, or there is not enough
time to explore the system comprehensively. Such systems
were used to study CPS in the field’s “early days” with the
aim of emulating “real world” problems as closely as pos-
sible. We argue, though, that most people deal with sick
cats more frequently than they become almost omnipotent
rulers of midsized cities (the scenario of one of the most
famous CPS tasks; Dörner, Kreuzig, Reither, & Stäudel,
1983). While there is a great deal of research on prob-
lem solving in controllable systems (such as the food you
feed your cat), research on uncontrollable systems needs
to be strengthened. For instance, we face the problem of
how to talk to our colleagues during our daily interactions
with them. Telling jokes (i.e., an “input variable”) may
make some people like you more, whereas others may not
appreciate it (i.e., “outcome variables”). Systematically
isolating colleagues to tell them jokes in order to measure
their response is obviously not feasible. However, based on
data that has been generated in the past, we could gener-
ate knowledge and then use this knowledge to solve future
problems. This line of research is exciting and might help
us understand “real-world” problem solving in diverse sit-
uations.

Given the artificiality of the laboratory
situation, do participants really adopt the
presented problems? What insights can be
gained despite this artificiality and which
cannot?

In our experience, the artificial nature of the problem situ-
ation is not problematic as long as the cognitive (and non-
cognitive) processes involved are the same. There seems to
be no reason to assume that a person who is able to solve a
problem in a laboratory situation will not be able to solve a
similar problem in a more naturalistic situation. Examples
come from the fields of both medical, military, and teacher
training where complex skills are usually trained and as-
sessed using simulations (for an overview see Chernikova,
Heitzmann, Fink, Timothy, Seidel, T., & Fischer, 2019).

Corresponding author: Matthias Stadler, Ludwig-Maximilians-University
Munich, Geschwister-Scholl-Platz 1, 80539 Munich, Germany. E-
mail: matthias.stadler@uni-muenchen.de

10.11588/jddm.2019.1.69302 JDDM | 2019 | Volume 5 | Article 13 | 1

mailto:
https://doi.org/10.11588/jddm.2019.1.69302


Stadler & Greiff: Quo Vadis CPS?

What evidence exists for the influence of
other kinds of knowledge besides
structural knowledge on the results of
CPS? Which of these kinds of knowledge
should be examined in future research?

The amount of evidence on the impact of knowledge on
CPS is plentiful. As mentioned above, various fields such
as medical, military, and teacher training use complex sim-
ulations in which participants need to engage in CPS in
knowledge-rich situations. Combining the theories of these
fields with the methodology and theory of the more cogni-
tive research on “knowledge lean” CPS will, in our view,
be one of the most exciting challenges for future research.

What evidence is available for the impact
of strategies (except VOTAT) on the
results of CPS? Which of these strategies
should be examined more closely?

Interestingly, the VOTAT strategy has, by far, received the
highest level of attention when it comes to understanding
strategic behavior in CPS. This is mirrored in the field of
science inquiry in which the very same strategy only with
the different label CVS (control of variables) has received
a similar amount of attention. Moreover, a study based on
the PISA 2012 data found that use of VOTAT in one task
was highly predictive of overall CPS score that required
solving tasks with different strategies (Greiff, Wüstenberg,
& Avvisati, 2015). Thus, VOTAT (just as other strategies)
might not be limited to a specific behavior but also indicate
a more general level of strategic competence. We know
little about what this competence might be even though
some recent studies have looked at other strategic behav-
iors in CPS research (Beckmann, Birney, & Goode, 2017;
Schoppek & Fischer, 2017). An often neglected way for-
ward might be to look at (longer) sequences of behaviors
instead of the single use of strategies using (educational)
data mining techniques to discover those fuzzy relations
(Stadler, Fischer, & Greiff, 2019). Another interesting
topic are heuristics that are needed in complex environ-
ments and that have not been sufficiently focused from
an individual difference perspective (Gigerenzer & Gaiss-
maier, 2006).

Is there intuitive CPS?

–

What distinguishes experts in CPS from
laypersons?

The term “experts” is often employed in the context of
specific domains in which individuals can – partly through
practice and experience – achieve an extremely high level
of competency and/or knowledge with chess experts being
the classical example (Detterman, 2014). Thus, usually
experts are found in specific areas and we are not sure
whether the term equally applies to a broad mental abil-
ity such as CPS. In fact, one would not consider highly
intelligent people as experts in intelligence and a gradual
distinction between different levels of CPS might be more

appropriate (for an example that distinguishes a continu-
ous scale into distinct level for ease of communication, see
the PISA 2012 problem solving assessment; OECD 2010).
Of course, people with high levels of CPS are likely to dif-
fer from people with low levels of CPS, for instance with
regard to fundamental cognitive abilities, meta-cognition,
or the available set of strategies. To the best of our knowl-
edge, there is no research indicating clear qualitative shifts
(e.g., from layperson to expert) beyond what could be de-
scribed in quantitative models of CPS.

Declaration of conflicting interests: The authors de-
clare they have no conflict of interests.

Author contributions: Both authors contributed to
the content of this manuscript. The abstract was added
by the editors.

Handling editor: Andreas Fischer and Wolfgang
Schoppek

Copyright: This work is licensed under a Creative Com-
mons Attribution-NonCommercial-NoDerivatives 4.0 In-
ternational License.

Citation: Stadler, M. & Greiff, S. (2019). Quo
Vadis CPS? Brief answers to big questions. Jour-
nal of Dynamic Decision Making, 5, 13. doi:
10.11588/jddm.2019.1.69302

Published: 31 Dec 2019

References

Beckmann, J. F., Birney, D. P., & Goode, N. (2017). Beyond
psychometrics: the difference between difficult problem solving
and complex problem solving. Frontiers in psychology, 8, 1739.
doi:10.3389/fpsyg.2017.01739

Chernikova, O., Heitzmann, N., Fink, M., Timothy, V., Sei-
del, T., & Fischer, F. (2019). Facilitating diagnostic com-
petences in higher education - A meta-analysis in medical
and teacher education. Educational Psychology Review, 1-40.
doi:10.1007/s10648-019-09492-2

Detterman, D. K. (2014). Introduction to the intelligence spe-
cial issue on the development of expertise: Is ability necessary?
Intelligence, 45, 1–5. doi:10.1016/j.intell.2014.02.004

Dörner, D., Kreuzig, H.W., Reither, F., & Stäudel, T. (Eds.).
(1983). Lohhausen: Vom Umgang mit Unbestimmtheit und
Komplexität [Lohausen: On the handling of uncertainty and
complexity]. Bern: Huber (Retrieved from http://www.verlag-
hanshuber.com/).

Gigerenzer, G. & Gaissmaier, W. (2006): Denken und Urteilen
unter Unsicherheit: Kognitive Heuristiken. In: Funke, J. (Hrsg.):
Enzyklopädie der Psychologie, Bd. C, II, 8. Denken und Prob-
lemlösen, Göttingen 2006, S. 329-374.

Greiff, S., Fischer, A., Stadler, M., & Wüstenberg, S. (2015).
Assessing complex problem-solving skills with multiple com-
plex systems. Thinking & Reasoning, 21(3), 356-382.
doi:10.1080/13546783.2014.989263

10.11588/jddm.2019.1.69302 JDDM | 2019 | Volume 5 | Article 13 | 2

https://doi.org/10.11588/jddm.2019.1.69302
https://doi.org/10.3389/fpsyg.2017.01739
http://10.1007/s10648-019-09492-2
http://dx.doi.org/10.1016/j.intell.2014.02.004
http://www.verlag-hanshuber.com/
http://www.verlag-hanshuber.com/
http://dx.doi.org/10.1080/13546783.2014.989263
https://doi.org/10.11588/jddm.2019.1.69302


Stadler & Greiff: Quo Vadis CPS?

Greiff, S., Wüstenberg, S., & Avvisati, F. (2015). Computer-
generated log-file analyses as a window into students’ minds?
A showcase study based on the PISA 2012 assessment of
problem solving. Computers & Education, 91, 92-105.
doi:10.1016/j.compedu.2015.10.018

OECD. (2010). PISA 2012 assessment and analytical framework:
Mathematics, reading, science, problem solving and financial lit-
eracy. Paris: OECD. doi:10.1787/9789264190511-6-en

Schoppek, W., & Fischer, A. (2017). Common process demands
of two complex dynamic control tasks: transfer is mediated
by comprehensive strategies. Frontiers in psychology, 8, 2145.
doi:10.3389/fpsyg.2017.02145

Stadler, M., Fischer, F., & Greiff, S. (2019). Taking a closer
look: An exploratory analysis of successful and unsuccessful
strategy use in complex problems. Frontiers in psychology, 10.
doi:10.3389/fpsyg.2019.00777

10.11588/jddm.2019.1.69302 JDDM | 2019 | Volume 5 | Article 13 | 3

https://doi.org/10.1016/j.compedu.2015.10.018
http://dx.doi.org/10.1787/9789264190511-6-en
https://doi.org/10.3389/fpsyg.2017.02145
https://doi.org/10.3389/fpsyg.2019.00777
https://doi.org/10.11588/jddm.2019.1.69302