JISIB-vol-12_Nr-3(2022).pdf


Journal of Intelligence Studies in Business 
Vol. 12 No. 3 (2022) 
Open Access: Freely available at: https://ojs.hh.se/

pp. 54–65

Does more intelligent trading strategy win? 
Interacting trading strategies: an agent-based 
approach

Burc Ulengin

ABSTRACT:
The market is populated with agents having different trading strategies and they are let to interact 
with each other. Agents differ in the trading method they use to trade, and they are grouped 
as noise, technical, statistical analysis, and machine learning traders. The model is validated 

shape of probability density function, volatility clustering and absence of autocorrelation in 
asset returns. The wealth dynamics for each agent group is analysed throughout trading 
period. Agents with a higher time complexity trading strategy outperform those with strategy 

KEYWORDS: 

1. INTRODUCTION

The World Bank statistics reveal that the mar-
ket capitalisation of all listed companies on 
stock exchanges in the world reaches a total 
of 94 trillion US dollars in 2020.1 There have 
broad range of studies aiming to explain 
dynamics of asset prices and model this com-

the capital market theory for asset pricing 

assumption were the most common approaches 
used. These approaches assume that prices are 

-
-

tions have been challenged by both empirical 

Therefore, alternative approaches have been 
introduced, Kahneman and Tversky (1979) 
proposed the prospect theory as a part of 

irrationally assess gain and losses asymmet-
rically. Cont (2001) also present a set of styl-

be explained by these traditional approaches. 
In this sense, agent-based models (ABMs) are 
introduced as a “paradigm shift” with more real-
istic assumptions as boundedly rational agents 
with heterogenous expectations. ABMs offer 

as emergent behaviour of system as result of 
interaction among system entities. Therefore, 



55

ABMs draw a wide attention and Jean-Claude 
Trichet, the former ECB president, writes that 
“We need to deal better with heterogeneity 
across agents and the interaction among those 
heterogeneous agents”.4

An ABM is a simulation to model a system 
consisting of interacting agents. Agents can 
have static or adaptive rules to initiate their 
interactions with other agents and environ-
ment. It has great importance in terms of pro-
viding bottom-up understanding of systems. 

model the interactions among market entities 
and agents can also apply range of sophisticated 
learning capabilities especially when continu-
ous adaptation exists.5 -
spective, traditional models fall short to explain 
the behaviour of market through extreme situ-

6,7 since there is no 
such classical approach to capture behaviour 
of crashing markets. In this sense, ABMs can 
capture such extreme moves when built with 
necessary components and optimal parameter 
calibrations.

Simulating stock markets has been growing 

market mechanism, wealth dynamics and price 
dynamics.9,10,11,12 The seminal paper of the Santa 

13

14,15,16 
These models are differing in the way they 
set the market microstructure, agents trading 
strategies, network among agents and intelli-
gence level in agents. A review of ABMs and 

found in the literature.9,17,18,19 The main stud-

requires a proper design and four main design 
elements are needed: market mechanism, trad-
ing strategies, traded assets and trader types. 
The built model is subject to be validated by 
measures of modelled market. 

The validation is the key part of ABMs since 
it ensures the appropriateness of the simulation 

-
cial market model is measured by the ability of 
reproducing stylized facts observed in the real 
market.3 Another approach for validation is to 
use modelled market parameters.20,21,22 Llacay 
and Peffer (2018) use face validation differing 
from the mainstream. The stylized facts in 

-
erature and they are -
relation23 24,25

3

There is no simulation model can 

reproduce all known facts due to increasing 
complexity of model, hence models are kept 
simple in compliance to Ockham’s razor prin-
ciple which asserts to use minimal entity for 
explanations. 

The trading strategies agents employ play 

simulation model.17 These strategies can range 
from zero-intelligent agents26 to very intelli-
gent agents compared to earlier studies.27 In 
a recent study, Llacay and Peffer (2018) used 
agents with realistic trading strategies that 
takes historical price into account. The method 
used to take trade action mainly relies on 
future price forecast which can be any method, 
for example, evolutionary techniques such as 

-
works. Agents can also employ social learning 
method where agents observe other traders and 
change their strategy accordingly.9,28
this may lead a herding behaviour in the mar-

the herding behaviour as a reason for bubbles 

Considering main components of agent-

methods are main agent diversifying compo-
nent in the model. In this sense, considering 
existing studies, there are a few studies that 
takes realistic agent trading strategies since 
the earlier studies mainly employ agents with 
zero-intelligent and agents using fundamental 
value and genetic algorithms. In this study, we 

more realistic technical and fundamental trad-
ing strategies as well as machine learning 
approaches. The methods our agents use have 
been studied in the literature for price pre-

Autoregressive Integrated Moving Average 
(ARIMA) and Nelson et al. (2017) used Long 
Short-Term Memory (LSTM) as predicting 
method. On the other hand, Llacay and Peffer 
(2018) applied some realistic technical trad-

the most of prediction methods use historical 
data and do back testing to measure the suc-

method interaction with market environment, 
and this assumes no price impact in the mar-
ket. Considering this fact, we equipped our 
agents with realistic trading strategies and let 
them to interact with all market entities. With 
this, the agent’s market effect is considered, 
and the model provides an insight into wealth 
dynamics of interacting agents. The model pro-



56

market hyper-parameters such as price tick 
size.

We extend the GASM model by adding 
interacting intelligent agents and analyse mar-
ket dynamics and wealth dynamics. We aim to 
make four main contributions to the agent-

(1) reproduction and validation of the GASM 

strategies which are commonly used by practi-
tioners (3) we analyse wealth dynamics of agent 
types hence, the effect of intelligence level on 

noise traders in the market.
The rest of the paper is structured as fol-

low: Section 2 presents our simulation model. 
In Section 4, simulation results are given. 

concludes the study.

2. PROPOSED MODEL

-
lar microstructure with GASM model, for 
a detailed description of the model structure.40 
The herding behaviour phenomena is mod-
elled different from GASM model. Agents form 

cluster is activated with a given probability 
that all agents belong to the cluster are either 
seller or buyer. 

2.1 Trader Types

on behalf of another parties. Traders vary in 
perceiving the market, they therefore employ 
different strategies for trading. At this point, 
the market theories come into account and 
help traders to see different beliefs about these 
complex systems. There are several studies 

prices cannot predict the future prices while 
Brock et al. (1992) and Kwon and Kish (2002) 
evidence that technical trading rules can beat 

to this, statistical methods such as ARIMA30 
and LSTM31 are used to predict future stock 
price for trading. In this sense, an environment 

-
erogeneity of traders in real market. The liter-
ature in testing trading methods usually take 
a strategy as a baseline and do back testing to 
compare performances. Therefore, agent-based 

not possible to mimic the entire complex real 
market dynamics. 

market is populated with six types of agents 
who are named as the method they are equipped 
with: Noise, Moving Average Convergence 
Divergence (MACD), Relative Strength Index 
(RSI), Bollinger Bands, ARIMA and LSTM. 
Agents will be named with the method they 

the amount of assets (cash) to be traded is ran-
dom fraction of assets(cash) and the limit price 
is a draw from a interval that is attached to 
historical volatility. Agents rely on their signal 
function when taking trading decision.

 have a great importance 
in keeping the market working since they act 
as a catalyser in the market and supply volume 
for intelligent traders.8,35  is considered 
as a momentum indicator that gives signal of 
overbought or oversold. The method is devel-
oped by Wilder (1978) and the RSI value range 
from 0 to 100 and the RSI value is regarded 
as overbought if it is above 70 while it is over-
sold when it is below 30.  is a tech-
nical trader tool developed by Gerald Appel in 
late 1970s. It is mainly based on exponential 
moving average (EMA) which is a type of mov-
ing average that takes the more recent data 
points the greater weight.  
is a technical trader tool developed by John 
Bollinger in 1980s. It is volatility measure 
indicator that relies on the past price of asset 
and its volatility. The agents using ARMA(p, q) 
forecast with ARMA model is computed recur-
sively. The ARIMA model use integrated data 
by differencing the raw data to meet the time 
series stationary. The ARIMA traders checks 
stationarity of stock price and do differenc-
ing till obtain a stationary series. The traders 
estimate ARIMA models with different lags to 

p and q
the model with minimum Akaike information 
criterion (AIC). The forecast price values are 
predicted and that is fed into a decision-mak-
ing process.  is recurrent neural net-

Schmidhuber (1997). It is a machine learning 
method with deep networks and differs from 
feedforward neural networks with feedback 
connections since it can process sequences of 
data. The LSTM is widely used in predicting 
stock price movement and outperform baseline 
approaches.31,39 The LSTM traders use simu-
lation initialisation period stock price return 



57

to predict following 5-periods return so post 
orders accordingly. 

3. MARKET INITIALISATION

At the beginning of simulation, the stock price  
p0 is set to be $100. The wealth is equally dis-
tributed among agents,  each get 1000 stock 
(inventory) and $100000 cash. The hyperpa-
rameters for market is set before simulation 
run as in Table 1. 

There are total of 550 agent population of 
which 500 noise traders and 10 for each of RSI, 
MACD, Bollinger, ARIMA and LSTM trad-
ers. The tick size for asset price is one cent. 
Marchesi et al. (2003) extended the GASM 
model by populating the market with four 
different agents. Like this study, the most of 
agents are noise traders that enables the order 
matching mechanism working. The simulation 
time steps refer a trading day and simulation 
is consist of 5040 days which is approximately 

20-year trading period since a year has average 

Agents are in a partially observable envi-
ronment since they only can access asset price. 
Agent types use technical trading indicators, 
statistical model for time series, and a machine 
learning, deep learning. All intelligent agents 

they rely on signals for the forecast period. 
The stock market is closed form since there is 

The total wealth of agent  agent at time 
step  can be calculated as = + , 
where   and  are the cash amount and assets 
of  agent at time step  and pt asset price. 

-
wise. The wealth of a trader changes through-
out simulation as a result of their interactions. 
The actions within market environment are 
based on the strategy trader employ to take 
buy or sell action. Building these strategies 
rely on the parameters that emulate realistic 
trading strategies, which is given in Table 2. 

Market initial parameters.

Market Parameters Value Description
N 550 Total number of agents
T 5240 Simulation time steps
PAC 0.001 Probability that agents create a cluster
PCA 0.002 Probability that cluster is activated
BP 0.5 Buy probability of noise traders
SMu 1.01 Mean of sell limit orders 
SSK 4.5 Sell sigma K
BMu 1.01 Mean of buy limit orders
BSK 4.5 Buy sigma K
Agent Population [500, 10, 10, 10, 10, 10]  

[Noise, RSI, MACD, Bollinger, ARIMA, LSTM]



58
Agent initial parameters.

Agent Type Parameters Values Description

Noise [p ] [0,5] Buy probability

RSI r1 ] [14, 30, 70] Periods of RSI, buy signal threshold, sell signal threshold
MACD m1 m2 m3 [12, 26, 9, 2/(n+1) EMA(p), EMA(p), EMA(MACD) periods and smoothing constant
Bollinger [ b] [20, 2] Periods and constant k

ARIMA [p, d, q] [ [1], [0,1], [1] ]
p, d, q are lag order of AR, degree of differencing 
and MA window size, respectively. p, q take an 
integer value 1 and 2, depending on model selection 
AIC criteria. d is mainly 0 or 1. 

LSTM Optimiser, Epochs, 
LearningRate]

[20, “adam”, 50, 1, 
0.005]

input and output layers. “Adam” is an optimiser 
for training deep neural networks. Epochs is 
the number of learning algorithm works through 
the training set. LearningRate is the step size 

function.

2. SIMULATION MODEL AND  
 RESULTS

In this section, the extended GASM model is 
simulated, and the result of the experiments 

to trade in the market for a given initialisation 
period hence, initial stock price is generated. 

different traders who are called “intelligent” 
agents since those agents predict future price 
move. The market behaviour emerges under 
agent interactions. 

The simulation is run with 500 noise trad-
ers and 10 intelligent traders for each method. 
Since the amount of asset to trade is a random 
friction of agent’s wealth, having 10 agents for 
each method will decrease the effect of ran-
domness on average. Several simulations with 
same parameters were run and all give simi-
lar outputs. Therefore,  results here are a rep-
resentative simulation model for those series 

The model keeps the GASM main structure, 
however, some parameters are tuned after sev-
eral experiments and intelligent agents are 
added to the market. The population share 
of traders in the market are determined with 
experiments. A market with more than 10% 
of intelligent agent population leads stock  Stock Market Simulation Loop Structure.



59

price jumps and halt in price formation pro-
cess. The decision-making process is two part 
which are trading decision and the amount to 
trade. The amount to trade is random fraction 

-
sion depends on the method agents use, trad-
ing signal functions is summarised in Table 
3. It shows the tuning options on parameters 
for agent trading methods hence, mostly used 
realistic trading parameters are used to condi-
tion realistic trading strategies.

2.1 Price, return and volume analysis

approaches have assumption that stock returns 

than normal distribution.3 In addition to this 
-

acteristics that are well documented in Warner 
and Brown (1985). Therefore, the price and 
other emergent features of simulated market 

Agent decision estimation window and decision making.

Traders Estimation Period (day(s))
Forecast Period 

(day(s))
Buy Rule

(If …)
Sell Rule

(If …)

Noise - -   
RSI 14 1 RSI RSI
MACD 9, 12, 26 1
Bollinger Bands 20 1 SMA SMA
ARIMA t
LSTM t

 Market outputs over simulation time. Upper left panel: Asset price. Upper right panel: asset price log return. Lower left 
panel: asset price log return density distribution. Lower right panel : traded volume.



60

are supposed to exhibit these characteristics 
alongside stylized facts. 

The price, return and volume outputs are 

of price is expressed as returns in the rest of 
this paper. 

 Asset price and return related descriptive statistics.

Statistics Price Returns

Mean 81.98 0.000

Standard Deviation 6.58 0.0091    

Minimum 65.12 -0.0702   

Maximum 101.80 0.0754    

Skewness -0.33 -0.5172   

Kurtosis 2.09 9.5849   
Augmented Dickey- -0.6579 -87.2366

Descriptives of price returns are in line 
with real world stock return features which 
has zero mean and have heavy-tailed distri-
bution. The distribution is leptokurtic and left 
skewed with 11.65 kurtosis and -0.769 skew-
ness measure. The price is not-stationary at 

-
lation parameters are tuned for different com-
binations of market and agent parameters. 
The most striking result is that increasing pop-
ulation of intelligent agents halts price forma-
tion so the market.

2.2 Validation

market model is measured with the number of 
stylized facts the simulation model is capable 
to reproduce. The validity of our built model 

-
cial market features. As a seminal work, Cont 
(2001) documented a list of stylized facts for 

markets have reproduce some these stylized 
facts but not all of them, so do ours. In addition 
to all market microstructure parameters, there 
are also six different types of agents interact-
ing which increase the complexity of the stock 
market. The validation process is conducted for 
each fact given in Cont (2001).

Return autocorrelations
It is empirically showed that autocorrelation 

time scales could be exception.3 There would 
be a price to be exploited otherwise, and this 

-

function (acf) values for simulation generated 
asset price returns indicates that there is a sta-

afterwards. This is more like intraday small 

The slow decay behaviour in absolute return 
autocorrelation function is another real mar-

feature that is measured by squared returns 

 Return related autocorrelations. Left panel: return autocorrelation function. Right panel: Return partial autocorrelation 
function. 



61

with Ljung-Box Q-test. The test results show 
that there is an autocorrelation in squared 
return with test values [critical values] of 
1960.22 [11.07], 2155.86 [18.31] and 2176.27 
[24.99] for lag 5,10 and 15, respectively. This 
is a sign of long dependence of volatile mar-
ket conditions so the conditional volatility 
behaviour.

Volume/return corelations
It is expected to asset return has negative cor-
relation with volume, however the simulation 
output short fall to meet this feature since 
the calculated correlation is  = 0,03. Another 
as negative correlation between return and 
change in volatility. The simulation output was 
able to reproduce a weak  with 

. The validity of our model with styl-
ized facts is summarised on Table 5. 

Testing all stylized facts given in Cont 
(2001) for asset price and volume outputs from 
simulation show that the model can replicate 
real market features and they  are summarized 
in Table 5. 

2.3 Wealth analysis

The literature in testing trading strategy 
methods relies on back testing mostly where 
the agent is assumed to have no market impact 

on market dynamics since they interact with 
market participants. This study aims to create 
a stock market testbed where agent interaction 
is considered, hence variety of sensitivity analy-
sis can be applied. Satisfying some real market 
stylized facts, the agent-based model is capable 
of generate real market features. Therefore, 
the market is populated with different types of 

partial autocorrelation function. 

List of stylized facts for asset returns that is used for simulation model validations.

Stylized fact Testing Does our model meet?

Absence of return autocorrelations Autocorrelation plot Partially

Slow decay of autocorrelation in absolute returns Autocorrelation plot

Squared return autocorrelation plot 

Aggregational Gaussianity Skewness and Kurtosis No

Corelation No

Leverage effect Corelation Partially



62

agents who compete to increase their wealth at 
the end of trading period. 

One of the question this study aims to 
answer is if computationally intelligent agents 
can beat the overall market. In the light of this 

with the signal they receive. The rules agent 
use to trade were summarised in Table 3. 
Based on these rules, agents entered market 
and start to trade.  The average wealth of agent 

The agent named LSTM, which is a deep 
learning method, outperforms other agents 
by far. LSTM method is the most complicated 
and computationally costly method among 
others. Computation power can be considered 
as intelligence level in an interacting agent 
market. Therefore, it can be concluded that 
the more computational power the higher 
return. The number of days agents take long, 
and short positions is summarised in Table 5. 

Average number long and short positions over 
trading period.

Traders Long positions Short positions

Noise 2269 2266

RSI 120 131

MACD 368 369

Bollinger 125 126

ARIMA 127 4400

LSTM 2531 1811

Two agent group RSI and Bollinger are 
reluctant to take position since there is no 
up-down pattern in price long run. ARIMA and 
LSTM trade most of time since they take posi-
tion based on their future price move predic-

agent types, agent wealth differs statistically 

of agent type pairs was tested at 1%, except 
Noise-MACD agent pairs, the rest of 14 pairs 
has different wealth over the trading period. 
A boxplot for each agent group is created that 

Although all agents belong to the same 
group use the same trading method, they dif-
fer in the amount to trade at each trading 
decision. Therefore, randomness in amount to 
trade decision give advantage to some traders. 
In this sense, each group has at least ten mem-
bers and distribution checked at initial and 

homogenous. To measure this, the Gini coef-

inequality in wealth that ranges from 0 to 100. 

increase in it is a sign of inequality in wealth 
distribution. At the beginning of simulation 
all agents were endowed with same amount of 

-

small inequalities occur during trading period 

 Average wealth of agent types in cash throughout trading period.



63

type agents were kept, and it remains stable at 

is a measure of wealth inequality, the outliers 
-

3. DISCUSSION AND CONCLUSION

The study aims to gain a better understand-
ing of trader interaction in stock markets 
and reproduce real market price features.  

approach is employed to serve the purpose of 
this study since it takes agents’ market impact 

into account. The model was able to reproduce 
real market “stylized facts”, thus it is eligible to 

were able to equip agents with realistic trading 

into rivalry of different intelligence level in 
agents and supporting evidence to dominance 
of computationally powerful agents. It is  evi-
dent that agent using deep learning approach 
get the highest return among others with 
the highest time complexity method.

with agent groups using no  trading strat-
egy,  RSI, MACD, Bollinger, ARIMA and 
LSTM methods. Catalyser effect of noise trad-
ers is tested as the increase in population of 



64

intelligent agents halts market and that is 
-

ligence in agents helps market to move and 
provide liquidity to the market. 
are also in line with back testing on real data, 
Siami-Namini et al. (2018) compares perfor-
mance of ARIMA and LSTM methods where 
the LSTM trader outperforms. This is also 
can be taken as validity measure whereas 
Llacay and Peffer (2018) use also face valida-
tion and sensitivity analysis to validate their 
market model extended with realistic trading 
strategies.

Our results are consistent with the previ-
ous work of Raberto et al. (2001) and Marchesi 
et al. (2003) since it reproduces its results. 
Although it is challenging to represent com-

model can still reproduce most of price dynam-
ics.43

components is built and validity of empirical 
-

tic trading strategies compete alongside agent 
interactions in our bottom-up market model. 
The emergent behaviour of the market is 
a result of agent interactions which is hardly 

let agents to interact at micro level and analyse 
the behaviour of market dynamics under dif-
ferent parameter combinations. This can also 
be considered in a game theorical view since 
competence of different strategies resulted in 
price equilibria. Considering these aspects, 

help us to better understand market dynamics 
even in a competing strategies environment. 

There are potential limitations of study 
that heterogeneity in agents is more diverse in 
real markets such as informed and uninformed 

Although our model mimic real market price 
features, fundamental value of an asset is 
the key for major investors and could be added 
as one trader type. A more powerful compu-
tation can ease time complexity of simulation 
when agents with complex trading strategy 
is considered such as deep learning method. 

market parameters, different combination of 
parameters can be applied when modelling 

interest in high-frequency trading and limit 
order book modelling44, therefore there are 
variety of direction to apply machine learning 
tools for future research.

REFERENCES

 1. Market capitalization of listed domestic 
companies. (2020). Retrieved 10 Jan 2022, 
from https://data.worldbank.org/indicator/
CM.MKT.LCAP.CD.

 2. Kahneman, D., Tversky, A. Prospect The-
ory: An Analysis of Decision under Risk. 

 3. Cont, R. Empirical properties of asset returns: 
stylized facts and statistical issues. Quantita-

-
al economics. 4th ed. San Diego: Elsevier 

 5. Macal, C., North, M. Tutorial on agent-
based modelling and simulation. Journal of 

 6. Bouchaud J. Economics needs a scientif-

pp. 1181–1181.

 -

markets. -
–2259.

-
-

10. Chakraborti, A., Toke, I. M., Patriarca, M., 

11. Ponta, L., Scalas, E., Raberto, M., Cincotti, S. 
Statistical analysis and agent-based micro-

-
cial trading. IEEE Journal of selected topics 

12. Llacay, B., Peffer, G. Using realistic trading 
strategies in an agent-based stock market 
model. Computational and Mathematical 

Marchesi, M. Agent-based simulation of 
-

-
tor behavior to market phenomena. Elsevi-
er. 2000.

16. Martinez-Jaramillo, S., Tsang, E. P. An het-
erogeneous, endogenous and coevolutionary 



65

-

17. LeBaron, B. A builder’s guide to agent-
-

-
nancial markets. Reports on Progress in 

Critical overview of agent-based models for 
economics. arXiv preprint arXiv:1101.1847. 
2011.

20. Lux, T., Marchesi, M. Scaling and criticali-
-

498–500.

and the business school with genetic pro-
gramming: A new architecture of the agent-

-
nancial market. Working paper, Graduate 

-
nance, Brandeis University. 2003.

-
dence in daily foreign exchange rates. Jour-

-
al hetero scedasticity with estimates of 

Econo metrica: Journal of the Econometric 

25. Bollerslev T. Generalized autoregressive 
conditional heteroskedasticity. Journal of 

26. Gode, D.K., Sunder, S. ‘Double auction dy-
namics: structural effects of non-binding 
price controls’, Journal of economic dynam-

27. Lux, T., Schornstein, S. Genetic learning as 
an explanation of stylized facts of foreign 
exchange markets. Journal of Mathemati-

of agent-based models: the case of an asym-
metric herding model. Computational Eco-

35–56.
30. Ariyo, A. A., Adewumim A. O., Ayo, C. K. 

‘Stock price prediction using the ARI-
MA model’. 

Simulation
31. Nelson, D. M., Pereira, A. C., De Olivei-

ra, R. A. ‘Stock market’s price movement 
prediction with LSTM neural networks’. 

-

-

33. Brock, W., Lakonishok, J., LeBaron, B. Sim-
ple technical trading rules and the stochas-
tic properties of stock returns. The Journal 

639–653.
35. Palit, I., Phelps, S., Ng, W. L. Can a zero-in-

telligence plus model explain the stylized 
-

International 

Multiagent Systems.  (pp. 653–660).
36. Wilder  JW. New concepts in technical trad-

ing systems.  1978. 

term memory. 

-
dicting stock prices using LSTM. -

-
-

ket: Microstructure and Simulations. Lec-
ture Notes in Economics and Mathematical 

41. Brown, S. J., Warner, J. B. Using daily stock 
returns: The case of event studies. Journal 

42. Siami-Namini, S., Tavakoli, N., Namin, A. S. 
A com parison of ARIMA and LSTM in fore-
casting time series. 17th IEEE international 
conference on machine learning and applica-

-
nancial complexity. . 

-

Get real: Realism metrics for robust limit or-
der book market simulations. In