Dialogue & Discourse 12(2) (2021) 81–114 doi: 10.5210/dad.2021.203

User Satisfaction Reward Estimation Across Domains:
Domain-independent Dialogue Policy Learning

Stefan Ultes STEFAN.ULTES@DAIMLER.COM
Mercedes-Benz AG Research & Development
Sindelfingen, Germany

Wolfgang Maier WOLFGANG.MW.MAIER@DAIMLER.COM

Mercedes-Benz AG Research & Development
Sindelfingen, Germany

Editor: Kallirroi Georgila

Submitted 12/2020; Accepted 07/2021; Published online 09/2021

Abstract
Learning suitable and well-performing dialogue behaviour in statistical spoken dialogue systems
has been in the focus of research for many years. While most work that is based on reinforcement
learning employs an objective measure like task success for modelling the reward signal, we pro-
pose to use a reward signal based on user satisfaction. We propose a novel estimator and show that
it outperforms all previous estimators while learning temporal dependencies implicitly. We show
in simulated experiments that a live user satisfaction estimation model may be applied resulting
in higher estimated satisfaction whilst achieving similar success rates. Moreover, we show that a
satisfaction estimation model trained on one domain may be applied in many other domains that
cover a similar task. We verify our findings by employing the model to one of the domains for
learning a policy from real users and compare its performance to policies using user satisfaction
and task success acquired directly from the users as reward.
Keywords: statistical dialogue management, reinforcement learning, spoken dialogue systems

1. Introduction

Spoken dialogue systems (SDSs) enable voice interaction between technical systems and humans.
They have been advanced into our everyday lives. Prominent examples include Apple’s Siri, Ama-
zon Alexa, or Google Assistant as well as more specialised systems like the in-car voice assistant
HeyMercedes. Spoken dialogue systems that target the fulfilment of a certain task are called task-
oriented and are usually built using a modular pipeline architecture comprising speech recognition,
semantic decoding, dialogue management (consisting of dialogue state tracking and deciding on the
next system action), language generation, and speech synthesis (see Figure 1).

One prominent way of modelling the decision-making component of a spoken dialogue system
is to use (partially observable) Markov decision processes ((PO)MDPs) (Lemon and Pietquin, 2012;
Young et al., 2013). There, reinforcement learning (RL) (Sutton and Barto, 1998) is applied to find
the optimal system behaviour represented by the policy π. Task-oriented dialogue systems model
the reward r, which is used to guide the learning process, traditionally with task success as the
principal reward component (Gašić and Young, 2014; Lemon and Pietquin, 2007; Daubigney et al.,
2012; Levin and Pieraccini, 1997; Singh et al., 2002; Young et al., 2013; Su et al., 2015, 2016).

c©2021 Stefan Ultes and Wolfgang Maier

This is an open-access article distributed under the terms of a Creative Commons Attribution License
(http ://creativecommons.org/licenses/by/3.0/).



ULTES AND MAIER

Policy

State 
Tracking

Interaction Quality 
Reward Estimation

Semantic 
Decoding

Speech 
Recognition

Speech 
Synthesis

Language 
Generation Environment at

st

rt

Figure 1: The modular pipeline architecture of a spoken dialogue system with the focus of its ap-
plication in a reinforcement learning setup and the presented extension of integrating an
interaction quality reward estimator as originally proposed by Ultes et al. (2017a). The
policy learns to take action at at time t while being in state st and receiving reward rt.

The goal of this article is to demonstrate that user satisfaction (US) may be used as a reward
to learn dialogue policies which not only maximise US but also lead to high task success (TS)
rates. We apply two user satisfaction reward estimators (Ultes et al., 2017a; Ultes, 2019) where
one uses a support vector machine (Vapnik, 1995) and relies on handcrafted temporal features. The
other uses a deep learning network that uses long short-term memory (LSTM) cells (Hochreiter
and Schmidhuber, 1997) to learn the temporal dependencies of a multi-turn dialogue interaction
implicitly.

We argue that training a system to maximise US is a good alternative to TS for the following
reasons:

1. User satisfaction is favourable over task success as it represents more accurately the user’s
view and thus whether the user is likely to use the system again in the future. In fact, task
success has only been used as principal reward component as it has been shown to correlate
well with user satisfaction (Williams and Young, 2004).

2. User satisfaction—in contrast to TS—may be linked to interaction phenomena that are inde-
pendent of the user’s goal (Schmitt and Ultes, 2015) and hence, no prior knowledge of the
goal or any other domain-dependent information is required.

3. As user satisfaction is independent of application domain information, the use of an estima-
tor of user satisfaction has the potential to generalise well across domains. Thus, learning
dialogue policies for new, previously unseen domains becomes much easier.

Following up on previous work (Ultes et al., 2011, 2012; Ultes and Minker, 2014; Ultes et al.,
2015, 2019), interaction quality (IQ)—a less subjective version of user satisfaction1—will be used
for estimating the reward. We apply a conventional IQ estimator (Ultes et al., 2017a) using domain-
independent, interaction-related features that do not contain any information about the task or the

1. The relation of US and IQ has been closely investigated by Schmitt and Ultes (2015) and Ultes et al. (2013), see also
Section 3.1.

82



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

goal of the dialogue. This allows the reward estimator to be applicable for learning in unseen
domains. To circumvent the dependency of the convectional estimator on handcrafted temporal fea-
tures, we will additionally present a deep learning-based IQ estimator (Ultes, 2019) that utilises the
capabilities of recurrent neural networks to get rid of all handcrafted features that encode temporal
effects. By that, these temporal dependencies may be learned instead.

The applied RL framework is shown in Figure 1. It has previously been applied for in-domain
experiments and simulated evaluation (Ultes et al., 2019). Within this setup, both IQ estimators are
used for learning dialogue policies in several domains to analyse their impact on general dialogue
performance metrics. Moreover, one estimator is used in an experiment where the policy is learned
through interaction with real humans.

The remainder of the paper is organised as follows: in Section 2, related work is presented
focusing on dialogue learning and the type of reward that is applied. In Section 3, the interaction
quality is presented and how it is used in the reward model. The deep learning-based interaction
quality estimator is then described in detail in Section 3.3 followed by the experiments and results
both of the estimator itself and the resulting dialogue policies in Section 4.

The work described in this article builds upon and extends work published by Ultes (2019) and
Ultes et al. (2017a).

2. Relevant Related Work

Most of previous work on dialogue policy learning focuses on employing task success as the main
reward signal (Gašić and Young, 2014; Gašić et al., 2014; Lemon and Pietquin, 2007; Daubigney
et al., 2012; Levin and Pieraccini, 1997; Singh et al., 2002; Young et al., 2013; Su et al., 2015, 2016).
However, task success is usually only computable for predefined tasks, e.g., through interactions
with simulated or recruited users, where the underlying goal is known in advance. To overcome
this, the required information can be requested directly from users at the end of each dialogue (Gašić
et al., 2013). However, this can be intrusive, and users may not always cooperate.

An alternative is to use a task success estimator (El Asri et al., 2014b; Su et al., 2015, 2016).
With the right choice of features, such a task success estimator can also be applied to new and
unseen domains (Vandyke et al., 2015). However, these models still attempt to estimate completion
of the underlying task, whereas our model evaluates the overall user experience.

In this paper, we show that an interaction quality reward estimator trained on dialogues from a
bus information system will result in well-performing dialogues both in terms of success rate and
user satisfaction on five other domains, while only using interaction-related, domain-independent
information, i.e., not knowing anything about the task of the domain.

Others have previously introduced user satisfaction into the reward (Walker et al., 1998; Walker,
2000; Henderson et al., 2008; Rieser and Lemon, 2008b,a) by using the PARADISE framework ini-
tially proposed by Walker et al. (1997). However, PARADISE relies on the existence of explicit task
success information, which is usually hard to obtain. Furthermore, to derive user ratings within that
framework, users have to answer a questionnaire, which is usually not feasible in real world settings.
To overcome this, PARADISE has been used in conjunction with expert judges instead (El Asri
et al., 2012, 2013) to enable unintrusive acquisition of dialogues. However, the problem of mapping
the results of the questionnaire to a scalar reward value still exists.

A similar measure called response quality has been proposed as a measure to capture user sat-
isfaction as an alternative to interaction quality (Bodigutla et al., 2019b,a, 2020). In contrast to

83



ULTES AND MAIER

the interaction quality, the response quality focuses more on the overal performance of a system,
e.g., including the functionality of back-end services. Thus, it is hard to identify if a negative score
should be attributed to the performance of a back-end service or the interaction itself. Thus, the
response quality is not suitable for learning the behaviour of a dialogue system.

Other research uses different cues for reward estimation. For instance, Misu et al. (2012) inves-
tigate domain-independent dialogue policy learning in the context of question-answering dialogues.
Users are asked to rate dialogues on a Likert scale, then regression is used to compute rewards for
question-answer pairs. As there is no explicit task success metric (their dialogues are not strictly
task-oriented and somewhat similar to chat), their approach is also domain-independent. Shi and Yu
(2018) incorporate user sentiment information obtained from multi-modal cues in both a supervised
and a reinforcement learning setup. Liu and Lane (2018) forego the need for explicit ratings and rely
only on adversarial rewards. In the area of non-task-oriented dialogue, Cuayáhuitl et al. (2018) rely
on deep reinforcement learning and investigates the amount of dialogue history that has to be taken
into account for the estimation of reward, also without relying on explicit ratings.

In this work, we use interaction quality (Section 3) due to the following reasons: interaction
quality models user satisfaction instead of task success that targets the measurement of task com-
pletion. Other than automatically derived measures such as adversarial signals, interaction quality
provides expert ratings. These expert ratings reflect the quality of the interaction itself, with no
dependency on the connected APIs as it is the case for response quality; furthermore, they are eas-
ier to obtain than ratings from mixed sources (Shi and Yu, 2018). Interaction quality furthermore
uses scalar values applied by experts and only uses task-independent features that are valid across
domains and easy to derive, which gives it an edge over the approaches relying on the PARADISE
framework.

3. Interaction Quality Reward Estimation

In this work, the reward estimator is based on estimating the interaction quality (IQ) (Schmitt and
Ultes, 2015) for learning information-seeking dialogue policies. IQ represents a less subjective vari-
ant of user satisfaction: instead of being acquired from users directly, experts annotate pre-recorded
dialogues to avoid the large variance that is often encountered when users rate their dialogues di-
rectly (Schmitt and Ultes, 2015).

The IQ estimation model will be used as a reward estimator as depicted in Figure 1. With
parameters that are collected from the dialogue system modules for each time step t, the reward
estimator derives the reward rt that is used for learning the dialogue policy π.

IQ is defined on a five-point scale from five (satisfied) down to one (extremely unsatisfied). To
derive a reward from this value, the equation

RIQ = T · (−1) + (iq − 1) · 5 (1)

is used where RIQ describes the final reward. It is applied to the final turn of the dialogue of length
T with a final IQ value of iq. A per-turn penalty of −1 is added to the dialogue outcome. This
results in a reward range of 19 (because there is always at least one turn) down to −T , which is
consistent with related work (e.g., Gašić and Young, 2014; Vandyke et al., 2015; Su et al., 2016) in
which binary task success (TS) was used to define the reward as:

RTS = T · (−1) + 1TS · 20 , (2)

84



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

s u s u s u s u…s1 u1 s2 u2 s3 u3 sn un
…

e1 e2 e3 en

Figure 2: A dialogue may be separated into a sequence of system-user-exchanges where each ex-
change ei consists of a system turn si followed by a user turn ui.

where 1TS = 1 only if the dialogue was successful, 1TS = 0 otherwise. RTS will be used as a
baseline.

3.1 Interaction Quality and the LEGO Corpus

Interaction Quality is defined similarly to user satisfaction: while the latter represents the true dis-
position of the user, IQ is the disposition of the user assumed by an expert annotator. Here, expert
annotators are people who listen to recorded dialogues after the interactions and rate them by as-
suming the point of view of the actual person performing the dialogue. These experts are supposed
to have some experience with dialogue systems. For the LEGO corpus—the data set used in this
work—, expert annotators were “advanced students of computer science and engineering” (Schmitt
and Ultes, 2015; Schmitt et al., 2011), i.e., grad students.

Interaction quality has been shown to be a suitable surrogate for user satisfaction by Ultes et al.
(2013). Comparing user satisfaction ratings and interaction quality ratings for the same data showed
a high correlation between the labels. Estimation models trained on one (user satisfaction or inter-
action quality) and evaluated on the other result in estimation performances clearly above chance.
Furthermore, interaction quality estimation is much more reliable and accurate than user satisfaction
estimation. Moreover, the interaction quality matches requirements identified by Ultes et al. (2012)
for an estimation approach to be used for online adaptation: exchange-level ratings, automatically
derivable and domain-independent input features, a consistent labelling process and reproducible
and unbiased labels.

The LEGO corpus (Schmitt et al., 2012b) is based on 200 calls to the “Let’s Go Bus Informa-
tion System” of the Carnegie Mellon University in Pittsburgh (Raux et al., 2006) recorded in 2006.
Labels for IQ have been assigned by three expert annotators to 200 calls consisting of 4,885 system-
user-exchanges (see Figure 2) in total with an inter-annotator agreement of κ = 0.54 using Cohen’s
weighted kappa with a linear weighting function 13. This may be considered as a moderate agree-
ment (cf. Landis and Koch’s Kappa Benchmark Scale (1977)), which is quite good considering the
difficulty of the task that required to rate each exchange. For instance, if one annotator reduces the
IQ value only one exchange earlier than another annotator, both already disagree on two exchanges.
The final label of each exchange was derived by using the median of all three individual ratings as
the median showed the best overall agreement (Schmitt and Ultes, 2015).

IQ was labelled on a scale from 1 (extremely unsatisfied) to 5 (satisfied) considering the com-
plete dialogue up to the current exchange. Thus, each exchange has been rated without regarding

85



ULTES AND MAIER

Policy

Interaction Quality 
Reward Estimation

State 
Tracking

Semantic 
Decoding

Speech 
Recognition

Speech 
Synthesis

Language 
Generation Environment at

st

rt

en… en-1en-2e1 e2 en+1 …

Dialogue-Level Window-Level Exchange-Level

Recognition Dialogue

Figure 3: The overall architecture of deriving the interaction parameters from the spoken dialogue
system that model the temporal information explicitly and are used as input to the inter-
action quality reward estimator.

any upcoming user utterance. As the users are expected to be satisfied at the beginning, each dia-
logue’s initial rating is 5. In order to ensure consistent labelling, the expert annotators had to follow
distinct labelling guidelines (Schmitt et al., 2012b).

An example of an annotated dialogue of the LEGO corpus is shown in Table 1. The dialogue
starts off with a good IQ until the system first attempts to find results. There, the IQ falls drastically
as the user input does not correspond to what the system expects and the system remains in a loop
until the user reacts appropriately.

3.2 Conventional Interaction Quality Estimation

The problem of estimating IQ has been cast as a classification problem where the target classes are
the distinct IQ values. The input consists of domain-independent variables called interaction pa-
rameters. These parameters incorporate information from the automatic speech recognition (ASR)
output and the preceding system action (see Figure 3): the ASR status (one of success, no match, no
input), the ASR confidence of the highest ranked result, the general type of the system action (one
of statement, question), whether the system action is a repetition of the previous system action, and
whether the role of the system action is to confirm previous user input. This information, which is
available at every turn, is used to compute temporal features by taking sums, means or counts from
the turn-based information for a window of the last three system-user-exchanges2 and the complete

2. A system turn followed by a user turn.

86



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

Table 1: Example of a dialogue annotated with Interaction Quality for each system-user-exchange.
As the system fails to interact properly with the user in the end, the IQ value drops down
to one. Utterance represents either the system utterance or the recognised user utterance.
(CallID 2061123000 of the LEGO corpus.)

Utterance IQ
S: Welcome to the CMU Let’s Go bus information system.
U: 5
S: To get help at any time, just say Help or press zero.
U: 5
S: What can I do for you?
U: PENN AT BUTLER TIME IS IT IS THE CENTRE AT HIGHLAND FROM RUNS THE 61A EIGHT THIRTY

61B CAN’T OAKLAND
5

S: Leaving from OAKLAND . Is this correct?
U: OAKLAND 4
S: Leaving from OAKLAND . Is this correct?
U: YES 4
S: Right
U: - 4
S: Where do you want to go?
U: SQUIRREL HILL OF FIFTH AVENUE 4
S: Going to FIFTH AVENUE . Is this correct?
U: YES 5
S: Alright
U: - 5
S: When do you wanna travel?
U: SUNDAY 4
S: At what time do you wanna travel?
U: AT FIVE FOUR VERY ABOUT ELEVEN O’CLOCK 4
S: Leaving at 11 a.m. . Is this correct?
U: YES 5
S: Right
U: - 5
S: Hold on. Let me check that for you.
U: - 5
S: OAKLAND and FIFTH AVENUE are both the same stop. Please provide a different start or end point.
U: - 3
S: To ask about a different trip, you can say, start a new query. If you are finished, you can say goodbye.
U: 61A 2
S: You can say start a new query, or goodbye.
U: FIVE DUQUESNE 1
S: To ask about a different trip, you can say, start a new query. If you are finished, you can say goodbye.
U: LAST THE NEXT 1
S: You can say start a new query, or goodbye.
U: START WH 1
S: To ask about a different trip, you can say, start a new query. If you are finished, you can say goodbye.
U: THE 1
S: You can say start a new query, or goodbye.
U: THANK YOU QUERY ELEVEN BACK 1
S: To ask about a different trip, you can say, start a new query. If you are finished, you can say goodbye.
U: GOODBYE 1
S: Thank you for using the cmu Let’s Go Bus Information System. Goodbye.
U: - 1

dialogue (see lower part of Figure 3). The baseline IQ estimation approach as applied by Ultes et al.
(2017a) (and originating from Ultes et al. (2015)) uses a feature set of 16 parameters as shown in
Table 2 to train a support vector machine (SVM) (Vapnik, 1995; Chang and Lin, 2011) with linear
kernel.

87



ULTES AND MAIER

Table 2: The interaction parameters extracted from each user input (exchange level) plus counts,
sums and rates for the whole dialogue (#, %, Mean) and for a window ({·}) of the last 3
turns.

Parameter Description
E

xc
ha

ng
e

le
ve

l ASRRecognitionStatus ASR status: success, no match, no input
ASRConfidence confidence of top ASR results

RePrompt? is the system question the same as in the previous turn?
ActivityType general type of system action: statement, question

Confirmation? is system action confirm?

D
ia

lo
gu

e
le

ve
l MeanASRConfidence mean ASR confidence if ASR is success

#Exchanges number of exchanges (turns)
#ASRSuccess count of ASR status is success

%ASRSuccess rate of ASR status is success
#ASRRejections count of ASR status is reject

%ASRRejections rate of ASR status is reject

W
in

do
w

le
ve

l {Mean}ASRConfidence mean ASR confidence if ASR is success
{#}ASRSuccess count of ASR is success

{#}ASRRejections count of ASR status is reject
{#}RePrompts count of times RePromt? is true

{#}SystemQuestions count of ActivityType is question

The LEGO corpus (Schmitt et al., 2012a) provides data for training and testing and consists of
200 dialogues (4,885 turns) from the Let’s Go bus information system (Raux et al., 2006; Eskenazi
et al., 2008) of Carnegie Mellon University in Pittsburgh, PA. The system provided information
about bus schedules and connections to actual users with real needs and was live from 2006 until
2016. Each turn of these 200 dialogues has been annotated with IQ (representing the quality of the
dialogue up to the current turn) by three experts. The final IQ label has been assigned using the
median of the three individual labels.

Previous work has used the LEGO corpus with a full IQ feature set (which includes additional
partly domain-related information) achieving an unweighted average recall3 (UAR) of 0.55 using
ordinal regression (El Asri et al., 2014a), 0.53 using a two-level SVM approach (Ultes and Minker,
2013), and 0.51 using a hybrid-HMM (Ultes and Minker, 2014). Human performance on the same
task is 0.69 UAR (Schmitt and Ultes, 2015).

3.3 LSTM-based Interaction Quality Estimation

The architecture of the LSTM-based IQ estimation model is shown in Figure 4. It is based on the
idea that the temporal information may be learned by using recurrent neural networks instead of
encoding it explicitly with the window and dialogue interaction parameter levels as used by the
conventional estimator. Thus, only the exchange level parameters et are considered (see Table 2).
Long Short-Term Memory (LSTM) cells are at the core of the model and have originally been
proposed by Hochreiter and Schmidhuber (1997) as a recurrent variant that remedies the vanishing
gradient problem (Bengio et al., 1994).

3. UAR is the arithmetic average of all class-wise recalls.

88



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

…

…

L
S
T
M

<latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit>

L
S
T
M

<latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit>

L
S
T
M

<latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit>

L
S
T
M

<latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit> L
S
T
M

<latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit>

L
S
T
M

<latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit><latexit sha1_base64="n+ScoIfw9MA0pSNB8fVTOFfZTow=">AAAHSXiclVXLbtNAFL1tIG3Dq4UFCzYRaREStEoCEiwjtSAWIAo0baWmQrYzSa34EdlOcLD6DWzhm/gCPoMdQiw493oSmjiPYsueO3fuOfc1Y5tdxw6jcvnH0nLuytX8yupa4dr1GzdvrW/cPgz9XmCpuuU7fnBsGqFybE/VIzty1HE3UIZrOurI7Ozy+lFfBaHtewfRoKtOXaPt2S3bMiKo6g0njNyP66XyTlmuYlaoaKFE+tr3N3J3qUFN8smiHrmkyKMIskMGhbhPqEJl6kJ3Sgl0ASRb1hWdUwHYHqwULAxoO3i3MTvRWg9z5gwFbcGLgycAskhbeF4Kowlr9qoghxj/4PksuvZMD4kwc4QDjCYY14TxDfQRncFiEdLVlsNYFiM5q4ha9FyysRFfVzScpzXi2cNKAF1HVor0Qizb4DBl3kcFPIx1RMBVHjIUJeMmRkNGJSyeZjTAF2Dk6nM8s7Nrw86A9kz6FNNjWbd0Z0PpGqP3oEn7zH6G2b8FUglDJJ1KYBlCamHOFYuBTzKa/+NzpMuuML2mD3QA62FUrmSqxJ73WKT3RdbDe/CZUoOIdiG7guXKsQ/Otw9W3r/crS0wFHE3Rn3nuv3zkJ6BmsYlVALuXFBpdh70n2QHj/u5iGJv1QlvLXQhzvAm9Eizp1acT0uisRCNCc4YNpsSbU2iZVwDDGyRzEXM8riN9zhfCdHOj3ATT/penNH2peo1jp5WMRs2sc63iVmITnHPB9LpgeSc2g772ZEd4kk+TzDvjbiU7LVx+4v83PlYTkWWeRH7LA9ZLyGy5bPbneNpmrfqhLd5Hi+P5w4Ups6H8rSz6OtvY4A/TGXyf5IVDqs7FcjvnpZqD/S/ZpXu0X16iIo8Q/9f0T6+gBb8faGv9C3/Pf8z/yv/OzVdXtKYOzR2reT+Ak74UVw=</latexit>

softmax
<latexit sha1_base64="dM0oHcFw8TO97AKszqEzCqu4j9A=">AAAHTHiclVVLb9NAEJ62kLbh1cKBA5eItAgJWiUBCY6RWhAHEAWaNlJTIdvZpFb8iGwnOFj9E1zhN3Hnf3ADJL4Zb0IT51Fs2Ts7O983r13b7Dp2GJVKP5aWV65cza2ureevXb9x89bG5u2j0O8FlqpZvuMHddMIlWN7qhbZkaPq3UAZrumoY7Ozx+vHfRWEtu8dRoOuOnWNtme3bMuIoKo3Qr8VuUb8caNY2i3JVcgKZS0USV8H/ubKXWpQk3yyqEcuKfIoguyQQSHuEypTibrQnVICXQDJlnVF55QHtgcrBQsD2g7ebcxOtNbDnDlDQVvw4uAJgCzQNp6XwmjCmr0qyCHG33g+i64900MizBzhAKMJxnVhfAN9RGewWIR0teUwlsVIziqiFj2XbGzE1xUN52mNePaxEkDXkZUCvRDLNjhMmfdRAQ9jDRFwlYcMBcm4idGQUQmLpxkN8AUYufocz+zs2rAzoD2TPsX0WNYt3dlQusbofWjSPrOfYfZvgVTCEEmnEliGkFqYc8Vi4JOM5v/4HOmyK0yv6QMdwnoYlSuZKrHnPRbpfZH18B58ptQgoj3IrmC5cuyD8+2Dlfcvd2sbDAXcjVHfuW7/PKRnoKpxCRWBOxdUmp0H/SfZweN+LqLYW2XCWwtdiDO8CT3S7KkV59OSaCxEY4Izhs2WRFuVaBnXAANbJHMRszzu4D3OV0S08yPcwpO+F2e0c6l6jaOnVcyGTazzbWIWolPc84F0eiA5p7bDfnZkh3iSzxPMeyMuJXtt3P4iP3c+llORZV7EPstD1kuIbPnsdud4muatMuFtnsfL47kD+anzoTztLPr62xjgD1Oe/J9khaPKbhnyu6fF6gP9r1mje3SfHqIiz9D/V3SALyCf5i/0lb7lvud+5n7l/qSmy0sac4fGrtXcX3zkUrw=</latexit><latexit sha1_base64="dM0oHcFw8TO97AKszqEzCqu4j9A=">AAAHTHiclVVLb9NAEJ62kLbh1cKBA5eItAgJWiUBCY6RWhAHEAWaNlJTIdvZpFb8iGwnOFj9E1zhN3Hnf3ADJL4Zb0IT51Fs2Ts7O983r13b7Dp2GJVKP5aWV65cza2ureevXb9x89bG5u2j0O8FlqpZvuMHddMIlWN7qhbZkaPq3UAZrumoY7Ozx+vHfRWEtu8dRoOuOnWNtme3bMuIoKo3Qr8VuUb8caNY2i3JVcgKZS0USV8H/ubKXWpQk3yyqEcuKfIoguyQQSHuEypTibrQnVICXQDJlnVF55QHtgcrBQsD2g7ebcxOtNbDnDlDQVvw4uAJgCzQNp6XwmjCmr0qyCHG33g+i64900MizBzhAKMJxnVhfAN9RGewWIR0teUwlsVIziqiFj2XbGzE1xUN52mNePaxEkDXkZUCvRDLNjhMmfdRAQ9jDRFwlYcMBcm4idGQUQmLpxkN8AUYufocz+zs2rAzoD2TPsX0WNYt3dlQusbofWjSPrOfYfZvgVTCEEmnEliGkFqYc8Vi4JOM5v/4HOmyK0yv6QMdwnoYlSuZKrHnPRbpfZH18B58ptQgoj3IrmC5cuyD8+2Dlfcvd2sbDAXcjVHfuW7/PKRnoKpxCRWBOxdUmp0H/SfZweN+LqLYW2XCWwtdiDO8CT3S7KkV59OSaCxEY4Izhs2WRFuVaBnXAANbJHMRszzu4D3OV0S08yPcwpO+F2e0c6l6jaOnVcyGTazzbWIWolPc84F0eiA5p7bDfnZkh3iSzxPMeyMuJXtt3P4iP3c+llORZV7EPstD1kuIbPnsdud4muatMuFtnsfL47kD+anzoTztLPr62xjgD1Oe/J9khaPKbhnyu6fF6gP9r1mje3SfHqIiz9D/V3SALyCf5i/0lb7lvud+5n7l/qSmy0sac4fGrtXcX3zkUrw=</latexit><latexit sha1_base64="dM0oHcFw8TO97AKszqEzCqu4j9A=">AAAHTHiclVVLb9NAEJ62kLbh1cKBA5eItAgJWiUBCY6RWhAHEAWaNlJTIdvZpFb8iGwnOFj9E1zhN3Hnf3ADJL4Zb0IT51Fs2Ts7O983r13b7Dp2GJVKP5aWV65cza2ureevXb9x89bG5u2j0O8FlqpZvuMHddMIlWN7qhbZkaPq3UAZrumoY7Ozx+vHfRWEtu8dRoOuOnWNtme3bMuIoKo3Qr8VuUb8caNY2i3JVcgKZS0USV8H/ubKXWpQk3yyqEcuKfIoguyQQSHuEypTibrQnVICXQDJlnVF55QHtgcrBQsD2g7ebcxOtNbDnDlDQVvw4uAJgCzQNp6XwmjCmr0qyCHG33g+i64900MizBzhAKMJxnVhfAN9RGewWIR0teUwlsVIziqiFj2XbGzE1xUN52mNePaxEkDXkZUCvRDLNjhMmfdRAQ9jDRFwlYcMBcm4idGQUQmLpxkN8AUYufocz+zs2rAzoD2TPsX0WNYt3dlQusbofWjSPrOfYfZvgVTCEEmnEliGkFqYc8Vi4JOM5v/4HOmyK0yv6QMdwnoYlSuZKrHnPRbpfZH18B58ptQgoj3IrmC5cuyD8+2Dlfcvd2sbDAXcjVHfuW7/PKRnoKpxCRWBOxdUmp0H/SfZweN+LqLYW2XCWwtdiDO8CT3S7KkV59OSaCxEY4Izhs2WRFuVaBnXAANbJHMRszzu4D3OV0S08yPcwpO+F2e0c6l6jaOnVcyGTazzbWIWolPc84F0eiA5p7bDfnZkh3iSzxPMeyMuJXtt3P4iP3c+llORZV7EPstD1kuIbPnsdud4muatMuFtnsfL47kD+anzoTztLPr62xjgD1Oe/J9khaPKbhnyu6fF6gP9r1mje3SfHqIiz9D/V3SALyCf5i/0lb7lvud+5n7l/qSmy0sac4fGrtXcX3zkUrw=</latexit><latexit sha1_base64="dM0oHcFw8TO97AKszqEzCqu4j9A=">AAAHTHiclVVLb9NAEJ62kLbh1cKBA5eItAgJWiUBCY6RWhAHEAWaNlJTIdvZpFb8iGwnOFj9E1zhN3Hnf3ADJL4Zb0IT51Fs2Ts7O983r13b7Dp2GJVKP5aWV65cza2ureevXb9x89bG5u2j0O8FlqpZvuMHddMIlWN7qhbZkaPq3UAZrumoY7Ozx+vHfRWEtu8dRoOuOnWNtme3bMuIoKo3Qr8VuUb8caNY2i3JVcgKZS0USV8H/ubKXWpQk3yyqEcuKfIoguyQQSHuEypTibrQnVICXQDJlnVF55QHtgcrBQsD2g7ebcxOtNbDnDlDQVvw4uAJgCzQNp6XwmjCmr0qyCHG33g+i64900MizBzhAKMJxnVhfAN9RGewWIR0teUwlsVIziqiFj2XbGzE1xUN52mNePaxEkDXkZUCvRDLNjhMmfdRAQ9jDRFwlYcMBcm4idGQUQmLpxkN8AUYufocz+zs2rAzoD2TPsX0WNYt3dlQusbofWjSPrOfYfZvgVTCEEmnEliGkFqYc8Vi4JOM5v/4HOmyK0yv6QMdwnoYlSuZKrHnPRbpfZH18B58ptQgoj3IrmC5cuyD8+2Dlfcvd2sbDAXcjVHfuW7/PKRnoKpxCRWBOxdUmp0H/SfZweN+LqLYW2XCWwtdiDO8CT3S7KkV59OSaCxEY4Izhs2WRFuVaBnXAANbJHMRszzu4D3OV0S08yPcwpO+F2e0c6l6jaOnVcyGTazzbWIWolPc84F0eiA5p7bDfnZkh3iSzxPMeyMuJXtt3P4iP3c+llORZV7EPstD1kuIbPnsdud4muatMuFtnsfL47kD+anzoTztLPr62xjgD1Oe/J9khaPKbhnyu6fF6gP9r1mje3SfHqIiz9D/V3SALyCf5i/0lb7lvud+5n7l/qSmy0sac4fGrtXcX3zkUrw=</latexit>

attention
<latexit sha1_base64="NSg+J195rSwUHn+OjSfa//qlfNM=">AAAHpniclVXbbtNAEJ22QNpya+GRl4i0qBK0SgISPFZKQTxwKdA0FU1FbWeTWvElsjchwepf8Ar/xb/wwJnxpjRxLq0te2dn55y57dp2x3NjXSz+WVhcunHzVm55ZfX2nbv37q+tPziMw27kqKoTemF0ZFux8txAVbWrPXXUiZTl256q2e0Kr9d6KordMDjQg4468a1W4DZdx9JQfa1bWquAxW9rheJOUa58VigZoUDm2g/XlypUpwaF5FCXfFIUkIbskUUx7mMqUZE60J1QAl0EyZV1Ree0CmwXVgoWFrRtvFuYHRttgDlzxoJ24MXDEwGZp008b4TRhjV7VZBjjH/x/BBda6qHRJg5wgFGG4wrwvgeek1nsJiH9I3lMJb5SM5KU5NeSTYu4uuIhvN0Lnj2sBJB15aVPL0WyxY4bJn3UIEAYxURcJWHDHnJuIHRklEJS2AYLfBFGLn6HM/07Fqws6A9kz716ZmsO6azsXSN0XvQpH1mP8PsPwKphEFLpxJYxpCamHPF+sAnGc31+FjWJh5XqnEuXclqr8frye7xhe0dfaEDWA+z9aWCSux572qz37IePoPPltpqqkD2BcsdYR9cxx5Y+VzwLtgEQx53/WI/cT/+e0jP1q7BJVQA7lxQaXYB9N/lZIz6uYxib+Uxb03UqJ/hTeipYU+tOJ+mROMgGhucfdhsSLS7Ei3j6mBgi2QmYprHbbxH+QqIdnaEG3jS9/yMtq9Ur1H0pIq5sOmbfBuYxegU93wgnR5IzqntsJ9t2SGB5PMc8+4Fl5K9Nmp/mZ8735fTlmWexz7NQ9ZLjGz5m9CZ4WmSt/KYt1ker47nDqxOnA/lSWcxNN/cCH+u0vh/KisclndKkD+9KOw+Mf+wZXpEj2kLFXmJ/r+lfXxZHUT6k37R79xW7kOumqulposLBvOQRq7c6T+J4WNw</latexit><latexit sha1_base64="NSg+J195rSwUHn+OjSfa//qlfNM=">AAAHpniclVXbbtNAEJ22QNpya+GRl4i0qBK0SgISPFZKQTxwKdA0FU1FbWeTWvElsjchwepf8Ar/xb/wwJnxpjRxLq0te2dn55y57dp2x3NjXSz+WVhcunHzVm55ZfX2nbv37q+tPziMw27kqKoTemF0ZFux8txAVbWrPXXUiZTl256q2e0Kr9d6KordMDjQg4468a1W4DZdx9JQfa1bWquAxW9rheJOUa58VigZoUDm2g/XlypUpwaF5FCXfFIUkIbskUUx7mMqUZE60J1QAl0EyZV1Ree0CmwXVgoWFrRtvFuYHRttgDlzxoJ24MXDEwGZp008b4TRhjV7VZBjjH/x/BBda6qHRJg5wgFGG4wrwvgeek1nsJiH9I3lMJb5SM5KU5NeSTYu4uuIhvN0Lnj2sBJB15aVPL0WyxY4bJn3UIEAYxURcJWHDHnJuIHRklEJS2AYLfBFGLn6HM/07Fqws6A9kz716ZmsO6azsXSN0XvQpH1mP8PsPwKphEFLpxJYxpCamHPF+sAnGc31+FjWJh5XqnEuXclqr8frye7xhe0dfaEDWA+z9aWCSux572qz37IePoPPltpqqkD2BcsdYR9cxx5Y+VzwLtgEQx53/WI/cT/+e0jP1q7BJVQA7lxQaXYB9N/lZIz6uYxib+Uxb03UqJ/hTeipYU+tOJ+mROMgGhucfdhsSLS7Ei3j6mBgi2QmYprHbbxH+QqIdnaEG3jS9/yMtq9Ur1H0pIq5sOmbfBuYxegU93wgnR5IzqntsJ9t2SGB5PMc8+4Fl5K9Nmp/mZ8735fTlmWexz7NQ9ZLjGz5m9CZ4WmSt/KYt1ker47nDqxOnA/lSWcxNN/cCH+u0vh/KisclndKkD+9KOw+Mf+wZXpEj2kLFXmJ/r+lfXxZHUT6k37R79xW7kOumqulposLBvOQRq7c6T+J4WNw</latexit><latexit sha1_base64="NSg+J195rSwUHn+OjSfa//qlfNM=">AAAHpniclVXbbtNAEJ22QNpya+GRl4i0qBK0SgISPFZKQTxwKdA0FU1FbWeTWvElsjchwepf8Ar/xb/wwJnxpjRxLq0te2dn55y57dp2x3NjXSz+WVhcunHzVm55ZfX2nbv37q+tPziMw27kqKoTemF0ZFux8txAVbWrPXXUiZTl256q2e0Kr9d6KordMDjQg4468a1W4DZdx9JQfa1bWquAxW9rheJOUa58VigZoUDm2g/XlypUpwaF5FCXfFIUkIbskUUx7mMqUZE60J1QAl0EyZV1Ree0CmwXVgoWFrRtvFuYHRttgDlzxoJ24MXDEwGZp008b4TRhjV7VZBjjH/x/BBda6qHRJg5wgFGG4wrwvgeek1nsJiH9I3lMJb5SM5KU5NeSTYu4uuIhvN0Lnj2sBJB15aVPL0WyxY4bJn3UIEAYxURcJWHDHnJuIHRklEJS2AYLfBFGLn6HM/07Fqws6A9kz716ZmsO6azsXSN0XvQpH1mP8PsPwKphEFLpxJYxpCamHPF+sAnGc31+FjWJh5XqnEuXclqr8frye7xhe0dfaEDWA+z9aWCSux572qz37IePoPPltpqqkD2BcsdYR9cxx5Y+VzwLtgEQx53/WI/cT/+e0jP1q7BJVQA7lxQaXYB9N/lZIz6uYxib+Uxb03UqJ/hTeipYU+tOJ+mROMgGhucfdhsSLS7Ei3j6mBgi2QmYprHbbxH+QqIdnaEG3jS9/yMtq9Ur1H0pIq5sOmbfBuYxegU93wgnR5IzqntsJ9t2SGB5PMc8+4Fl5K9Nmp/mZ8735fTlmWexz7NQ9ZLjGz5m9CZ4WmSt/KYt1ker47nDqxOnA/lSWcxNN/cCH+u0vh/KisclndKkD+9KOw+Mf+wZXpEj2kLFXmJ/r+lfXxZHUT6k37R79xW7kOumqulposLBvOQRq7c6T+J4WNw</latexit><latexit sha1_base64="NSg+J195rSwUHn+OjSfa//qlfNM=">AAAHpniclVXbbtNAEJ22QNpya+GRl4i0qBK0SgISPFZKQTxwKdA0FU1FbWeTWvElsjchwepf8Ar/xb/wwJnxpjRxLq0te2dn55y57dp2x3NjXSz+WVhcunHzVm55ZfX2nbv37q+tPziMw27kqKoTemF0ZFux8txAVbWrPXXUiZTl256q2e0Kr9d6KordMDjQg4468a1W4DZdx9JQfa1bWquAxW9rheJOUa58VigZoUDm2g/XlypUpwaF5FCXfFIUkIbskUUx7mMqUZE60J1QAl0EyZV1Ree0CmwXVgoWFrRtvFuYHRttgDlzxoJ24MXDEwGZp008b4TRhjV7VZBjjH/x/BBda6qHRJg5wgFGG4wrwvgeek1nsJiH9I3lMJb5SM5KU5NeSTYu4uuIhvN0Lnj2sBJB15aVPL0WyxY4bJn3UIEAYxURcJWHDHnJuIHRklEJS2AYLfBFGLn6HM/07Fqws6A9kz716ZmsO6azsXSN0XvQpH1mP8PsPwKphEFLpxJYxpCamHPF+sAnGc31+FjWJh5XqnEuXclqr8frye7xhe0dfaEDWA+z9aWCSux572qz37IePoPPltpqqkD2BcsdYR9cxx5Y+VzwLtgEQx53/WI/cT/+e0jP1q7BJVQA7lxQaXYB9N/lZIz6uYxib+Uxb03UqJ/hTeipYU+tOJ+mROMgGhucfdhsSLS7Ei3j6mBgi2QmYprHbbxH+QqIdnaEG3jS9/yMtq9Ur1H0pIq5sOmbfBuYxegU93wgnR5IzqntsJ9t2SGB5PMc8+4Fl5K9Nmp/mZ8735fTlmWexz7NQ9ZLjGz5m9CZ4WmSt/KYt1ker47nDqxOnA/lSWcxNN/cCH+u0vh/KisclndKkD+9KOw+Mf+wZXpEj2kLFXmJ/r+lfXxZHUT6k37R79xW7kOumqulposLBvOQRq7c6T+J4WNw</latexit> ·<latexit sha1_base64="csy7Gs5UZWaSyVtGePdHTbWldig=">AAAHoXiclVXbbtNAEJ22QNpwa+GRl4i0EhK0SgISPEZKQSCBCKVpKzUVsp1NasWXyN6UBCvfwCt8Gv/CA2fGm9LEubS27J2dnXPmtmvbPc+Ndan0Z2V17dbtO7n1jfzde/cfPNzcenQUh/3IUQ0n9MLoxLZi5bmBamhXe+qkFynLtz11bHdrvH58oaLYDYNDPeypM9/qBG7bdSwNVaPptEL9bbNY2ivJVcgKZSMUyVz1cGutRk1qUUgO9cknRQFpyB5ZFOM+pTKVqAfdGSXQRZBcWVc0ojywfVgpWFjQdvHuYHZqtAHmzBkL2oEXD08EZIF28LwTRhvW7FVBjjH+xfNDdJ25HhJh5giHGG0wbgjjJ+g1ncNiGdI3luNYliM5K01teiPZuIivJxrO07nk2cdKBF1XVgr0Viw74LBlfoEKBBgbiICrPGYoSMYtjJaMSlgCw2iBL8LI1ed45mfXgZ0F7bn0aUAvZN0xnY2la4zehybtM/sZZ/8ZSCUMWjqVwDKG1MacKzYAPslobsbHsjbxuFKNkXQlq70Zrye7xxe2j/SVDmE9ztaXCiqx572rzX7LejgAny211VSD7AuWO8I+uI4XYOVzwbtgBwwF3M3L/cT9+O8hPVtVg0uoCNxIUGl2AfTf5WRM+rmKYm+VKW9t1GiQ4U3ouWFPrTiftkTjIBobnAPYbEu0VYmWcU0wsEWyEDHP4y7ek3xFRLs4wm086Xt5RrvXqtckelbFXNgMTL4tzGJ0ins+lE4PJefUdtzPruyQQPJ5iXn/kkvJXpu0v8rPnR/IacsyL2Of5yHrJUa2/E3oLfA0y1tlytsij9fHcwfyM+djedZZDM03N8Kfqzz9n8oKR5W9MuQvr4rVivmHrdMTekrPUJHX6P97quPL6sDfT/pFv3PF3IdcPXeQmq6uGMxjmrhyp/8A+NNhHg==</latexit><latexit sha1_base64="csy7Gs5UZWaSyVtGePdHTbWldig=">AAAHoXiclVXbbtNAEJ22QNpwa+GRl4i0EhK0SgISPEZKQSCBCKVpKzUVsp1NasWXyN6UBCvfwCt8Gv/CA2fGm9LEubS27J2dnXPmtmvbPc+Ndan0Z2V17dbtO7n1jfzde/cfPNzcenQUh/3IUQ0n9MLoxLZi5bmBamhXe+qkFynLtz11bHdrvH58oaLYDYNDPeypM9/qBG7bdSwNVaPptEL9bbNY2ivJVcgKZSMUyVz1cGutRk1qUUgO9cknRQFpyB5ZFOM+pTKVqAfdGSXQRZBcWVc0ojywfVgpWFjQdvHuYHZqtAHmzBkL2oEXD08EZIF28LwTRhvW7FVBjjH+xfNDdJ25HhJh5giHGG0wbgjjJ+g1ncNiGdI3luNYliM5K01teiPZuIivJxrO07nk2cdKBF1XVgr0Viw74LBlfoEKBBgbiICrPGYoSMYtjJaMSlgCw2iBL8LI1ed45mfXgZ0F7bn0aUAvZN0xnY2la4zehybtM/sZZ/8ZSCUMWjqVwDKG1MacKzYAPslobsbHsjbxuFKNkXQlq70Zrye7xxe2j/SVDmE9ztaXCiqx572rzX7LejgAny211VSD7AuWO8I+uI4XYOVzwbtgBwwF3M3L/cT9+O8hPVtVg0uoCNxIUGl2AfTf5WRM+rmKYm+VKW9t1GiQ4U3ouWFPrTiftkTjIBobnAPYbEu0VYmWcU0wsEWyEDHP4y7ek3xFRLs4wm086Xt5RrvXqtckelbFXNgMTL4tzGJ0ins+lE4PJefUdtzPruyQQPJ5iXn/kkvJXpu0v8rPnR/IacsyL2Of5yHrJUa2/E3oLfA0y1tlytsij9fHcwfyM+djedZZDM03N8Kfqzz9n8oKR5W9MuQvr4rVivmHrdMTekrPUJHX6P97quPL6sDfT/pFv3PF3IdcPXeQmq6uGMxjmrhyp/8A+NNhHg==</latexit><latexit sha1_base64="csy7Gs5UZWaSyVtGePdHTbWldig=">AAAHoXiclVXbbtNAEJ22QNpwa+GRl4i0EhK0SgISPEZKQSCBCKVpKzUVsp1NasWXyN6UBCvfwCt8Gv/CA2fGm9LEubS27J2dnXPmtmvbPc+Ndan0Z2V17dbtO7n1jfzde/cfPNzcenQUh/3IUQ0n9MLoxLZi5bmBamhXe+qkFynLtz11bHdrvH58oaLYDYNDPeypM9/qBG7bdSwNVaPptEL9bbNY2ivJVcgKZSMUyVz1cGutRk1qUUgO9cknRQFpyB5ZFOM+pTKVqAfdGSXQRZBcWVc0ojywfVgpWFjQdvHuYHZqtAHmzBkL2oEXD08EZIF28LwTRhvW7FVBjjH+xfNDdJ25HhJh5giHGG0wbgjjJ+g1ncNiGdI3luNYliM5K01teiPZuIivJxrO07nk2cdKBF1XVgr0Viw74LBlfoEKBBgbiICrPGYoSMYtjJaMSlgCw2iBL8LI1ed45mfXgZ0F7bn0aUAvZN0xnY2la4zehybtM/sZZ/8ZSCUMWjqVwDKG1MacKzYAPslobsbHsjbxuFKNkXQlq70Zrye7xxe2j/SVDmE9ztaXCiqx572rzX7LejgAny211VSD7AuWO8I+uI4XYOVzwbtgBwwF3M3L/cT9+O8hPVtVg0uoCNxIUGl2AfTf5WRM+rmKYm+VKW9t1GiQ4U3ouWFPrTiftkTjIBobnAPYbEu0VYmWcU0wsEWyEDHP4y7ek3xFRLs4wm086Xt5RrvXqtckelbFXNgMTL4tzGJ0ins+lE4PJefUdtzPruyQQPJ5iXn/kkvJXpu0v8rPnR/IacsyL2Of5yHrJUa2/E3oLfA0y1tlytsij9fHcwfyM+djedZZDM03N8Kfqzz9n8oKR5W9MuQvr4rVivmHrdMTekrPUJHX6P97quPL6sDfT/pFv3PF3IdcPXeQmq6uGMxjmrhyp/8A+NNhHg==</latexit><latexit sha1_base64="csy7Gs5UZWaSyVtGePdHTbWldig=">AAAHoXiclVXbbtNAEJ22QNpwa+GRl4i0EhK0SgISPEZKQSCBCKVpKzUVsp1NasWXyN6UBCvfwCt8Gv/CA2fGm9LEubS27J2dnXPmtmvbPc+Ndan0Z2V17dbtO7n1jfzde/cfPNzcenQUh/3IUQ0n9MLoxLZi5bmBamhXe+qkFynLtz11bHdrvH58oaLYDYNDPeypM9/qBG7bdSwNVaPptEL9bbNY2ivJVcgKZSMUyVz1cGutRk1qUUgO9cknRQFpyB5ZFOM+pTKVqAfdGSXQRZBcWVc0ojywfVgpWFjQdvHuYHZqtAHmzBkL2oEXD08EZIF28LwTRhvW7FVBjjH+xfNDdJ25HhJh5giHGG0wbgjjJ+g1ncNiGdI3luNYliM5K01teiPZuIivJxrO07nk2cdKBF1XVgr0Viw74LBlfoEKBBgbiICrPGYoSMYtjJaMSlgCw2iBL8LI1ed45mfXgZ0F7bn0aUAvZN0xnY2la4zehybtM/sZZ/8ZSCUMWjqVwDKG1MacKzYAPslobsbHsjbxuFKNkXQlq70Zrye7xxe2j/SVDmE9ztaXCiqx572rzX7LejgAny211VSD7AuWO8I+uI4XYOVzwbtgBwwF3M3L/cT9+O8hPVtVg0uoCNxIUGl2AfTf5WRM+rmKYm+VKW9t1GiQ4U3ouWFPrTiftkTjIBobnAPYbEu0VYmWcU0wsEWyEDHP4y7ek3xFRLs4wm086Xt5RrvXqtckelbFXNgMTL4tzGJ0ins+lE4PJefUdtzPruyQQPJ5iXn/kkvJXpu0v8rPnR/IacsyL2Of5yHrJUa2/E3oLfA0y1tlytsij9fHcwfyM+djedZZDM03N8Kfqzz9n8oKR5W9MuQvr4rVivmHrdMTekrPUJHX6P97quPL6sDfT/pFv3PF3IdcPXeQmq6uGMxjmrhyp/8A+NNhHg==</latexit>

~ht
<latexit sha1_base64="sl470xuPnl4/sCI3j+f53kDxhIM=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg6qubhSKe0W58lmhZIQCmeso3Fw5oArVKCSHOuSTooA0ZI8sinFfUImK1IbukhLoIkiurCsa0DqwHVgpWFjQtvBuYHZhtAHmzBkL2oEXD08EZJ528LwTRhvW7FVBjjH+xfNDdI2ZHhJh5gj7GG0wrgnjR+g1NWGxCOkby2Esi5GclaY6vZFsXMTXFg3n6Yx4DrESQdeSlTy9FcsGOGyZd1GBAOMJIuAqDxnyknENoyWjEpbAMFrgizBy9Tme2dk1YGdB25Q+9eiFrDums7F0jdGH0KR9Zj/D7D8BqYRBS6cSWMaQ6phzxXrAJxnNzfhY1iYeV6oxkK5ktTfj9WT3+ML2gb7SMayH2fpSQSX2vHe12W9ZD1/AZ0ttNR1A9gXLHWEfXMcuWPlc8C7YAUMed2W0n7gf/z2kZ2vf4BIqADcQVJpdAP13ORnjfq6i2Ft5wlsdNepleBN6bthTK86nLtE4iMYGZw822xLtvkTLuAoY2CKZi5jlcRfvcb4Cop0f4Tae9L04o91r1WscPa1iLmx6Jt8aZjE6xT3vS6f7knNqO+xnS3ZIIPm8xLwz4lKy18btr/Jz53ty2rLMi9hnech6iZEtfxPaczxN81ae8DbP4/Xx3IH1qfOhPO0shuabG+HPVZr8T2WF0/JeCfLnV4X9p+YftkqP6Qk9Q0Veo//v6QhfVv7u/aRf9DtXzp3nqjkrNV1eMphHNHblvv0DnjxnUg==</latexit><latexit sha1_base64="sl470xuPnl4/sCI3j+f53kDxhIM=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg6qubhSKe0W58lmhZIQCmeso3Fw5oArVKCSHOuSTooA0ZI8sinFfUImK1IbukhLoIkiurCsa0DqwHVgpWFjQtvBuYHZhtAHmzBkL2oEXD08EZJ528LwTRhvW7FVBjjH+xfNDdI2ZHhJh5gj7GG0wrgnjR+g1NWGxCOkby2Esi5GclaY6vZFsXMTXFg3n6Yx4DrESQdeSlTy9FcsGOGyZd1GBAOMJIuAqDxnyknENoyWjEpbAMFrgizBy9Tme2dk1YGdB25Q+9eiFrDums7F0jdGH0KR9Zj/D7D8BqYRBS6cSWMaQ6phzxXrAJxnNzfhY1iYeV6oxkK5ktTfj9WT3+ML2gb7SMayH2fpSQSX2vHe12W9ZD1/AZ0ttNR1A9gXLHWEfXMcuWPlc8C7YAUMed2W0n7gf/z2kZ2vf4BIqADcQVJpdAP13ORnjfq6i2Ft5wlsdNepleBN6bthTK86nLtE4iMYGZw822xLtvkTLuAoY2CKZi5jlcRfvcb4Cop0f4Tae9L04o91r1WscPa1iLmx6Jt8aZjE6xT3vS6f7knNqO+xnS3ZIIPm8xLwz4lKy18btr/Jz53ty2rLMi9hnech6iZEtfxPaczxN81ae8DbP4/Xx3IH1qfOhPO0shuabG+HPVZr8T2WF0/JeCfLnV4X9p+YftkqP6Qk9Q0Veo//v6QhfVv7u/aRf9DtXzp3nqjkrNV1eMphHNHblvv0DnjxnUg==</latexit><latexit sha1_base64="sl470xuPnl4/sCI3j+f53kDxhIM=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg6qubhSKe0W58lmhZIQCmeso3Fw5oArVKCSHOuSTooA0ZI8sinFfUImK1IbukhLoIkiurCsa0DqwHVgpWFjQtvBuYHZhtAHmzBkL2oEXD08EZJ528LwTRhvW7FVBjjH+xfNDdI2ZHhJh5gj7GG0wrgnjR+g1NWGxCOkby2Esi5GclaY6vZFsXMTXFg3n6Yx4DrESQdeSlTy9FcsGOGyZd1GBAOMJIuAqDxnyknENoyWjEpbAMFrgizBy9Tme2dk1YGdB25Q+9eiFrDums7F0jdGH0KR9Zj/D7D8BqYRBS6cSWMaQ6phzxXrAJxnNzfhY1iYeV6oxkK5ktTfj9WT3+ML2gb7SMayH2fpSQSX2vHe12W9ZD1/AZ0ttNR1A9gXLHWEfXMcuWPlc8C7YAUMed2W0n7gf/z2kZ2vf4BIqADcQVJpdAP13ORnjfq6i2Ft5wlsdNepleBN6bthTK86nLtE4iMYGZw822xLtvkTLuAoY2CKZi5jlcRfvcb4Cop0f4Tae9L04o91r1WscPa1iLmx6Jt8aZjE6xT3vS6f7knNqO+xnS3ZIIPm8xLwz4lKy18btr/Jz53ty2rLMi9hnech6iZEtfxPaczxN81ae8DbP4/Xx3IH1qfOhPO0shuabG+HPVZr8T2WF0/JeCfLnV4X9p+YftkqP6Qk9Q0Veo//v6QhfVv7u/aRf9DtXzp3nqjkrNV1eMphHNHblvv0DnjxnUg==</latexit><latexit sha1_base64="sl470xuPnl4/sCI3j+f53kDxhIM=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg6qubhSKe0W58lmhZIQCmeso3Fw5oArVKCSHOuSTooA0ZI8sinFfUImK1IbukhLoIkiurCsa0DqwHVgpWFjQtvBuYHZhtAHmzBkL2oEXD08EZJ528LwTRhvW7FVBjjH+xfNDdI2ZHhJh5gj7GG0wrgnjR+g1NWGxCOkby2Esi5GclaY6vZFsXMTXFg3n6Yx4DrESQdeSlTy9FcsGOGyZd1GBAOMJIuAqDxnyknENoyWjEpbAMFrgizBy9Tme2dk1YGdB25Q+9eiFrDums7F0jdGH0KR9Zj/D7D8BqYRBS6cSWMaQ6phzxXrAJxnNzfhY1iYeV6oxkK5ktTfj9WT3+ML2gb7SMayH2fpSQSX2vHe12W9ZD1/AZ0ttNR1A9gXLHWEfXMcuWPlc8C7YAUMed2W0n7gf/z2kZ2vf4BIqADcQVJpdAP13ORnjfq6i2Ft5wlsdNepleBN6bthTK86nLtE4iMYGZw822xLtvkTLuAoY2CKZi5jlcRfvcb4Cop0f4Tae9L04o91r1WscPa1iLmx6Jt8aZjE6xT3vS6f7knNqO+xnS3ZIIPm8xLwz4lKy18btr/Jz53ty2rLMi9hnech6iZEtfxPaczxN81ae8DbP4/Xx3IH1qfOhPO0shuabG+HPVZr8T2WF0/JeCfLnV4X9p+YftkqP6Qk9Q0Veo//v6QhfVv7u/aRf9DtXzp3nqjkrNV1eMphHNHblvv0DnjxnUg==</latexit>

~h2
<latexit sha1_base64="qWE2Mz+eWBETA+vLuNLoej03zDA=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg2q5ulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPwWxnEA==</latexit><latexit sha1_base64="qWE2Mz+eWBETA+vLuNLoej03zDA=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg2q5ulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPwWxnEA==</latexit><latexit sha1_base64="qWE2Mz+eWBETA+vLuNLoej03zDA=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg2q5ulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPwWxnEA==</latexit><latexit sha1_base64="qWE2Mz+eWBETA+vLuNLoej03zDA=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg2q5ulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPwWxnEA==</latexit>

~h1
<latexit sha1_base64="n2AtWvtb8O5LLvtE8oZt847KyAY=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg2qpulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPujNnDw==</latexit><latexit sha1_base64="n2AtWvtb8O5LLvtE8oZt847KyAY=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg2qpulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPujNnDw==</latexit><latexit sha1_base64="n2AtWvtb8O5LLvtE8oZt847KyAY=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg2qpulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPujNnDw==</latexit><latexit sha1_base64="n2AtWvtb8O5LLvtE8oZt847KyAY=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Kl3lJBXf0k27njQHg2qpulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPujNnDw==</latexit>

~h1
<latexit sha1_base64="pBqKXot9+qet8pcHYFe8/EBh7Rk=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg2qpulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPug1nDw==</latexit><latexit sha1_base64="pBqKXot9+qet8pcHYFe8/EBh7Rk=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg2qpulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPug1nDw==</latexit><latexit sha1_base64="pBqKXot9+qet8pcHYFe8/EBh7Rk=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg2qpulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPug1nDw==</latexit><latexit sha1_base64="pBqKXot9+qet8pcHYFe8/EBh7Rk=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg2qpulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPug1nDw==</latexit>

~h2
<latexit sha1_base64="YyNceqUBClnsydXvRj0M4rGU6oU=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg2q5ulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPwUZnEA==</latexit><latexit sha1_base64="YyNceqUBClnsydXvRj0M4rGU6oU=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg2q5ulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPwUZnEA==</latexit><latexit sha1_base64="YyNceqUBClnsydXvRj0M4rGU6oU=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg2q5ulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPwUZnEA==</latexit><latexit sha1_base64="YyNceqUBClnsydXvRj0M4rGU6oU=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg2q5ulEo7hXlymeFkhEKZK6jcHPlgCpUo5Ac6pBPigLSkD2yKMZ9QSUqUhu6S0qgiyC5sq5oQOvAdmClYGFB28K7gdmF0QaYM2csaAdePDwRkHnawfNOGG1Ys1cFOcb4F88P0TVmekiEmSPsY7TBuCaMH6HX1ITFIqRvLIexLEZyVprq9EaycRFfWzScpzPiOcRKBF1LVvL0Viwb4LBl3kUFAowniICrPGTIS8Y1jJaMSlgCw2iBL8LI1ed4ZmfXgJ0FbVP61KMXsu6YzsbSNUYfQpP2mf0Ms/8EpBIGLZ1KYBlDqmPOFesBn2Q0N+NjWZt4XKnGQLqS1d6M15Pd4wvbB/pKx7AeZutLBZXY897VZr9lPXwBny211XQA2Rcsd4R9cB27YOVzwbtgBwx53JXRfuJ+/PeQnq19g0uoANxAUGl2AfTf5WSM+7mKYm/lCW911KiX4U3ouWFPrTifukTjIBobnD3YbEu0+xIt4ypgYItkLmKWx128x/kKiHZ+hNt40vfijHavVa9x9LSKubDpmXxrmMXoFPe8L53uS86p7bCfLdkhgeTzEvPOiEvJXhu3v8rPne/JacsyL2Kf5SHrJUa2/E1oz/E0zVt5wts8j9fHcwfWp86H8rSzGJpvboQ/V2nyP5UVTst7JcifXxX2n5p/2Co9pif0DBV5jf6/pyN8Wfm795N+0e9cOXeeq+as1HR5yWAe0diV+/YPwUZnEA==</latexit>

~ht
<latexit sha1_base64="fjSkHt3n88x0XZEZyeYbOK6LyrU=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg6qubhSKe0W58lmhZIQCmeso3Fw5oArVKCSHOuSTooA0ZI8sinFfUImK1IbukhLoIkiurCsa0DqwHVgpWFjQtvBuYHZhtAHmzBkL2oEXD08EZJ528LwTRhvW7FVBjjH+xfNDdI2ZHhJh5gj7GG0wrgnjR+g1NWGxCOkby2Esi5GclaY6vZFsXMTXFg3n6Yx4DrESQdeSlTy9FcsGOGyZd1GBAOMJIuAqDxnyknENoyWjEpbAMFrgizBy9Tme2dk1YGdB25Q+9eiFrDums7F0jdGH0KR9Zj/D7D8BqYRBS6cSWMaQ6phzxXrAJxnNzfhY1iYeV6oxkK5ktTfj9WT3+ML2gb7SMayH2fpSQSX2vHe12W9ZD1/AZ0ttNR1A9gXLHWEfXMcuWPlc8C7YAUMed2W0n7gf/z2kZ2vf4BIqADcQVJpdAP13ORnjfq6i2Ft5wlsdNepleBN6bthTK86nLtE4iMYGZw822xLtvkTLuAoY2CKZi5jlcRfvcb4Cop0f4Tae9L04o91r1WscPa1iLmx6Jt8aZjE6xT3vS6f7knNqO+xnS3ZIIPm8xLwz4lKy18btr/Jz53ty2rLMi9hnech6iZEtfxPaczxN81ae8DbP4/Xx3IH1qfOhPO0shuabG+HPVZr8T2WF0/JeCfLnV4X9p+YftkqP6Qk9Q0Veo//v6QhfVv7u/aRf9DtXzp3nqjkrNV1eMphHNHblvv0DnhZnUg==</latexit><latexit sha1_base64="fjSkHt3n88x0XZEZyeYbOK6LyrU=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg6qubhSKe0W58lmhZIQCmeso3Fw5oArVKCSHOuSTooA0ZI8sinFfUImK1IbukhLoIkiurCsa0DqwHVgpWFjQtvBuYHZhtAHmzBkL2oEXD08EZJ528LwTRhvW7FVBjjH+xfNDdI2ZHhJh5gj7GG0wrgnjR+g1NWGxCOkby2Esi5GclaY6vZFsXMTXFg3n6Yx4DrESQdeSlTy9FcsGOGyZd1GBAOMJIuAqDxnyknENoyWjEpbAMFrgizBy9Tme2dk1YGdB25Q+9eiFrDums7F0jdGH0KR9Zj/D7D8BqYRBS6cSWMaQ6phzxXrAJxnNzfhY1iYeV6oxkK5ktTfj9WT3+ML2gb7SMayH2fpSQSX2vHe12W9ZD1/AZ0ttNR1A9gXLHWEfXMcuWPlc8C7YAUMed2W0n7gf/z2kZ2vf4BIqADcQVJpdAP13ORnjfq6i2Ft5wlsdNepleBN6bthTK86nLtE4iMYGZw822xLtvkTLuAoY2CKZi5jlcRfvcb4Cop0f4Tae9L04o91r1WscPa1iLmx6Jt8aZjE6xT3vS6f7knNqO+xnS3ZIIPm8xLwz4lKy18btr/Jz53ty2rLMi9hnech6iZEtfxPaczxN81ae8DbP4/Xx3IH1qfOhPO0shuabG+HPVZr8T2WF0/JeCfLnV4X9p+YftkqP6Qk9Q0Veo//v6QhfVv7u/aRf9DtXzp3nqjkrNV1eMphHNHblvv0DnhZnUg==</latexit><latexit sha1_base64="fjSkHt3n88x0XZEZyeYbOK6LyrU=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg6qubhSKe0W58lmhZIQCmeso3Fw5oArVKCSHOuSTooA0ZI8sinFfUImK1IbukhLoIkiurCsa0DqwHVgpWFjQtvBuYHZhtAHmzBkL2oEXD08EZJ528LwTRhvW7FVBjjH+xfNDdI2ZHhJh5gj7GG0wrgnjR+g1NWGxCOkby2Esi5GclaY6vZFsXMTXFg3n6Yx4DrESQdeSlTy9FcsGOGyZd1GBAOMJIuAqDxnyknENoyWjEpbAMFrgizBy9Tme2dk1YGdB25Q+9eiFrDums7F0jdGH0KR9Zj/D7D8BqYRBS6cSWMaQ6phzxXrAJxnNzfhY1iYeV6oxkK5ktTfj9WT3+ML2gb7SMayH2fpSQSX2vHe12W9ZD1/AZ0ttNR1A9gXLHWEfXMcuWPlc8C7YAUMed2W0n7gf/z2kZ2vf4BIqADcQVJpdAP13ORnjfq6i2Ft5wlsdNepleBN6bthTK86nLtE4iMYGZw822xLtvkTLuAoY2CKZi5jlcRfvcb4Cop0f4Tae9L04o91r1WscPa1iLmx6Jt8aZjE6xT3vS6f7knNqO+xnS3ZIIPm8xLwz4lKy18btr/Jz53ty2rLMi9hnech6iZEtfxPaczxN81ae8DbP4/Xx3IH1qfOhPO0shuabG+HPVZr8T2WF0/JeCfLnV4X9p+YftkqP6Qk9Q0Veo//v6QhfVv7u/aRf9DtXzp3nqjkrNV1eMphHNHblvv0DnhZnUg==</latexit><latexit sha1_base64="fjSkHt3n88x0XZEZyeYbOK6LyrU=">AAAHsHiclVXbbtNAEJ22QNpya8sjLxFpERK0SgISPFZqQTyAKNCb1FSR7WwSE18iexMSrPwKr/BL/AsPnBlvQhPn0tqyd3Z2zpnbrm23PTfWxeKfpeWVW7fv5FbX1u/eu//g4cbm1mkcdiJHnTihF0bnthUrzw3UiXa1p87bkbJ821NnduuA18+6KordMDjW/ba69K1G4NZdx9JQVTe2Ko7qJhXf0k27njQHg6qubhSKe0W58lmhZIQCmeso3Fw5oArVKCSHOuSTooA0ZI8sinFfUImK1IbukhLoIkiurCsa0DqwHVgpWFjQtvBuYHZhtAHmzBkL2oEXD08EZJ528LwTRhvW7FVBjjH+xfNDdI2ZHhJh5gj7GG0wrgnjR+g1NWGxCOkby2Esi5GclaY6vZFsXMTXFg3n6Yx4DrESQdeSlTy9FcsGOGyZd1GBAOMJIuAqDxnyknENoyWjEpbAMFrgizBy9Tme2dk1YGdB25Q+9eiFrDums7F0jdGH0KR9Zj/D7D8BqYRBS6cSWMaQ6phzxXrAJxnNzfhY1iYeV6oxkK5ktTfj9WT3+ML2gb7SMayH2fpSQSX2vHe12W9ZD1/AZ0ttNR1A9gXLHWEfXMcuWPlc8C7YAUMed2W0n7gf/z2kZ2vf4BIqADcQVJpdAP13ORnjfq6i2Ft5wlsdNepleBN6bthTK86nLtE4iMYGZw822xLtvkTLuAoY2CKZi5jlcRfvcb4Cop0f4Tae9L04o91r1WscPa1iLmx6Jt8aZjE6xT3vS6f7knNqO+xnS3ZIIPm8xLwz4lKy18btr/Jz53ty2rLMi9hnech6iZEtfxPaczxN81ae8DbP4/Xx3IH1qfOhPO0shuabG+HPVZr8T2WF0/JeCfLnV4X9p+YftkqP6Qk9Q0Veo//v6QhfVv7u/aRf9DtXzp3nqjkrNV1eMphHNHblvv0DnhZnUg==</latexit>

et
<latexit sha1_base64="XYDWjSnHe/vb664WFbXwk1KEfnc=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZqIGZ/pso1DcLcqVzwolIxTIXAfh5kqF6tSgkBzqkk+KAtKQPbIoxn1CJSpSB7pTSqCLILmyrmhA68B2YaVgYUHbxruF2YnRBpgzZyxoB148PBGQedrG804YbVizVwU5xvgXzw/RtWZ6SISZI+xjtMG4Jowfodd0DotFSN9YDmNZjOSsNDXpjWTjIr6OaDhPZ8Szj5UIuras5OmtWLbAYcv8AhUIMFYRAVd5yJCXjBsYLRmVsASG0QJfhJGrz/HMzq4FOwvac+lTj17IumM6G0vXGL0PTdpn9jPM/hOQShi0dCqBZQypiTlXrAd8ktFcj49lbeJxpRoD6UpWez1eT3aPL2wf6CsdwnqYrS8VVGLPe1eb/Zb18AV8ttRWUwWyL1juCPvgOl6Alc8F74JtMORx10f7ifvx30N6tsoGl1ABuIGg0uwC6L/LyRj3cxnF3vYmvDVRo16GN6Hnhj214nyaEo2DaGxw9mCzJdGWJVrG1cHAFslcxCyPO3iP8xUQ7fwIt/Ck78UZ7VypXuPoaRVzYdMz+TYwi9Ep7nlfOt2XnFPbYT/bskMCyecl5t0Rl5K9Nm5/mZ8735PTlmVexD7LQ9ZLjGz5m9CZ42mat70Jb/M8Xh3PHVifOh/K085iaL65Ef5cpcn/VFY42tstQf78qlB+Yv5hq/SIHtNTVOQ1+v+eDvBldVCTn/SLfuee5Q5ytdy31HR5yWAe0tiVs/8BqV5kUA==</latexit><latexit sha1_base64="XYDWjSnHe/vb664WFbXwk1KEfnc=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZqIGZ/pso1DcLcqVzwolIxTIXAfh5kqF6tSgkBzqkk+KAtKQPbIoxn1CJSpSB7pTSqCLILmyrmhA68B2YaVgYUHbxruF2YnRBpgzZyxoB148PBGQedrG804YbVizVwU5xvgXzw/RtWZ6SISZI+xjtMG4Jowfodd0DotFSN9YDmNZjOSsNDXpjWTjIr6OaDhPZ8Szj5UIuras5OmtWLbAYcv8AhUIMFYRAVd5yJCXjBsYLRmVsASG0QJfhJGrz/HMzq4FOwvac+lTj17IumM6G0vXGL0PTdpn9jPM/hOQShi0dCqBZQypiTlXrAd8ktFcj49lbeJxpRoD6UpWez1eT3aPL2wf6CsdwnqYrS8VVGLPe1eb/Zb18AV8ttRWUwWyL1juCPvgOl6Alc8F74JtMORx10f7ifvx30N6tsoGl1ABuIGg0uwC6L/LyRj3cxnF3vYmvDVRo16GN6Hnhj214nyaEo2DaGxw9mCzJdGWJVrG1cHAFslcxCyPO3iP8xUQ7fwIt/Ck78UZ7VypXuPoaRVzYdMz+TYwi9Ep7nlfOt2XnFPbYT/bskMCyecl5t0Rl5K9Nm5/mZ8735PTlmVexD7LQ9ZLjGz5m9CZ42mat70Jb/M8Xh3PHVifOh/K085iaL65Ef5cpcn/VFY42tstQf78qlB+Yv5hq/SIHtNTVOQ1+v+eDvBldVCTn/SLfuee5Q5ytdy31HR5yWAe0tiVs/8BqV5kUA==</latexit><latexit sha1_base64="XYDWjSnHe/vb664WFbXwk1KEfnc=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZqIGZ/pso1DcLcqVzwolIxTIXAfh5kqF6tSgkBzqkk+KAtKQPbIoxn1CJSpSB7pTSqCLILmyrmhA68B2YaVgYUHbxruF2YnRBpgzZyxoB148PBGQedrG804YbVizVwU5xvgXzw/RtWZ6SISZI+xjtMG4Jowfodd0DotFSN9YDmNZjOSsNDXpjWTjIr6OaDhPZ8Szj5UIuras5OmtWLbAYcv8AhUIMFYRAVd5yJCXjBsYLRmVsASG0QJfhJGrz/HMzq4FOwvac+lTj17IumM6G0vXGL0PTdpn9jPM/hOQShi0dCqBZQypiTlXrAd8ktFcj49lbeJxpRoD6UpWez1eT3aPL2wf6CsdwnqYrS8VVGLPe1eb/Zb18AV8ttRWUwWyL1juCPvgOl6Alc8F74JtMORx10f7ifvx30N6tsoGl1ABuIGg0uwC6L/LyRj3cxnF3vYmvDVRo16GN6Hnhj214nyaEo2DaGxw9mCzJdGWJVrG1cHAFslcxCyPO3iP8xUQ7fwIt/Ck78UZ7VypXuPoaRVzYdMz+TYwi9Ep7nlfOt2XnFPbYT/bskMCyecl5t0Rl5K9Nm5/mZ8735PTlmVexD7LQ9ZLjGz5m9CZ42mat70Jb/M8Xh3PHVifOh/K085iaL65Ef5cpcn/VFY42tstQf78qlB+Yv5hq/SIHtNTVOQ1+v+eDvBldVCTn/SLfuee5Q5ytdy31HR5yWAe0tiVs/8BqV5kUA==</latexit><latexit sha1_base64="XYDWjSnHe/vb664WFbXwk1KEfnc=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZqIGZ/pso1DcLcqVzwolIxTIXAfh5kqF6tSgkBzqkk+KAtKQPbIoxn1CJSpSB7pTSqCLILmyrmhA68B2YaVgYUHbxruF2YnRBpgzZyxoB148PBGQedrG804YbVizVwU5xvgXzw/RtWZ6SISZI+xjtMG4Jowfodd0DotFSN9YDmNZjOSsNDXpjWTjIr6OaDhPZ8Szj5UIuras5OmtWLbAYcv8AhUIMFYRAVd5yJCXjBsYLRmVsASG0QJfhJGrz/HMzq4FOwvac+lTj17IumM6G0vXGL0PTdpn9jPM/hOQShi0dCqBZQypiTlXrAd8ktFcj49lbeJxpRoD6UpWez1eT3aPL2wf6CsdwnqYrS8VVGLPe1eb/Zb18AV8ttRWUwWyL1juCPvgOl6Alc8F74JtMORx10f7ifvx30N6tsoGl1ABuIGg0uwC6L/LyRj3cxnF3vYmvDVRo16GN6Hnhj214nyaEo2DaGxw9mCzJdGWJVrG1cHAFslcxCyPO3iP8xUQ7fwIt/Ck78UZ7VypXuPoaRVzYdMz+TYwi9Ep7nlfOt2XnFPbYT/bskMCyecl5t0Rl5K9Nm5/mZ8735PTlmVexD7LQ9ZLjGz5m9CZ42mat70Jb/M8Xh3PHVifOh/K085iaL65Ef5cpcn/VFY42tstQf78qlB+Yv5hq/SIHtNTVOQ1+v+eDvBldVCTn/SLfuee5Q5ytdy31HR5yWAe0tiVs/8BqV5kUA==</latexit>

e2
<latexit sha1_base64="Jb19Hy94NRcldwg+Pilg6BWpahc=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoAEj5VaEA8gCjRNUVNVtrNJrPgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzHYTdyVMUJvTA6sa1YeW6gKtrVnjrpRMrybU9V7fY+r1cvVBS7YXCk+x115lvNwG24jqWhqtV8S7fsRqIG5+XzjUJxtyhXPiuUjFAgcx2Gmyv7VKM6heRQl3xSFJCG7JFFMe5TKlGROtCdUQJdBMmVdUUDWge2CysFCwvaNt5NzE6NNsCcOWNBO/Di4YmAzNM2nnfCaMOavSrIMca/eH6IrjnTQyLMHGEfow3GNWH8CL2mFiwWIX1jOYxlMZKz0tSgN5KNi/g6ouE8nRHPAVYi6Nqykqe3YtkEhy3zC1QgwFhBBFzlIUNeMq5jtGRUwhIYRgt8EUauPsczO7sm7CxoW9KnHr2Qdcd0NpauMfoAmrTP7GeY/ScglTBo6VQCyxhSA3OuWA/4JKO5Hh/L2sTjSjUG0pWs9nq8nuweX9g+0Fc6gvUwW18qqMSe9642+y3r4Qv4bKmtpn3IvmC5I+yD63gBVj4XvAu2wZDHXRvtJ+7Hfw/p2dozuIQKwA0ElWYXQP9dTsa4n8so9lae8NZAjXoZ3oSeG/bUivNpSDQOorHB2YPNlkS7J9EyrgYGtkjmImZ53MF7nK+AaOdHuIUnfS/OaOdK9RpHT6uYC5ueybeOWYxOcc/70um+5JzaDvvZlh0SSD4vMe+OuJTstXH7y/zc+Z6ctizzIvZZHrJeYmTL34TOHE/TvJUnvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhuLxbgvz5VWHvifmHrdIjekxPUZHX6P97OsSX1UFNftIv+p17ljvMVXPfUtPlJYN5SGNXzv4HzI5kDg==</latexit><latexit sha1_base64="Jb19Hy94NRcldwg+Pilg6BWpahc=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoAEj5VaEA8gCjRNUVNVtrNJrPgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzHYTdyVMUJvTA6sa1YeW6gKtrVnjrpRMrybU9V7fY+r1cvVBS7YXCk+x115lvNwG24jqWhqtV8S7fsRqIG5+XzjUJxtyhXPiuUjFAgcx2Gmyv7VKM6heRQl3xSFJCG7JFFMe5TKlGROtCdUQJdBMmVdUUDWge2CysFCwvaNt5NzE6NNsCcOWNBO/Di4YmAzNM2nnfCaMOavSrIMca/eH6IrjnTQyLMHGEfow3GNWH8CL2mFiwWIX1jOYxlMZKz0tSgN5KNi/g6ouE8nRHPAVYi6Nqykqe3YtkEhy3zC1QgwFhBBFzlIUNeMq5jtGRUwhIYRgt8EUauPsczO7sm7CxoW9KnHr2Qdcd0NpauMfoAmrTP7GeY/ScglTBo6VQCyxhSA3OuWA/4JKO5Hh/L2sTjSjUG0pWs9nq8nuweX9g+0Fc6gvUwW18qqMSe9642+y3r4Qv4bKmtpn3IvmC5I+yD63gBVj4XvAu2wZDHXRvtJ+7Hfw/p2dozuIQKwA0ElWYXQP9dTsa4n8so9lae8NZAjXoZ3oSeG/bUivNpSDQOorHB2YPNlkS7J9EyrgYGtkjmImZ53MF7nK+AaOdHuIUnfS/OaOdK9RpHT6uYC5ueybeOWYxOcc/70um+5JzaDvvZlh0SSD4vMe+OuJTstXH7y/zc+Z6ctizzIvZZHrJeYmTL34TOHE/TvJUnvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhuLxbgvz5VWHvifmHrdIjekxPUZHX6P97OsSX1UFNftIv+p17ljvMVXPfUtPlJYN5SGNXzv4HzI5kDg==</latexit><latexit sha1_base64="Jb19Hy94NRcldwg+Pilg6BWpahc=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoAEj5VaEA8gCjRNUVNVtrNJrPgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzHYTdyVMUJvTA6sa1YeW6gKtrVnjrpRMrybU9V7fY+r1cvVBS7YXCk+x115lvNwG24jqWhqtV8S7fsRqIG5+XzjUJxtyhXPiuUjFAgcx2Gmyv7VKM6heRQl3xSFJCG7JFFMe5TKlGROtCdUQJdBMmVdUUDWge2CysFCwvaNt5NzE6NNsCcOWNBO/Di4YmAzNM2nnfCaMOavSrIMca/eH6IrjnTQyLMHGEfow3GNWH8CL2mFiwWIX1jOYxlMZKz0tSgN5KNi/g6ouE8nRHPAVYi6Nqykqe3YtkEhy3zC1QgwFhBBFzlIUNeMq5jtGRUwhIYRgt8EUauPsczO7sm7CxoW9KnHr2Qdcd0NpauMfoAmrTP7GeY/ScglTBo6VQCyxhSA3OuWA/4JKO5Hh/L2sTjSjUG0pWs9nq8nuweX9g+0Fc6gvUwW18qqMSe9642+y3r4Qv4bKmtpn3IvmC5I+yD63gBVj4XvAu2wZDHXRvtJ+7Hfw/p2dozuIQKwA0ElWYXQP9dTsa4n8so9lae8NZAjXoZ3oSeG/bUivNpSDQOorHB2YPNlkS7J9EyrgYGtkjmImZ53MF7nK+AaOdHuIUnfS/OaOdK9RpHT6uYC5ueybeOWYxOcc/70um+5JzaDvvZlh0SSD4vMe+OuJTstXH7y/zc+Z6ctizzIvZZHrJeYmTL34TOHE/TvJUnvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhuLxbgvz5VWHvifmHrdIjekxPUZHX6P97OsSX1UFNftIv+p17ljvMVXPfUtPlJYN5SGNXzv4HzI5kDg==</latexit><latexit sha1_base64="Jb19Hy94NRcldwg+Pilg6BWpahc=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoAEj5VaEA8gCjRNUVNVtrNJrPgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzHYTdyVMUJvTA6sa1YeW6gKtrVnjrpRMrybU9V7fY+r1cvVBS7YXCk+x115lvNwG24jqWhqtV8S7fsRqIG5+XzjUJxtyhXPiuUjFAgcx2Gmyv7VKM6heRQl3xSFJCG7JFFMe5TKlGROtCdUQJdBMmVdUUDWge2CysFCwvaNt5NzE6NNsCcOWNBO/Di4YmAzNM2nnfCaMOavSrIMca/eH6IrjnTQyLMHGEfow3GNWH8CL2mFiwWIX1jOYxlMZKz0tSgN5KNi/g6ouE8nRHPAVYi6Nqykqe3YtkEhy3zC1QgwFhBBFzlIUNeMq5jtGRUwhIYRgt8EUauPsczO7sm7CxoW9KnHr2Qdcd0NpauMfoAmrTP7GeY/ScglTBo6VQCyxhSA3OuWA/4JKO5Hh/L2sTjSjUG0pWs9nq8nuweX9g+0Fc6gvUwW18qqMSe9642+y3r4Qv4bKmtpn3IvmC5I+yD63gBVj4XvAu2wZDHXRvtJ+7Hfw/p2dozuIQKwA0ElWYXQP9dTsa4n8so9lae8NZAjXoZ3oSeG/bUivNpSDQOorHB2YPNlkS7J9EyrgYGtkjmImZ53MF7nK+AaOdHuIUnfS/OaOdK9RpHT6uYC5ueybeOWYxOcc/70um+5JzaDvvZlh0SSD4vMe+OuJTstXH7y/zc+Z6ctizzIvZZHrJeYmTL34TOHE/TvJUnvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhuLxbgvz5VWHvifmHrdIjekxPUZHX6P97OsSX1UFNftIv+p17ljvMVXPfUtPlJYN5SGNXzv4HzI5kDg==</latexit>

e1
<latexit sha1_base64="xsc7pt4lmUi/aHBC8GGYDvxrhg0=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZqIGZ6WzjUJxtyhXPiuUjFAgcx2EmysVqlODQnKoSz4pCkhD9siiGPcJlahIHehOKYEuguTKuqIBrQPbhZWChQVtG+8WZidGG2DOnLGgHXjx8ERA5mkbzzthtGHNXhXkGONfPD9E15rpIRFmjrCP0QbjmjB+hF7TOSwWIX1jOYxlMZKz0tSkN5KNi/g6ouE8nRHPPlYi6Nqykqe3YtkChy3zC1QgwFhFBFzlIUNeMm5gtGRUwhIYRgt8EUauPsczO7sW7Cxoz6VPPXoh647pbCxdY/Q+NGmf2c8w+09AKmHQ0qkEljGkJuZcsR7wSUZzPT6WtYnHlWoMpCtZ7fV4Pdk9vrB9oK90COthtr5UUIk9711t9lvWwxfw2VJbTRXIvmC5I+yD63gBVj4XvAu2wZDHXR/tJ+7Hfw/p2SobXEIF4AaCSrMLoP8uJ2Pcz2UUe9ub8NZEjXoZ3oSeG/bUivNpSjQOorHB2YPNlkRblmgZVwcDWyRzEbM87uA9zldAtPMj3MKTvhdntHOleo2jp1XMhU3P5NvALEanuOd96XRfck5th/1syw4JJJ+XmHdHXEr22rj9ZX7ufE9OW5Z5EfssD1kvMbLlb0Jnjqdp3vYmvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhaG+3BPnzq0L5ifmHrdIjekxPUZHX6P97OsCX1UFNftIv+p17ljvI1XLfUtPlJYN5SGNXzv4HxVVkDQ==</latexit><latexit sha1_base64="xsc7pt4lmUi/aHBC8GGYDvxrhg0=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZqIGZ6WzjUJxtyhXPiuUjFAgcx2EmysVqlODQnKoSz4pCkhD9siiGPcJlahIHehOKYEuguTKuqIBrQPbhZWChQVtG+8WZidGG2DOnLGgHXjx8ERA5mkbzzthtGHNXhXkGONfPD9E15rpIRFmjrCP0QbjmjB+hF7TOSwWIX1jOYxlMZKz0tSkN5KNi/g6ouE8nRHPPlYi6Nqykqe3YtkChy3zC1QgwFhFBFzlIUNeMm5gtGRUwhIYRgt8EUauPsczO7sW7Cxoz6VPPXoh647pbCxdY/Q+NGmf2c8w+09AKmHQ0qkEljGkJuZcsR7wSUZzPT6WtYnHlWoMpCtZ7fV4Pdk9vrB9oK90COthtr5UUIk9711t9lvWwxfw2VJbTRXIvmC5I+yD63gBVj4XvAu2wZDHXR/tJ+7Hfw/p2SobXEIF4AaCSrMLoP8uJ2Pcz2UUe9ub8NZEjXoZ3oSeG/bUivNpSjQOorHB2YPNlkRblmgZVwcDWyRzEbM87uA9zldAtPMj3MKTvhdntHOleo2jp1XMhU3P5NvALEanuOd96XRfck5th/1syw4JJJ+XmHdHXEr22rj9ZX7ufE9OW5Z5EfssD1kvMbLlb0Jnjqdp3vYmvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhaG+3BPnzq0L5ifmHrdIjekxPUZHX6P97OsCX1UFNftIv+p17ljvI1XLfUtPlJYN5SGNXzv4HxVVkDQ==</latexit><latexit sha1_base64="xsc7pt4lmUi/aHBC8GGYDvxrhg0=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZqIGZ6WzjUJxtyhXPiuUjFAgcx2EmysVqlODQnKoSz4pCkhD9siiGPcJlahIHehOKYEuguTKuqIBrQPbhZWChQVtG+8WZidGG2DOnLGgHXjx8ERA5mkbzzthtGHNXhXkGONfPD9E15rpIRFmjrCP0QbjmjB+hF7TOSwWIX1jOYxlMZKz0tSkN5KNi/g6ouE8nRHPPlYi6Nqykqe3YtkChy3zC1QgwFhFBFzlIUNeMm5gtGRUwhIYRgt8EUauPsczO7sW7Cxoz6VPPXoh647pbCxdY/Q+NGmf2c8w+09AKmHQ0qkEljGkJuZcsR7wSUZzPT6WtYnHlWoMpCtZ7fV4Pdk9vrB9oK90COthtr5UUIk9711t9lvWwxfw2VJbTRXIvmC5I+yD63gBVj4XvAu2wZDHXR/tJ+7Hfw/p2SobXEIF4AaCSrMLoP8uJ2Pcz2UUe9ub8NZEjXoZ3oSeG/bUivNpSjQOorHB2YPNlkRblmgZVwcDWyRzEbM87uA9zldAtPMj3MKTvhdntHOleo2jp1XMhU3P5NvALEanuOd96XRfck5th/1syw4JJJ+XmHdHXEr22rj9ZX7ufE9OW5Z5EfssD1kvMbLlb0Jnjqdp3vYmvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhaG+3BPnzq0L5ifmHrdIjekxPUZHX6P97OsCX1UFNftIv+p17ljvI1XLfUtPlJYN5SGNXzv4HxVVkDQ==</latexit><latexit sha1_base64="xsc7pt4lmUi/aHBC8GGYDvxrhg0=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZqIGZ6WzjUJxtyhXPiuUjFAgcx2EmysVqlODQnKoSz4pCkhD9siiGPcJlahIHehOKYEuguTKuqIBrQPbhZWChQVtG+8WZidGG2DOnLGgHXjx8ERA5mkbzzthtGHNXhXkGONfPD9E15rpIRFmjrCP0QbjmjB+hF7TOSwWIX1jOYxlMZKz0tSkN5KNi/g6ouE8nRHPPlYi6Nqykqe3YtkChy3zC1QgwFhFBFzlIUNeMm5gtGRUwhIYRgt8EUauPsczO7sW7Cxoz6VPPXoh647pbCxdY/Q+NGmf2c8w+09AKmHQ0qkEljGkJuZcsR7wSUZzPT6WtYnHlWoMpCtZ7fV4Pdk9vrB9oK90COthtr5UUIk9711t9lvWwxfw2VJbTRXIvmC5I+yD63gBVj4XvAu2wZDHXR/tJ+7Hfw/p2SobXEIF4AaCSrMLoP8uJ2Pcz2UUe9ub8NZEjXoZ3oSeG/bUivNpSjQOorHB2YPNlkRblmgZVwcDWyRzEbM87uA9zldAtPMj3MKTvhdntHOleo2jp1XMhU3P5NvALEanuOd96XRfck5th/1syw4JJJ+XmHdHXEr22rj9ZX7ufE9OW5Z5EfssD1kvMbLlb0Jnjqdp3vYmvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhaG+3BPnzq0L5ifmHrdIjekxPUZHX6P97OsCX1UFNftIv+p17ljvI1XLfUtPlJYN5SGNXzv4HxVVkDQ==</latexit>

lt
<latexit sha1_base64="H1u+oqlb/GBf+bh0c/cV3LP/6XM=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZuINzvTZRqG4W5QrnxVKRiiQuQ7CzZUK1alBITnUJZ8UBaQhe2RRjPuESlSkDnSnlEAXQXJlXdGA1oHtwkrBwoK2jXcLsxOjDTBnzljQDrx4eCIg87SN550w2rBmrwpyjPEvnh+ia830kAgzR9jHaINxTRg/Qq/pHBaLkL6xHMayGMlZaWrSG8nGRXwd0XCezohnHysRdG1ZydNbsWyBw5b5BSoQYKwiAq7ykCEvGTcwWjIqYQkMowW+CCNXn+OZnV0Ldha059KnHr2Qdcd0NpauMXofmrTP7GeY/ScglTBo6VQCyxhSE3OuWA/4JKO5Hh/L2sTjSjUG0pWs9nq8nuweX9g+0Fc6hPUwW18qqMSe9642+y3r4Qv4bKmtpgpkX7DcEfbBdbwAK58L3gXbYMjjro/2E/fjv4f0bJUNLqECcANBpdkF0H+XkzHu5zKKve1NeGuiRr0Mb0LPDXtqxfk0JRoH0djg7MFmS6ItS7SMq4OBLZK5iFked/Ae5ysg2vkRbuFJ34sz2rlSvcbR0yrmwqZn8m1gFqNT3PO+dLovOae2w362ZYcEks9LzLsjLiV7bdz+Mj93vienLcu8iH2Wh6yXGNnyN6Ezx9M0b3sT3uZ5vDqeO7A+dT6Up53F0HxzI/y5SpP/qaxwtLdbgvz5VaH8xPzDVukRPaanqMhr9P89HeDL6qAmP+kX/c49yx3karlvqenyksE8pLErZ/8D3AJkVw==</latexit><latexit sha1_base64="H1u+oqlb/GBf+bh0c/cV3LP/6XM=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZuINzvTZRqG4W5QrnxVKRiiQuQ7CzZUK1alBITnUJZ8UBaQhe2RRjPuESlSkDnSnlEAXQXJlXdGA1oHtwkrBwoK2jXcLsxOjDTBnzljQDrx4eCIg87SN550w2rBmrwpyjPEvnh+ia830kAgzR9jHaINxTRg/Qq/pHBaLkL6xHMayGMlZaWrSG8nGRXwd0XCezohnHysRdG1ZydNbsWyBw5b5BSoQYKwiAq7ykCEvGTcwWjIqYQkMowW+CCNXn+OZnV0Ldha059KnHr2Qdcd0NpauMXofmrTP7GeY/ScglTBo6VQCyxhSE3OuWA/4JKO5Hh/L2sTjSjUG0pWs9nq8nuweX9g+0Fc6hPUwW18qqMSe9642+y3r4Qv4bKmtpgpkX7DcEfbBdbwAK58L3gXbYMjjro/2E/fjv4f0bJUNLqECcANBpdkF0H+XkzHu5zKKve1NeGuiRr0Mb0LPDXtqxfk0JRoH0djg7MFmS6ItS7SMq4OBLZK5iFked/Ae5ysg2vkRbuFJ34sz2rlSvcbR0yrmwqZn8m1gFqNT3PO+dLovOae2w362ZYcEks9LzLsjLiV7bdz+Mj93vienLcu8iH2Wh6yXGNnyN6Ezx9M0b3sT3uZ5vDqeO7A+dT6Up53F0HxzI/y5SpP/qaxwtLdbgvz5VaH8xPzDVukRPaanqMhr9P89HeDL6qAmP+kX/c49yx3karlvqenyksE8pLErZ/8D3AJkVw==</latexit><latexit sha1_base64="H1u+oqlb/GBf+bh0c/cV3LP/6XM=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZuINzvTZRqG4W5QrnxVKRiiQuQ7CzZUK1alBITnUJZ8UBaQhe2RRjPuESlSkDnSnlEAXQXJlXdGA1oHtwkrBwoK2jXcLsxOjDTBnzljQDrx4eCIg87SN550w2rBmrwpyjPEvnh+ia830kAgzR9jHaINxTRg/Qq/pHBaLkL6xHMayGMlZaWrSG8nGRXwd0XCezohnHysRdG1ZydNbsWyBw5b5BSoQYKwiAq7ykCEvGTcwWjIqYQkMowW+CCNXn+OZnV0Ldha059KnHr2Qdcd0NpauMXofmrTP7GeY/ScglTBo6VQCyxhSE3OuWA/4JKO5Hh/L2sTjSjUG0pWs9nq8nuweX9g+0Fc6hPUwW18qqMSe9642+y3r4Qv4bKmtpgpkX7DcEfbBdbwAK58L3gXbYMjjro/2E/fjv4f0bJUNLqECcANBpdkF0H+XkzHu5zKKve1NeGuiRr0Mb0LPDXtqxfk0JRoH0djg7MFmS6ItS7SMq4OBLZK5iFked/Ae5ysg2vkRbuFJ34sz2rlSvcbR0yrmwqZn8m1gFqNT3PO+dLovOae2w362ZYcEks9LzLsjLiV7bdz+Mj93vienLcu8iH2Wh6yXGNnyN6Ezx9M0b3sT3uZ5vDqeO7A+dT6Up53F0HxzI/y5SpP/qaxwtLdbgvz5VaH8xPzDVukRPaanqMhr9P89HeDL6qAmP+kX/c49yx3karlvqenyksE8pLErZ/8D3AJkVw==</latexit><latexit sha1_base64="H1u+oqlb/GBf+bh0c/cV3LP/6XM=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZuINzvTZRqG4W5QrnxVKRiiQuQ7CzZUK1alBITnUJZ8UBaQhe2RRjPuESlSkDnSnlEAXQXJlXdGA1oHtwkrBwoK2jXcLsxOjDTBnzljQDrx4eCIg87SN550w2rBmrwpyjPEvnh+ia830kAgzR9jHaINxTRg/Qq/pHBaLkL6xHMayGMlZaWrSG8nGRXwd0XCezohnHysRdG1ZydNbsWyBw5b5BSoQYKwiAq7ykCEvGTcwWjIqYQkMowW+CCNXn+OZnV0Ldha059KnHr2Qdcd0NpauMXofmrTP7GeY/ScglTBo6VQCyxhSE3OuWA/4JKO5Hh/L2sTjSjUG0pWs9nq8nuweX9g+0Fc6hPUwW18qqMSe9642+y3r4Qv4bKmtpgpkX7DcEfbBdbwAK58L3gXbYMjjro/2E/fjv4f0bJUNLqECcANBpdkF0H+XkzHu5zKKve1NeGuiRr0Mb0LPDXtqxfk0JRoH0djg7MFmS6ItS7SMq4OBLZK5iFked/Ae5ysg2vkRbuFJ34sz2rlSvcbR0yrmwqZn8m1gFqNT3PO+dLovOae2w362ZYcEks9LzLsjLiV7bdz+Mj93vienLcu8iH2Wh6yXGNnyN6Ezx9M0b3sT3uZ5vDqeO7A+dT6Up53F0HxzI/y5SpP/qaxwtLdbgvz5VaH8xPzDVukRPaanqMhr9P89HeDL6qAmP+kX/c49yx3karlvqenyksE8pLErZ/8D3AJkVw==</latexit>

yt
<latexit sha1_base64="1O62eFKHj55B+NpfluuE5BPq7l8=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZtIfnOmzjUJxtyhXPiuUjFAgcx2EmysVqlODQnKoSz4pCkhD9siiGPcJlahIHehOKYEuguTKuqIBrQPbhZWChQVtG+8WZidGG2DOnLGgHXjx8ERA5mkbzzthtGHNXhXkGONfPD9E15rpIRFmjrCP0QbjmjB+hF7TOSwWIX1jOYxlMZKz0tSkN5KNi/g6ouE8nRHPPlYi6Nqykqe3YtkChy3zC1QgwFhFBFzlIUNeMm5gtGRUwhIYRgt8EUauPsczO7sW7Cxoz6VPPXoh647pbCxdY/Q+NGmf2c8w+09AKmHQ0qkEljGkJuZcsR7wSUZzPT6WtYnHlWoMpCtZ7fV4Pdk9vrB9oK90COthtr5UUIk9711t9lvWwxfw2VJbTRXIvmC5I+yD63gBVj4XvAu2wZDHXR/tJ+7Hfw/p2SobXEIF4AaCSrMLoP8uJ2Pcz2UUe9ub8NZEjXoZ3oSeG/bUivNpSjQOorHB2YPNlkRblmgZVwcDWyRzEbM87uA9zldAtPMj3MKTvhdntHOleo2jp1XMhU3P5NvALEanuOd96XRfck5th/1syw4JJJ+XmHdHXEr22rj9ZX7ufE9OW5Z5EfssD1kvMbLlb0Jnjqdp3vYmvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhaG+3BPnzq0L5ifmHrdIjekxPUZHX6P97OsCX1UFNftIv+p17ljvI1XLfUtPlJYN5SGNXzv4HOh1kZA==</latexit><latexit sha1_base64="1O62eFKHj55B+NpfluuE5BPq7l8=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZtIfnOmzjUJxtyhXPiuUjFAgcx2EmysVqlODQnKoSz4pCkhD9siiGPcJlahIHehOKYEuguTKuqIBrQPbhZWChQVtG+8WZidGG2DOnLGgHXjx8ERA5mkbzzthtGHNXhXkGONfPD9E15rpIRFmjrCP0QbjmjB+hF7TOSwWIX1jOYxlMZKz0tSkN5KNi/g6ouE8nRHPPlYi6Nqykqe3YtkChy3zC1QgwFhFBFzlIUNeMm5gtGRUwhIYRgt8EUauPsczO7sW7Cxoz6VPPXoh647pbCxdY/Q+NGmf2c8w+09AKmHQ0qkEljGkJuZcsR7wSUZzPT6WtYnHlWoMpCtZ7fV4Pdk9vrB9oK90COthtr5UUIk9711t9lvWwxfw2VJbTRXIvmC5I+yD63gBVj4XvAu2wZDHXR/tJ+7Hfw/p2SobXEIF4AaCSrMLoP8uJ2Pcz2UUe9ub8NZEjXoZ3oSeG/bUivNpSjQOorHB2YPNlkRblmgZVwcDWyRzEbM87uA9zldAtPMj3MKTvhdntHOleo2jp1XMhU3P5NvALEanuOd96XRfck5th/1syw4JJJ+XmHdHXEr22rj9ZX7ufE9OW5Z5EfssD1kvMbLlb0Jnjqdp3vYmvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhaG+3BPnzq0L5ifmHrdIjekxPUZHX6P97OsCX1UFNftIv+p17ljvI1XLfUtPlJYN5SGNXzv4HOh1kZA==</latexit><latexit sha1_base64="1O62eFKHj55B+NpfluuE5BPq7l8=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZtIfnOmzjUJxtyhXPiuUjFAgcx2EmysVqlODQnKoSz4pCkhD9siiGPcJlahIHehOKYEuguTKuqIBrQPbhZWChQVtG+8WZidGG2DOnLGgHXjx8ERA5mkbzzthtGHNXhXkGONfPD9E15rpIRFmjrCP0QbjmjB+hF7TOSwWIX1jOYxlMZKz0tSkN5KNi/g6ouE8nRHPPlYi6Nqykqe3YtkChy3zC1QgwFhFBFzlIUNeMm5gtGRUwhIYRgt8EUauPsczO7sW7Cxoz6VPPXoh647pbCxdY/Q+NGmf2c8w+09AKmHQ0qkEljGkJuZcsR7wSUZzPT6WtYnHlWoMpCtZ7fV4Pdk9vrB9oK90COthtr5UUIk9711t9lvWwxfw2VJbTRXIvmC5I+yD63gBVj4XvAu2wZDHXR/tJ+7Hfw/p2SobXEIF4AaCSrMLoP8uJ2Pcz2UUe9ub8NZEjXoZ3oSeG/bUivNpSjQOorHB2YPNlkRblmgZVwcDWyRzEbM87uA9zldAtPMj3MKTvhdntHOleo2jp1XMhU3P5NvALEanuOd96XRfck5th/1syw4JJJ+XmHdHXEr22rj9ZX7ufE9OW5Z5EfssD1kvMbLlb0Jnjqdp3vYmvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhaG+3BPnzq0L5ifmHrdIjekxPUZHX6P97OsCX1UFNftIv+p17ljvI1XLfUtPlJYN5SGNXzv4HOh1kZA==</latexit><latexit sha1_base64="1O62eFKHj55B+NpfluuE5BPq7l8=">AAAHqHiclVXbbtNAEJ22QNpya+GRl4i0CAGtkoIEj5FSEA8gCjRNUVNVtrNJrfgS2ZuSYOU3eIXf4l944Mx4E5o4l9aWvbOzc87cdm2747mxLhb/LC2v3Lh5K7e6tn77zt179zc2HxzFYTdyVNUJvTA6tq1YeW6gqtrVnjruRMrybU/V7HaF12sXKordMDjU/Y469a1W4DZdx9JQ1eu+pc/tZtIfnOmzjUJxtyhXPiuUjFAgcx2EmysVqlODQnKoSz4pCkhD9siiGPcJlahIHehOKYEuguTKuqIBrQPbhZWChQVtG+8WZidGG2DOnLGgHXjx8ERA5mkbzzthtGHNXhXkGONfPD9E15rpIRFmjrCP0QbjmjB+hF7TOSwWIX1jOYxlMZKz0tSkN5KNi/g6ouE8nRHPPlYi6Nqykqe3YtkChy3zC1QgwFhFBFzlIUNeMm5gtGRUwhIYRgt8EUauPsczO7sW7Cxoz6VPPXoh647pbCxdY/Q+NGmf2c8w+09AKmHQ0qkEljGkJuZcsR7wSUZzPT6WtYnHlWoMpCtZ7fV4Pdk9vrB9oK90COthtr5UUIk9711t9lvWwxfw2VJbTRXIvmC5I+yD63gBVj4XvAu2wZDHXR/tJ+7Hfw/p2SobXEIF4AaCSrMLoP8uJ2Pcz2UUe9ub8NZEjXoZ3oSeG/bUivNpSjQOorHB2YPNlkRblmgZVwcDWyRzEbM87uA9zldAtPMj3MKTvhdntHOleo2jp1XMhU3P5NvALEanuOd96XRfck5th/1syw4JJJ+XmHdHXEr22rj9ZX7ufE9OW5Z5EfssD1kvMbLlb0Jnjqdp3vYmvM3zeHU8d2B96nwoTzuLofnmRvhzlSb/U1nhaG+3BPnzq0L5ifmHrdIjekxPUZHX6P97OsCX1UFNftIv+p17ljvI1XLfUtPlJYN5SGNXzv4HOh1kZA==</latexit>

Figure 4: The architecture of the proposed BiLSTM model with self attention. For each time t,
the exchange level parameter of all exchanges ei of the sub-dialogue i ∈ {1 . . . t} are
encoded to their respective hidden representation hi and are considered and weighted
with the self attention mechanism to finally estimate the IQ value yt at time t.

As shown in Figure 4, the exchange level parameters form the input vector et for each time step
or turn t to a bi-directional LSTM (Graves et al., 2013) layer. The input vector et encodes the nom-
inal parameters ASRRecognitionStatus, ActivityType, and Confirmation? as 1-hot representations.
In the BiLSTM layer, two hidden states are computed: ~ht constitutes the forward pass through the
current sub-dialogue and ~ht the backwards pass:

~ht = LSTM(et, ~ht−1) (3)
~ht = LSTM(et, ~ht+1) (4)

The final hidden layer is then computed by concatenating both hidden states:

ht = [~ht, ~ht] . (5)

Even though information from all time steps may contribute to the final IQ value, not all time
steps may be equally important. Thus, an attention mechanism (Vaswani et al., 2017) is used that

89



ULTES AND MAIER

evaluates the importance of each time step t′ for estimating the IQ value at time t by calculating a
weight vector αt,t′ .

gt,t′ = tanh(hT
t Wt + hT

t′Wt′ + bt) (6)

αt,t′ = softmax(σ(Wagt,t′ + ba)) (7)

lt =
∑
t′

αt,t′ht′ (8)

Zheng et al. (2018) describe this as follows: “The attention-focused hidden state representation lt of
an [exchange] at time step t is given by the weighted summation of the hidden state representation
ht′ of all [exchanges] at time steps t′, and their similarity αt,t′ to the hidden state representation ht

of the current [exchange]. Essentially, lt dictates how much to attend to an [exchange] at any time
step conditioned on their neighbourhood context.”

To calculate the final estimate yt of the current IQ value at time t, a softmax layer is introduced:

yt = softmax(lt) (9)

For estimating the interaction quality using a BiLSTM, the proposed architecture frames the
task as a classification problem where each sequence is labelled with one IQ value. Thus, for each
time step t, the IQ value needs to be estimated for the corresponding sub-dialogue consisting of
all exchanges from the beginning up to t. Framing the problem like this is necessary to allow the
application of a BiLSTM-approach while still being able to only use information that would be
present at the current time step t in an ongoing dialogue interaction.

To analyse the influence of the BiLSTM, a model with a single forward-LSTM layer is also
investigated where

ht = ~ht . (10)

Similarly, a model without attention is also analysed where

lt = ht . (11)

A deep learning approach using only non-temporal features has been previously proposed and
achieved an UAR on the full feature set of 0.55 (Rach et al., 2017).

4. Simulation Experiments and Results

The IQ estimators are both trained and evaluated on the LEGO corpus and applied within the IQ
reward estimation framework (Figure 1) on several domains within a simulated environment.

4.1 Interaction Quality Estimation

To evaluate the BiLSTM model with attention (BiLSTM+att), it is compared with three of its own
variants: a BiLSTM without attention (BiLSTM) as well as a single forward-LSTM layer with
attention (LSTM+att) and without attention (LSTM). An additional baseline is defined by Rach et al.
(2017) who already proposed an LSTM-based architecture that only uses non-temporal features.
Furthermore, the conventional IQ estimator using a linear SVM is evaluated as originally used for
reward estimation by Ultes et al. (2015).

90



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

Table 3: Performance of the proposed LSTM-based variants with the traditional cross-validation
setup. Due to overlapping sub-dialogues in the train and test sets, the performance of the
LSTM-based models achieve unrealistically high performance. Significant differences are
observed between BiLSTM and all other variants/models (p < 0.05, Wilcoxon signed-
rank test (Wilcoxon, 1945)).

UAR κ ρ eA Ep.

LSTM 0.78 0.85 0.91 0.99 101
BiLSTM 0.78 0.85 0.92 0.99 100
LSTM+att 0.74 0.82 0.91 0.99 101
BiLSTM+att 0.75 0.83 0.91 0.99 93

Rach et al. (2017) 0.55 0.68 0.83 0.94 -
Ultes et al. (2015) 0.55 - - 0.89 -

The deep neural net models have been implemented with Keras4 using the self-attention imple-
mentation as provided by Zheng et al. (2018)5. The input vector consists of one-hot-encodings of
the three nominal features ASRRecognitionStatus, ActivityType, and Confirmation? and the nu-
merical features ASRConfidence and RePrompt? resulting in a vector of size 11. The target IQ
labels are also one-hot-encoded. The LSTM and BiLSTM embeddings have a dimension of 64 and
128, respectively. The maximum dialogue length has been set to 60: dialogues with fewer turns
are padded wiht zero vectors and dialogues with more turns are truncated. All models were trained
against cross-entropy loss using RmsProp (Tieleman and Hinton, 2012) optimisation with a learning
rate of 0.001 and a mini-batch size of 32.

Interaction Quality estimation is evaluated by using three commonly applied evaluation metrics:
Unweighted Average Recall (UAR), Cohen’s Kappa (Cohen, 1960), and Spearman’s Rho (Spear-
man, 1904) comparing the estimated IQ ratings x with the true IQ ratings y. As missing the correct
estimated IQ value by only one has little impact for modelling the reward, a measure we call the
extended accuracy (eA) is used where neighbouring values are taken into account as well.

Recall in general is defined as the rate of correctly classified samples belonging to one class.
The unweighted average recall for multi-class classification problems with C classes is computed
by the class-wise recalls recallc for each class c and then averaged over all class-wise recalls:

UAR =
1

C

N∑
c=1

recallc . (12)

Cohen’s Kappa measures the relative agreement between two corresponding sets of ratings,
here the estimate x and ground truth y. In our case, Cohen’s weighted kappa is applied as ordinal
scores are compared (Cohen, 1968):

κ = 1−
∑C

c=1

∑C
k=1wck ·mck∑C

c=1

∑C
k=1wck · mc.·m.k

N

. (13)

4. https://keras.io/
5. Code freely available at https://github.com/CyberZHG/keras-self-attention

91



ULTES AND MAIER

mc,k is the number of samples where, for a corresponding (xi, yi) pair, the estimator predicted k for
the true class c, and N is the total number of samples. mc. represents the sum of all estimated class
samples for true class c and m.k the sum of all true class samples for estimated class k. A weighting
factor w is introduced reducing the discount of disagreements the smaller the difference is between
two ratings:

wck =
|rc − rk|

|rmax − rmin|
. (14)

Here, rc and rk denote the rating pair and rmax and rmin the maximum and minimum ratings
possible.

The correlation of two variables describes the degree by that one variable can be expressed
by the other. Spearman’s Rank Correlation Coefficient is a non-parametric method assuming a
monotonic function between the two variables (Spearman, 1904). It is defined by

ρ =

∑
i(xi − x̄)(yi − ȳ)√∑

i(xi − x̄)2
∑

i(yi − ȳ)2
, (15)

where xi and yi are corresponding ranked ratings and x̄ and ȳ the mean ranks. Thus, two sets of
ratings can have total correlation even if they never agree. This would happen if all ratings are
shifted by the same value, for example.

The extended accuracy is computed similar to regular accuracy. However, instead of only using
the main diagonal of the confusion matrix, the two secondary diagonals are considered additionally.
For N classes, the extended accuracy is computed by

eA =

∑C
c=1mc,c +mc,c−1 +mc,c+1∑N

c=1

∑N
k=1mc,k

, (16)

where mc,k is the number of samples where the estimator predicted k for the true class c. Thus,
neighbouring values are taken into account as well and being off by one is still considered as correct
estimation6.

All experiments were conducted with the LEGO corpus (Schmitt et al., 2012a) in a 10-fold
cross-validation setup for a total of 100 epochs per fold. The results are presented in Table 3. Due
to the way the task is framed (one label for each sub-dialogue), memorising effects may be observed
with the traditional cross-validation setup that has been used in previous work. Hence, the results
in Table 3 show very high performance, which is likely to further increase with ongoing training.
However, the corresponding models are likely to generalise poorly.

To alleviate this, a dialogue-wise cross-validation setup has been employed also consisting of
10 folds of disjoint sets of dialogues. By that, it can be guaranteed that there are no overlapping sub-
dialogues in the training and test sets. All results of these experiments are presented in Table 4 with
the absolute improvement of the two main measures UAR and eA over the SVM-based conventional
approach of Ultes et al. (2015) visualised in Figure 5.

The BiLSTM+att model outperforms the other models and baselines in all four performance
measures by achieving an UAR of 0.54 and an eA of 0.94 after 40 epochs. Furthermore, both the
BiLSTM and the attention mechanism by themselves improve the performance in terms of UAR.

92



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

Table 4: Performance of the proposed LSTM-based variants with the dialogue-wise cross-
validation setup. The models by Rach et al. (2017) and Ultes et al. (2015) have been
re-implemented. The BiLSTM with attention mechanism performs best in all evaluation
metrics. All results are significantly different to each other (p < 0.05, Wilcoxon signed-
rank test (Wilcoxon, 1945)).

UAR κ ρ eA Ep.

LSTM 0.51 0.63 0.78 0.93 8
BiLSTM 0.53 0.63 0.78 0.93 8
LSTM+att 0.52 0.63 0.79 0.92 40
BiLSTM+att 0.54 0.65 0.81 0.94 40

Rach et al. (2017) 0.45 0.58 0.79 0.88 82
Ultes et al. (2015) 0.44 0.53 0.69 0.86 -

+0
.0

7

+0
.0

7+0
.0

9

+0
.0

7

+0
.0

8

+0
.0

6

+0
.1

0

+0
.0

8

+0
.0

1

+0
.0

2
0.00

+0.02

+0.04

+0.06

+0.08

+0.10

+0.12

UAR eA

ab
so

lu
te

 im
pr

ov
em

en
t

LSTM BiLSTM LSTM+att

BiLSTM+att Rach et al. (2017)

Figure 5: Absolute improvement of the IQ estimation models over the conventional model origi-
nally proposed by (Ultes et al., 2015) for IQ-based reward estimation with the dialogue-
wise cross-validation setup. UAR and eA take values from 0.0 to 1.0.

4.2 Dialogue Policy Learning

To analyse the impact of the IQ reward estimator on the resulting dialogue policy, experiments
are conducted comparing three different reward models. The baseline is in accordance to Ultes
et al. (2017a): having the objective task success as principal reward component (RTS). It will
be compared to the conventional estimator having the interaction quality estimated by a support
vector machine as principal reward component (Rs

IQ) and to the BiLSTM+att model to estimate the
interaction quality used as principal reward component (Rbi

IQ). TS can be computed by comparing

6. For the bounds, the respective non-existing mc,k is omitted.

93



ULTES AND MAIER

0.00

0.20

0.40

0.60

0.80

1.00

0% 15% 30% 0% 15% 30% 0% 15% 30% 0% 15% 30% 0% 15% 30% 0% 15% 30%

CR CH SR SH L All

Ta
sk

 S
uc

ce
ss

 R
at

e

Reward using IQ (BiLSTM) Reward using IQ (SVM) Reward using TS

Figure 6: Results using GP-SARSA in task success rate (TSR) of the simulated experiments for all
domains and semantic error rates 0%, 15%, and 30%. Each value is computed after 100
evaluation / 1,000 training dialogues averaged over three trials. Numerical results with
significance indicators are shown in Table 6.

0.00

0.20

0.40

0.60

0.80

1.00

0% 15% 30% 0% 15% 30% 0% 15% 30% 0% 15% 30% 0% 15% 30% 0% 15% 30%

CR SR ALL CR SR All

DQN eNAC

Ta
sk

 S
uc

ce
ss

 R
at

e

Reward using IQ (BiLSTM) Reward using IQ (SVM) Reward using TS

Figure 7: Results using DQN and eNAC in task success rate (TSR) of the simulated experiments for
CamRestaurants and SFRestaurants and semantic error rates 0%, 15%, and 30%. Each
value is computed after 100 evaluation / 1,000 training dialogues averaged over three
trials. Numerical results with significance indicators are shown in Table 8 and Table 7.

the outcome of each dialogue with the pre-defined goal. Of course, this is only possible in simulation
and when evaluating with paid subjects. This goal information is not available to the IQ estimators,
nor is it required.

For learning the dialogue behaviour, three policy models are applied. The first is a policy model
based on the GP-SARSA algorithm (Gašić and Young, 2014), which is a value-based method that

94



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

Table 5: Statistics of the domains the interaction quality estimators are trained on (LetsGo) and
applied to (rest).

Domain Code # constraints # DB items state size

LetsGo 4 - -

CamRestaurants CR 3 110 268
CamHotels CH 5 33 111

SFRestaurants SR 6 271 636
SFHotels SH 6 182 438

Laptops L 6 126 204

uses a Gaussian process to approximate the state-value function. As it takes into account the un-
certainty of the approximation, it is very sample efficient and may even be used to learn a policy
directly through real human interaction (Gašić et al., 2013).

Additionally, two deep reinforcement learning algorithms are applied: Deep Q-Network (DQN)
(Mnih et al., 2015) and episodic natural actor critic (eNAC) (Su et al., 2017). Similar to the GP-
SARSA, the DQN also approximates the state-value function also known as Q-function. The eNAC
directly learns the policy using a policy gradient approach in an actor-critic framework.

The decisions of the policy are based on a summary space representation of the dialogue state
tracker. In this work, the focus tracker (Henderson et al., 2014)—an effective rule-based tracker—
is used. The state space and summary state space follow previous work (e.g., Gašić et al., 2013)
and comprise multiple dimensions: for each informable slot one probability distribution over all
slot values plus the special values DONTCARE and NONE, for each requestable slot a Bernoulli
distribution indicating whether the slot has been requested by the user, a probability distribution
over the query method (e.g., search by constraints, search by name), and a status vector of the
search results given the current state. Informable slots contain all information that the user has
provided to the system as search constraints during the interaction. Requestable slots are usually a
superset of the informable slots and contain additionally information that is part of the search result
(e.g., the phone number). To map this dialogue state to a summary space, each of the informable slot
probability distribution vectors are sorted (excluding NONE). This follows the idea that for making
a decision, the actual slot value is not important, only how the probabilities are distributed over all
values.

For each dialogue decision, the policy chooses exactly one summary action out of a set of
summary actions, which are based on general dialogue acts like request, confirm or inform. The
exact number of system actions varies for the domains and ranges from 16 to 25.

To measure the dialogue performance, the task success rate (TSR) and the average interaction
quality (AIQ) are measured: the TSR represents the ratio of dialogues for which the system was
able to provide the correct result. AIQ is calculated based on the estimated IQ values of the respec-
tive model (AIQbi) for the BiLSTM and AIQs for the SVM) at the end of each dialogue. As there
are two IQ estimators, a distinction is made between AIQs and AIQbi. Additionally, the average
dialogue length (ADL) is reported.

95



ULTES AND MAIER

Table 6: Results of the simulated experiments for all domains using GP-SARSA showing task
success rate (TSR), average interaction quality estimated with the SVM (AIQs) and the
BiLSTM (AIQbi), and average dialogue length (ADL) in number of turns. Each value is
computed after 100 evaluation / 1,000 training dialogues averaged over three trials with
different random seeds. 1,2,3 marks statistically significant difference compared to RTS ,
to Rs

IQ, and to Rbi
IQ, respectively (p < 0.05, T-test for TSR and ADL, Mann-Whitney-U

test for AIQ).

Domain SER
TSR AIQs AIQbi ADL

RTS Rs
IQ Rbi

IQ RTS Rs
IQ RTS Rbi

IQ RTS Rs
IQ Rbi

IQ

CR
0% 1.002,3 0.991 0.991 3.642 3.901 3.683 3.831 4.68 4.88 4.59

15% 0.97 0.94 0.96 3.352 3.651 3.453 3.631 5.853 5.33 5.101
30% 0.94 0.92 0.90 3.152 3.341 3.22 3.30 6.34 6.30 6.25

CH
0% 0.98 0.99 0.99 3.262 3.621 3.33 3.44 5.71 5.61 5.40

15% 0.962 0.891,3 0.932 2.90 2.88 3.14 3.14 6.282 7.261,3 6.312

30% 0.86 0.88 0.87 2.382 2.791 2.793 3.021 7.943 7.31 6.991

SR
0% 0.98 0.97 0.98 3.042 3.531 3.133 3.371 6.26 6.03 5.80

15% 0.903 0.88 0.841 2.402 3.001 2.853 3.011 7.99 7.55 7.33
30% 0.71 0.77 0.78 2.032 2.521 2.463 2.781 9.773 9.41 8.501

SH
0% 0.97 0.99 0.98 3.152 3.521 3.173 3.361 5.992 5.501 5.76

15% 0.88 0.88 0.89 2.632 2.941 2.773 3.171 7.983 7.593 6.631,2
30% 0.832 0.761 0.80 2.50 2.63 2.703 2.871 8.38 9.21 8.37

L
0% 0.98 0.99 0.99 3.262 3.611 3.28 3.41 5.78 5.44 5.60

15% 0.89 0.88 0.92 2.582 2.971 2.923 3.171 7.19 7.34 6.73
30% 0.80 0.74 0.77 2.43 2.57 2.79 2.92 8.222 9.321,3 7.972

All
0% 0.98 0.98 0.98 3.232 3.651 3.31 3.48 5.76 5.50 5.47

15% 0.92 0.89 0.91 2.762 3.101 3.023 3.201 7.13 7.06 6.52
30% 0.83 0.81 0.82 2.49 2.80 2.78 2.97 8.202 8.231,3 7.662

For the simulation experiments with the GP-SARSA, the performance of the trained policies
on five different domains was evaluated: Cambridge Hotels and Restaurants, San Francisco Hotels
and Restaurants, and Laptops. The complexity of each domain is shown in Table 5 and compared
to the LetsGo domain (the domain the estimators have been trained on). The DQN and eNAC are
evaluated only on the two domains Cambridge (most simple) and San Francisco Restaurants (most
complex).

The dialogues were created using the publicly available spoken dialogue system toolkit Py-
Dial (Ultes et al., 2017b)7, which contains implementations of all the applied policy models8 and
an implementation of the agenda-based user simulator (Schatzmann and Young, 2009) with an ad-

7. Code freely available at http://www.pydial.org
8. Please refer to PyDial code for details about the exact model implementation.

96



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

Table 7: Results of the simulated experiments for CR and SFR using DQN showing task success
rate (TSR), average interaction quality estimated with the SVM (AIQs) and the BiLSTM
(AIQbi), and average dialogue length (ADL) in number of turns. Each value is computed
after 100 evaluation / 1,000 training dialogues averaged over three trials with different
random seeds. 1,2,3 marks statistically significant difference compared to RTS , to Rs

IQ,
and to Rbi

IQ, respectively (p < 0.05, T-test for TSR and ADL, Mann-Whitney-U test for
AIQ).

Domain SER
TSR AIQs AIQbi ADL

RTS Rs
IQ Rbi

IQ RTS Rs
IQ RTS Rbi

IQ RTS Rs
IQ Rbi

IQ

CR
0% 0.983 0.96 0.951 3.692 3.171 3.52 3.51 4.322 4.681 4.53

15% 0.87 0.87 0.86 3.05 3.11 3.39 3.42 5.37 5.32 5.14
30% 0.76 0.82 0.77 2.652 2.941 3.34 3.28 6.04 5.59 5.74

SR
0% 0.682,3 0.531 0.591 2.572 2.011 3.10 3.07 6.29 6.56 6.56

15% 0.59 0.62 0.57 2.20 2.18 3.10 3.08 6.582 7.101 6.85
30% 0.462 0.381 0.39 1.85 1.71 3.013 3.081 7.552 8.791,3 7.662

All
0% 0.833 0.75 0.771 3.132 2.591 3.31 3.29 5.302 5.621 5.54

15% 0.73 0.75 0.72 2.63 2.65 3.25 3.25 5.98 6.21 5.99
30% 0.61 0.60 0.58 2.252 2.321 3.17 3.18 6.79 7.19 6.70

Table 8: Results of the simulated experiments for CR and SFR using eNAC showing task success
rate (TSR), average interaction quality estimated with the SVM (AIQs) and the BiLSTM
(AIQbi), and average dialogue length (ADL) in number of turns. Each value is computed
after 100 evaluation / 1,000 training dialogues averaged over three trials with different
random seeds. 1,2,3 marks statistically significant difference compared to RTS , to Rs

IQ,
and to Rbi

IQ, respectively (p < 0.05, T-test for TSR and ADL, Mann-Whitney-U test for
AIQ).

Domain SER
TSR AIQs AIQbi ADL

RTS Rs
IQ Rbi

IQ RTS Rs
IQ RTS Rbi

IQ RTS Rs
IQ Rbi

IQ

CR
0% 0.973 0.993 0.741,2 3.312 3.651 3.45 3.41 4.783 4.783 5.821,2

15% 0.912,3 0.741,3 0.341,2 3.25 3.16 3.463 3.181 5.181,2 7.721,3 8.711,2

30% 0.733 0.763 0.391,2 2.78 2.98 3.293 3.141 6.491,2 7.231,3 8.771,2

SR
0% 0.952,3 0.721,3 0.601,2 2.52 2.65 3.213 3.111 5.382,3 7.401,3 8.471,2

15% 0.802,3 0.571,3 0.401,2 2.23 2.35 3.133 3.071 6.672,3 9.391,3 7.901,2

30% 0.632,3 0.511 0.471 1.712 1.941 3.03 3.03 8.602,3 10.011 9.381

All
0% 0.963 0.853 0.671,2 2.922 3.151 3.33 3.26 5.083 6.093 7.151,2

15% 0.862,3 0.651,3 0.371,2 2.74 2.76 3.293 3.131 5.932,3 8.561,3 8.311,2

30% 0.683 0.643 0.431,2 2.25 2.46 3.163 3.091 7.542,3 8.621,3 9.081,2

97



ULTES AND MAIER

ditional error model. The error model simulates the required semantic error rate (SER) caused in a
real system by the noisy speech channel. For each domain, all three reward models are compared
on three SERs: 0%, 15%, and 30%. More specifically, the applied evaluation environments are
based on Env. 1, Env. 3, and Env. 6, respectively, as defined by Casanueva et al. (2017). These
environments also contain all parameters used for the training of GP-SARSA, DQN and eNAC
implementations of PyDial.

For each domain and for each SER, policies have been trained using 1,000 dialogues followed
by an evaluation step of 100 dialogues. The task success rates for GP-SARSA in Figure 6 and DQN
and eNAC in Figure 7 with exact numbers shown in Table 6, Table 7, and Table 8, respectively, were
computed based on the evaluation step averaged over three train/evaluation cycles with different
random seeds.

The results of training the GP-SARSA with the SVM IQ reward estimator are similar in terms
of TSR for Rs

IQ and RTS in all domains for an SER of 0%. This finding is even stronger when
comparing Rbi

IQ and RTS . These high TSRs are achieved while having the dialogues of both IQ-
based models result in higher AIQ values compared to RTS throughout the experiments. Of course,
only the IQ-based model is aware of the IQ concept and indeed is trained to optimise it.

For higher SERs, the TSRs lightly degrade for the IQ-based reward estimators. However, there
seems to be a tendency that the TSR for Rbi

IQ is more robust against noise compared to Rs
IQ while

still resulting in better AIQ values.

Finally, even though the differences are mostly not significant, there is also a tendency for Rbi
IQ

to result in shorter dialogues compared to both Rs
IQ and RTS .

The results of training a DQN or an eNAC policy model are different, though. While the DQN
shows similar TSRs for all SERs and reward models in the CR domain, RTS clearly shows best
performance in the SFR domain. Furthermore, even when using an IQ-based reward model, the re-
spective AIQ does not result in higher scores than usingRTS which shows the stability problems that
come with the usage of deep reinforcement learning. The eNAC policy model is even more prone to
these effects having RTS always resulting in the highest TSR with Rs

IQ and Rbi
IQ performing rather

poorly.

5. Analysis of Learned Behaviour

To further analyse the learning behaviour of the different reward estimators and policy models and
thus to gain deeper insights, the similarity scoring framework (Ultes and Maier, 2020) is applied. It
uses a standardised setup to feed a fixed set of dialogue states into each trained policy model and
compares the resulting system actions and quantifies their similarities. Three different metrics are
computed: the total match rate (TMR), the dialogue act match rate (DMR) and the concept match
rate (CMR).

A similarity score is computed for the comparison of two behavioural models π and π′. De-
pending on the nature of the behavioural model, for each context ci ∈ C, each may produce
an abstract system response actions ai, and an text response pi. Each abstract system action
ai = acti(s

i
1 = vi1, . . . , s

i
j = vij) consists of a dialogue act acti, representing the communica-

tive function like inform or request, and a set Si of j slot-value-pairs Si = {(si1, vi1), . . . , (sij , vij)}

98



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

representing the concepts and their respective values9. To compute each similarity score, |C| ac-
tion/text response pairs are compared using the following similarity score measures.

According to Ultes and Maier (2020), the metrics are defined as follows: the total match rate
(TMR) is based on a binary score that regards two actions a, a′ as equal only if they completely
match, i.e., δa,a′ = 1 iff a = a′, else 0. The TMR is then defined by

TMR =
1

|C|

|C|∑
i=1

δai,a′i . (17)

The dialogue act match rate (DMR) is based on a binary score comparing the actions a, a′

where both match if the corresponding dialogue acts are the same: δact ,act ′ = 1 iff act = act ′, else
0. The DMR is defined by

DMR =
1

|C|

|C|∑
i=1

δacti,act ′i . (18)

The symmetric concept match rate (CMR) counts concepts γ that are present in both dialogue
actions where m̃(a, a′, γ) defines if a match occurred:

m̃(a, a′, γ) =

{
1, if γ ∈ S and γ ∈ S′

0, otherwise .
(19)

The concept match CM takes into account the dialogue acts and the unified set of concepts S̃ =
S1 ∪ S2 of both dialogue actions treating slots s and values v in S̃ as individual γ:

C̃M (a, a′) = δact ,act ′ +
∑
∈S̃

m̃(a, a′, γ) (20)

A concept match of two dialogue actions a and a′ is thus defined by

CM (a, a′) =
C̃M (a, a′)

1 + |S̃|
(21)

and the concept match rate by

CMR =
1

|C|

|C|∑
i=1

CM (a, a′) . (22)

In short, the TMR only evaluates to true in case of a complete match of both actions. The DMR
evaluates to true already if only the dialogue acts of the system actions are equal. The CMR counts
concepts that are present in both dialogue actions and normalises this count by the total number of
distinct concepts in both actions. Using this analysis setup, the following questions are addressed
within this section:

1. How dependent is the resulting policy on the chosen random seed?

9. For the abstract system action a = inform(name=’Golden House’, area=centre), act = inform and S = {(name,’Golden
House’),(area,centre)}.

99



ULTES AND MAIER

2. How similar are learned policies across different reward models?

3. How similar are learned policies across different policy models?

For analysing the learned behaviour and computing similarity scores within and between learn-
ing setups, the CamRestaurants domain is used.

5.1 Consistency of Policies

As each combination of reward model, policy model and semantic error rate has been trained with
three random seeds, this section addresses the question about the consistency of the resulting learned
behaviour. Thus, for each setup, the policy of each random seed has been evaluated with the dialogue
states from 100 evaluation dialogues taken from one of the policies. A different set of states is used
for each noise level. Each random seed’s policy is then compared with the policy of the two other
random seeds resulting in a total of three comparisons. For each policy model and reward model, the
similarity scores of these three comparisons are averaged. The full scores are shown in the appendix
in Table 12.

The similarity scores shown in Table 9 show the average scores for TMR, DMR, and CMR.
In the left table, the scores are averaged over the different reward models to identify which policy
model offers the most consistency during training, i.e., ends up in similar behaviour independent
of the random seed. Overall, eNAC shows the most consistency in all three metrics. For 0% SER,
eNAC seems to be less dependent on the random seed. The GP-SARSA shows average consistency
with the worst CMR for 0% and 15% SER.

On the right side of Table 9, the scores are averaged over the policy models to identify which
reward model offers the most consistency during training. Here, RTS shows overall good perfor-
mance but also Rs

IQ and Rbi
IQ are not fare off and in some cases even more consistent than RTS ,

e.g., TMR with 30% SER.

5.2 Similarity between Reward Models

To gain deeper understanding of how similar the behaviour of the trained policies are that originate
from the different reward models, this section compares policies from different reward models re-
sulting in pair-wise comparisons of the policy of each random seed of one reward model with the
policies of each random seed of another reward model within one policy model. With three random
seeds each, this results in nine comparisons. As in the previous section, each policy is evaluated
with 100 evaluation dialogues taken from one of the policies. For each policy model and reward
model comparison, the similarity scores are averaged. The full scores are shown in the appendix in
Table 13.

The similarity scores shown in Table 10 show the average scores for TMR, DMR, and CMR.
In the left table, the scores are averaged over the different reward model pairs to identify which
policy model results in the most similar behaviour during training, i.e., ends up in similar behaviour
independent of the reward model. Overall, DQN shows the highest similarity in all three metrics.
For 0% SER, GP-SARSA seems to result in more similar behaviour independent for the reward
model.

On the right side of Table 10, the scores are averaged over the policy models to identify which
reward model pair results in the highest similarity in behaviour independent of the chosen policy
model. Overall, RTS – Rbi

IQ shows the highest similarity. Interestingly, the low score of Rs
IQ – Rbi

IQ

100



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

Table 9: The similarity scores analysing the consistency of policies. On the left side, the mean
match rates for different policy models is shown averaged over the respective reward mod-
els. The right side shows the mean match rates for different reward models averaged over
the respective policy models.

SER Type TMR DMR CMR

0%
GP 0.46 0.72 0.44
DQN 0.42 0.73 0.50
eNAC 0.55 0.83 0.64

15%
GP 0.52 0.73 0.48
DQN 0.50 0.76 0.42
eNAC 0.49 0.72 0.62

30%
GP 0.45 0.64 0.52
DQN 0.52 0.70 0.45
eNAC 0.42 0.69 0.57

All
GP 0.48 0.70 0.48
DQN 0.48 0.73 0.46
eNAC 0.49 0.75 0.61

SER Type TMR DMR CMR

0%
RTS 0.51 0.79 0.58
Rs

IQ 0.47 0.75 0.46
Rbi

IQ 0.46 0.73 0.55

15%
RTS 0.49 0.77 0.57
Rs

IQ 0.48 0.71 0.54
Rbi

IQ 0.54 0.73 0.41

30%
RTS 0.43 0.68 0.52
Rs

IQ 0.51 0.69 0.53
Rbi

IQ 0.44 0.66 0.48

All
RTS 0.48 0.74 0.56
Rs

IQ 0.49 0.72 0.51
Rbi

IQ 0.48 0.71 0.48

shows that both reward models result in quite different behaviour even though both models estimate
the same quantity, i.e., the interaction quality.

5.3 Similarity between Policy Models

To analyse the behaviour of the trained policies that originate from the different policy models,
this section compares policies from different policy models resulting in pair-wise comparisons of
the policy of each random seed of one policy model with the policies of each random seed of
another policy model within one reward model. With three random seeds each, this results in nine
comparisons. As in the previous sections, each policy is evaluated with 100 evaluation dialogues
taken from one of the policies. For each policy model and reward model comparison, the similarity
scores are averaged. The full scores are shown in the appendix in Table 14.

The similarity scores shown in Table 11 show the average scores for TMR, DMR, and CMR.
In the left table, the scores are averaged over the different reward models to identify which policy
model pair learns the most similar behaviour, i.e., ends up in similar behaviour independent of the
reward model. Overall, GP – eNAC shows the highest similarity in all three metrics. For 15% and
30% SER, the two deep reinforcement learning models DQN – eNAC show rather low similarity in
behaviour.

On the right side of Table 11, the scores are averaged over the policy model pairs to identify
which reward model results in the highest similarity among the policy models. Overall, Rs

IQ seems
to be the strongest learning signal resulting in the most similar behaviour between policy models.
The similarity resulting from using Rs

TS , instead, is rather low.

101



ULTES AND MAIER

Table 10: The similarity scores analysing the similarity between reward models. On the left side,
the mean match rates for different policy models is shown averaged over the respective
reward model pairs. The right side shows the mean match rates for different reward model
pairs averaged over the respective policy models.

SER Type TMR DMR CMR

0%
GP 0.42 0.62 0.47
DQN 0.38 0.59 0.42
eNAC 0.37 0.53 0.39

15%
GP 0.33 0.61 0.38
DQN 0.40 0.64 0.46
eNAC 0.37 0.58 0.42

30%
GP 0.36 0.52 0.42
DQN 0.41 0.60 0.47
eNAC 0.40 0.56 0.45

All
GP 0.37 0.59 0.42
DQN 0.40 0.61 0.45
eNAC 0.38 0.56 0.42

SER Type TMR DMR CMR

0%
RTS – Rs

IQ 0.34 0.54 0.41
RTS – Rbi

IQ 0.53 0.71 0.59
Rs

IQ – Rbi
IQ 0.29 0.48 0.27

15%
RTS – Rs

IQ 0.37 0.58 0.43
RTS – Rbi

IQ 0.47 0.70 0.53
Rs

IQ – Rbi
IQ 0.27 0.55 0.31

30%
RTS – Rs

IQ 0.34 0.48 0.41
RTS – Rbi

IQ 0.52 0.71 0.59
Rs

IQ – Rbi
IQ 0.30 0.49 0.34

All
RTS – Rs

IQ 0.35 0.53 0.42
RTS – Rbi

IQ 0.51 0.71 0.57
Rs

IQ – Rbi
IQ 0.29 0.51 0.31

Table 11: The similarity scores analysing the similarity between policy models. On the left side,
the mean match rates for different policy model pairs is shown averaged over the re-
spective reward models. The right side shows the mean match rates for different reward
models averaged over the respective policy model pairs.

SER Type TMR DMR CMR

0%
GP – DQN 0.23 0.41 0.27
GP – eNAC 0.35 0.52 0.39
DQN – eNAC 0.33 0.53 0.41

15%
GP – DQN 0.27 0.50 0.35
GP – eNAC 0.30 0.54 0.38
DQN – eNAC 0.25 0.53 0.32

30%
GP – DQN 0.38 0.54 0.45
GP – eNAC 0.33 0.51 0.42
DQN – eNAC 0.34 0.51 0.39

All
GP – DQN 0.29 0.48 0.36
GP – eNAC 0.33 0.52 0.40
DQN – eNAC 0.31 0.52 0.37

SER Type TMR DMR CMR

0%
RTS 0.29 0.48 0.38
Rs

IQ 0.30 0.50 0.37
Rbi

IQ 0.31 0.48 0.33

15%
RTS 0.26 0.52 0.35
Rs

IQ 0.31 0.51 0.38
Rbi

IQ 0.25 0.54 0.32

30%
RTS 0.26 0.43 0.34
Rs

IQ 0.51 0.68 0.62
Rbi

IQ 0.28 0.44 0.31

All
RTS 0.27 0.48 0.36
Rs

IQ 0.38 0.56 0.45
Rbi

IQ 0.28 0.49 0.32

102



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

0

0.2

0.4

0.6

0.8

1

0 100 200 300 400

Moving	Task	Success	Rate

TS IQ US

1

2

3

4

5

0 100 200 300 400

Moving	Average	Interaction	Quality

TS IQ US

1

2

3

4

5

6

0 100 200 300 400

Moving	Average	User	Satisfaction

TS IQ US

Figure 8: Moving TSR (left), moving AIQs (middle) and moving AUS (right) for using either
RTS , Rs

IQ, or RUS as reward averaged over two policies respectively, computed on win-
dows consisting of 120 dialogues.

6. Learning from Real Humans

For learning a policy directly from the interaction with real humans, the IQ-based policy was only
learned using Rs

IQ using GP-SARSA for dialogues in the CR domain10. Subjects were recruited
through Amazon Mechanical Turk to talk to a telephone-based dialogue system. At the end of each
dialogue, users were asked two questions. The first was a yes/no question targeting the dialogue
success (“Have you found all the information you were looking for?”) and has been used as the
baseline for RTS . As this label is noisy, only the dialogues where this subjective success label
matches the objective success were used for policy training (Gašić et al., 2013) (obj = subj).

A second baseline was also included: directly acquiring a user satisfaction (US) rating from the
users after each dialogue. For this, the second question posed was: “How satisfied are you with
the interaction?” The users were able to respond on a six-point scale: 6=very satisfied, 5=satis-
fied, 4=generally ok, 3=unsatisfied, 2=very unsatisfied or 1=extremely unsatisfied. This rating was
converted to a reward in correspondence with Rs

IQ:

RUS = T · (−1) + (US − 1) · 5 . (23)

Hence, each dialogue was also evaluated using the average user satisfaction (AUS).
Two policies were trained for each reward function. The learning curves show moving TSR,

movingAIQs and moving AUS and are presented in Figure 8. Each value in the graphs is calculated
using the respective previous 120 training dialogues averaged over the three trials.

The results show that Rs
IQ may successfully be used for learning through interactions with real

users. It achieves the same TSR as with RTS while resulting in better results for AIQ. The results
for moving AUS also indicate a slightly better user satisfaction compared to RTS .

The second baseline of using RUS also resulted in competitive results showing a similar TSR as
Rs

IQ and RTS whilst resulting in slightly better AUS. This indicates that for a problem as defined
by the CR domain, using user satisfaction directly as a learning signal is also a viable option for
learning policies.

10. These experiments and results have previously been published by Ultes et al. (2017a).

103



ULTES AND MAIER

Sys Welcome to the Cambridge restaurant dialogue system. How may I help you?
Usr i’m looking for european food in the expensive price range
Sys Welcome to the Cambridge restaurant dialogue system. How may I help you?
Usr i want to european restaurant
Sys Would you like something in the cheap, moderate, or expensive price range?
Usr i don’t care the area
Sys Would you like something in the cheap, moderate, [. . . ] ?
Usr european expensive
Sys Would you like something in the cheap, moderate, [. . . ] ?
Usr european food
Sys What part of town do you have in mind?
Usr i don’t care
Sys michaelhouse cafe is a nice place [. . . ] .
Usr thank you goodbye

Figure 9: Example dialogue observed during training using RTS after 387 training dialogues re-
sulting in iq = 1.

Sys Welcome to the Cambridge restaurant dialogue system. How may I help you?
Usr international food
Sys You are looking for a international venue right?
Usr international
Sys Would you like something in the cheap, moderate, or expensive price range?
Usr don’t care
Sys You are looking for a international venue right?
Usr yes
Sys What part of town do you have in mind?
Usr center
Sys bloomsbury restaurant is a nice place [. . . ] .
Usr thank you goodbye

Figure 10: Example dialogue observed during training using Rs
IQ after 365 training dialogues re-

sulting in iq = 4.

Figures 9 and 10 show two successful example dialogues for the models trained with RTS and
Rs

IQ, respectively. One effect of training with Rs
IQ was a reduced number of system repetitions

(which may be linked to the RePrompt? feature).

Obviously, the human experiments have only been conducted using Rs
IQ and not with Rbi

IQ.
However, the general framework of applying an IQ reward estimator for learning a dialogue policy
has shown to be applicable in such a setup and it seems rather unlikely that the changes we induce
by changing the reward estimator lead to a fundamentally different result.

104



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

7. Discussion

One of the major questions of this work addresses the impact of an IQ reward estimator on the
resulting dialogues as different IQ estimators achieves different levels of performance. Analysing
the results of the dialogue policy learning experiment leads to the conclusion that, for GP-SARSA,
the policy learned with Rbi

IQ performs similar or better than Rs
IQ throughout all experiments while

still achieving better average user satisfaction compared toRTS . Especially for noisy environments,
the improvement is relevant.

This finding does not transfer to the other policy models, though. For both, DQN and eNAC,
using RTS results in the overall best peformance for both. This can be explained by the nature of
the IQ reward estimates as they are not as noise-free as RTS . Here, Gaussian processes are known
to be able to deal with this type of noisy targets better while deep learning approaches are known to
be quite sensitive.

The BiLSTM clearly performs better on the LEGO corpus while learning the temporal depen-
dencies instead of using handcrafted ones. However, it entails the risk that these learned temporal
dependencies are too specific to the original data so that the model does not generalise well any-
more. This would mean that it would be less suitable to be applied to dialogue policy learning for
different domains. The experiments clearly show that this is not the case.

One additional aspect for discussion is the definition of RIQ, where the interaction quality
is paired with a dialogue length penalty to guarantee a better comparison with RTS . This length
penalty is not strictly necessary as the annotated scores of the LEGO corpus already contain a notion
of dialogue length implicitly as long dialogues usually tend to have lower quality ratings (in contrast
to other work, e.g., (Foster et al., 2009)). Furthermore, the IQ estimation approach described in the
following uses the dialogue length as an explicit input parameter. However, adding a dialogue
length penalty to RIQ is not regarded as harmfull to the overall approach as dialogue length plays a
subordinate role in the learning process. Moreover, even though this limits the usage of the LEGO
corpus for its applicability to types of interactions, it does not limit the overall approach presented
in this work: for different tasks, new data needs to be collected and annotated.

8. Conclusion

This article has demonstrated that employing a user satisfaction reward estimator for learning di-
alogue policies without any knowledge about the domain can yield good performance in terms of
both task success rate and (estimated) user satisfaction. This has been demonstrated by training
reward estimators on a bus information domain and applying it to learn dialogue policies in five dif-
ferent domains (Cambridge restaurants and hotels, San Francisco restaurants and hotels, Laptops) in
a simulated experiment. The reward estimator using BiLSTMs with attention mechanism achieved
better estimation performance than a SVM-based estimator while learning all temporal dependen-
cies implicitly. Moreover, one of the estimators has successfully been applied to learning dialogue
policies in the domain of finding a restaurant in Cambridge through interaction with real users.

For future work, we aim at extending the user satisfaction estimator by incorporating domain-
independent linguistic data to further improve the estimation performance. To tackle the problem of
degrading performance if the noise level increases, one possible solution would be to have a com-
bination of success and satisfaction as the reward. In addition, active learning will be investigated
to mitigate the requirement for IQ annotated training data. Furthermore, the effects of using a user

105



ULTES AND MAIER

satisfaction-based reward estimator needs to be applied to more complex tasks, e.g., as defined by
the Conversational Entity Dialogue Model (Ultes et al., 2018).

9. Acknowledgements

Part of this research was funded by the EPSRC grant EP/M018946/1 Open Domain Statistical Spo-
ken Dialogue Systems.

References

Yoshua Bengio, Patrice Simard, Paolo Frasconi, et al. Learning long-term dependencies with gra-
dient descent is difficult. IEEE transactions on neural networks, 5(2):157–166, 1994.

Praveen Kumar Bodigutla, Lazaros Polymenakos, and Spyros Matsoukas. Multi-domain conversa-
tion quality evaluation via user satisfaction estimation. arXiv preprint arXiv:1911.08567, 2019a.

Praveen Kumar Bodigutla, Longshaokan Wang, Kate Ridgeway, Joshua Levy, Swanand Joshi, Al-
borz Geramifard, and Spyros Matsoukas. Domain-independent turn-level dialogue quality evalu-
ation via user satisfaction estimation. arXiv preprint arXiv:1908.07064, 2019b.

Praveen Kumar Bodigutla, Aditya Tiwari, Spyros Matsoukas, Josep Valls-Vargas, and Lazaros
Polymenakos. Joint turn and dialogue level user satisfaction estimation on multi-domain con-
versations. In Findings of the Association for Computational Linguistics: EMNLP 2020,
pages 3897–3909, Online, November 2020. Association for Computational Linguistics. doi:
10.18653/v1/2020.findings-emnlp.347. URL https://www.aclweb.org/anthology/
2020.findings-emnlp.347.

Iñigo Casanueva, Paweł Budzianowski, Pei-Hao Su, Nikola Mrkšić, Tsung-Hsien Wen, Stefan
Ultes, Lina Rojas-Barahona, Steve Young, and Milica Gašić. A benchmarking environment
for reinforcement learning based task oriented dialogue management. In Deep Reinforcement
Learning Symposium, 31st Conference on Neural Information Processing Systems (NIPS), 2017.

Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines. ACM
Transactions on Intelligent Systems and Technology, 2:27:1–27:27, 2011. Software available at
http://www.csie.ntu.edu.tw/˜cjlin/libsvm.

Jacob Cohen. A coefficient of agreement for nominal scales. In Educational and Psychological
Measurement, volume 20, pages 37–46, April 1960.

Jacob Cohen. Weighted kappa: Nominal scale agreement provision for scaled disagreement or
partial credit. Psychological bulletin, 70(4):213, 1968.

Heriberto Cuayáhuitl, Seonghan Ryu, Donghyeon Lee, and Jihie Kim. A study on dialogue reward
prediction for open-ended conversational agents. In 2018 NeurIPS Workshop on Conversational
AI: “Today’s Practice and Tomorrow’s Potential”, Montréal, Canada., 2018.

Lucie Daubigney, Matthieu Geist, and Olivier Pietquin. Off-policy Learning in Large-scale
POMDP-based Dialogue Systems. In Proceedings of the 37th IEEE International Conference

106



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

on Acoustics, Speech and Signal Processing (ICASSP 2012), pages 4989–4992, Kyoto (Japan),
2012. IEEE. doi: 10.1109/ICASSP.2012.6289040.

Layla El Asri, Romain Laroche, and Olivier Pietquin. Reward Function Learning for Dialogue
Management. In Proceedings of the 6th Starting AI Researchers’ Symposium (STAIRS), pages
95–106. IOS Press, 2012. doi: 10.3233/978-1-61499-096-3-95.

Layla El Asri, Romain Laroche, and Olivier Pietquin. Reward shaping for statistical optimisation of
dialogue management. In Adrian-Horia Dediu, Carlos Martı́n-Vide, Ruslan Mitkov, and Bianca
Truthe, editors, Statistical Language and Speech Processing, pages 93–101, Berlin, Heidelberg,
2013. Springer Berlin Heidelberg. ISBN 978-3-642-39593-2.

Layla El Asri, Hatim Khouzaimi, Romain Laroche, and Olivier Pietquin. Ordinal regression for
interaction quality prediction. In International Conference on Acoustics, Speech and Signal Pro-
cessing (ICASSP), pages 3245–3249. IEEE, May 2014a. doi: 10.1109/ICASSP.2014.6854195.

Layla El Asri, Romain Laroche, and Olivier Pietquin. Task completion transfer learning for reward
inference. In Workshops at the Twenty-Eighth AAAI Conference on Artificial Intelligence, 2014b.

Maxine Eskenazi, Alan W Black, Antoine Raux, and Brian Langner. Let’s go lab: a platform for
evaluation of spoken dialog systems with real world users. In Ninth Annual Conference of the
International Speech Communication Association, 2008.

Mary Ellen Foster, Manuel Giuliani, and Alois Knoll. Comparing objective and subjective measures
of usability in a human-robot dialogue system. In Proceedings of the Joint Conference of the 47th
Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Pro-
cessing of the AFNLP, pages 879–887, Suntec, Singapore, August 2009. Association for Compu-
tational Linguistics. URL https://www.aclweb.org/anthology/P09-1099.

Milica Gašić and Steve J. Young. Gaussian processes for POMDP-based dialogue manager opti-
mization. IEEEACM Transactions on Audio, Speech, and Language Processing, 22(1):28–40,
2014. doi: 10.1109/TASL.2013.2282190.

Milica Gašić, Catherine Breslin, Matthew Henderson, Dongho Kim, Martin Szummer, Blaise
Thomson, Pirros Tsiakoulis, and Steve J. Young. On-line policy optimisation of Bayesian spo-
ken dialogue systems via human interaction. In Proceedings of the International Conference
on Acoustics, Speech and Signal Processing (ICASSP), pages 8367–8371. IEEE, 2013. doi:
10.1109/ICASSP.2013.6639297.

Milica Gašić, Dongho Kim, Pirros Tsiakoulis, Catherine Breslin, Matthew Henderson, Martin
Szummer, Blaise Thomson, and Steve J. Young. Incremental on-line adaptation of POMDP-based
dialogue managers to extended domains. In Proceedings of the 15th International Conference on
Spoken Language Processing (INTERSPEECH), pages 140–144. ISCA, August 2014.

Alex Graves, Navdeep Jaitly, and Abdel-rahman Mohamed. Hybrid speech recognition with deep
bidirectional lstm. In 2013 IEEE workshop on automatic speech recognition and understanding,
pages 273–278. IEEE, 2013. doi: 10.1109/ASRU.2013.6707742.

James Henderson, Oliver Lemon, and Kallirroi Georgila. Hybrid reinforcement/supervised learning
of dialogue policies from fixed data sets. Computational Linguistics, 34(4):487–511, 2008.

107



ULTES AND MAIER

Matthew Henderson, Blaise Thomson, and Jason D. Williams. The second dialog state tracking
challenge. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse
and Dialogue (SIGDIAL), pages 263–272, Philadelphia, PA, U.S.A., June 2014. Association for
Computational Linguistics. doi: 10.3115/v1/W14-4337. URL https://www.aclweb.org/
anthology/W14-4337.

Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):
1735–1780, 1997. doi: 10.1162/neco.1997.9.8.1735. URL https://doi.org/10.1162/
neco.1997.9.8.1735.

J. R. Landis and G. G. Koch. The measurement of observer agreement for categorical data. Biomet-
rics, 33(1):159–174, March 1977. ISSN 0006-341X.

Oliver Lemon and Olivier Pietquin. Machine learning for spoken dialogue systems. In European
Conference on Speech Communication and Technologies (Interspeech’07), pages 2685–2688,
2007.

Oliver Lemon and Olivier Pietquin. Data-Driven Methods for Adaptive Spoken Dialogue Systems.
Springer New York, 2012. ISBN 978-1-4614-4802-0. doi: 10.1007/978-1-4614-4803-7.

Esther Levin and Roberto Pieraccini. A stochastic model of computer-human interaction for learn-
ing dialogue strategies. In Proc. 5th European Conference on Speech Communication and Tech-
nology (Eurospeech 1997), pages 1883–1886, 1997.

Bing Liu and Ian Lane. Adversarial learning of task-oriented neural dialog models. In 19th Annual
Meeting of the Special Interest Group on Discourse and Dialogue, pages 350–359, 2018.

Teruhisa Misu, Kallirroi Georgila, Anton Leuski, and David Traum. Reinforcement learning of
question-answering dialogue policies for virtual museum guides. In Proceedings of the 13th
Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 84–93, Seoul,
South Korea, July 2012. Association for Computational Linguistics. URL http://aclweb.
org/anthology-new/W/W12/W12-1611.

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G
Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, Stig Pe-
tersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran,
Daan Wierstra, Shane Legg, and Demis Hassabis. Human-level control through deep rein-
forcement learning. Nature, 518(7540):529–533, 2015. doi: 10.1038/nature14236. URL
https://doi.org/10.1038/nature14236.

Niklas Rach, Wolfgang Minker, and Stefan Ultes. Interaction quality estimation using long short-
term memories. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dia-
logue, pages 164–169, Saarbrücken, Germany, August 2017. Association for Computational Lin-
guistics. doi: 10.18653/v1/W17-5520. URL https://www.aclweb.org/anthology/
W17-5520.

Antoine Raux, Dan Bohus, Brian Langner, Alan W. Black, and Maxine Eskenazi. Doing research on
a deployed spoken dialogue system: One year of let’s go! experience. In Proc. of the International
Conference on Speech and Language Processing (ICSLP), September 2006.

108



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

Verena Rieser and Oliver Lemon. Automatic learning and evaluation of user-centered objective
functions for dialogue system optimisation. In Proceedings of the Sixth International Confer-
ence on Language Resources and Evaluation (LREC’08), Marrakech, Morocco, May 2008a. Eu-
ropean Language Resources Association (ELRA). URL http://www.lrec-conf.org/
proceedings/lrec2008/pdf/592_paper.pdf.

Verena Rieser and Oliver Lemon. Learning effective multimodal dialogue strategies from Wizard-
of-Oz data: Bootstrapping and evaluation. In Proceedings of ACL-08: HLT, pages 638–646,
Columbus, Ohio, June 2008b. Association for Computational Linguistics. URL https://
www.aclweb.org/anthology/P08-1073.

Jost Schatzmann and Steve J. Young. The hidden agenda user simulation model. IEEE Transactions
on Audio, Speech, and Language Processing, 17(4):733–747, 2009. doi: 10.1109/TASL.2008.
2012071.

Alexander Schmitt and Stefan Ultes. Interaction quality: Assessing the quality of ongoing spoken
dialog interaction by experts—and how it relates to user satisfaction. Speech Communication,
74:12–36, November 2015. ISSN 0167-6393. doi: 10.1016/j.specom.2015.06.003. URL http:
//www.sciencedirect.com/science/article/pii/S0167639315000679.

Alexander Schmitt, Benjamin Schatz, and Wolfgang Minker. Modeling and predicting quality in
spoken human-computer interaction. In Proceedings of the SIGDIAL 2011 Conference, pages
173–184, Portland, Oregon, USA, June 2011. Association for Computational Linguistics.

Alexander Schmitt, Stefan Ultes, and Wolfgang Minker. A parameterized and annotated spo-
ken dialog corpus of the CMU let’s go bus information system. In Proceedings of the Eighth
International Conference on Language Resources and Evaluation (LREC’12), pages 3369–
3373, Istanbul, Turkey, May 2012a. European Language Resources Association (ELRA). URL
http://www.lrec-conf.org/proceedings/lrec2012/pdf/333_Paper.pdf.

Alexander Schmitt, Stefan Ultes, and Wolfgang Minker. A parameterized and annotated spoken dia-
log corpus of the cmu let’s go bus information system. In International Conference on Language
Resources and Evaluation (LREC), pages 3369–337, May 2012b.

Weiyan Shi and Zhou Yu. Sentiment adaptive end-to-end dialog systems. In Proceedings of the 56th
Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages
1509–1519, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi:
10.18653/v1/P18-1140. URL https://www.aclweb.org/anthology/P18-1140.

Satinder Singh, Diane Litman, Michael Kearns, and Marilyn Walker. Optimizing dialogue man-
agement with reinforcement learning: Experiments with the njfun system. Journal of Artificial
Intelligence Research, 16:105–133, 2002.

Charles Edward Spearman. The proof and measurement of association between two things. Ameri-
can Journal of Psychology, 15:88–103, 1904.

Pei-Hao Su, David Vandyke, Milica Gašić, Dongho Kim, Nikola Mrkšić, Tsung-Hsien Wen, and
Steve J. Young. Learning from real users: Rating dialogue success with neural networks for

109



ULTES AND MAIER

reinforcement learning in spoken dialogue systems. In Interspeech, pages 2007–2011. ISCA,
September 2015.

Pei-Hao Su, Milica Gašić, Nikola Mrkšić, Lina M. Rojas-Barahona, Stefan Ultes, David Vandyke,
Tsung-Hsien Wen, and Steve Young. On-line active reward learning for policy optimisation
in spoken dialogue systems. In Proceedings of the 54th Annual Meeting of the Association for
Computational Linguistics (Volume 1: Long Papers), pages 2431–2441, Berlin, Germany, August
2016. Association for Computational Linguistics. doi: 10.18653/v1/P16-1230. URL https:
//www.aclweb.org/anthology/P16-1230.

Pei-Hao Su, Paweł Budzianowski, Stefan Ultes, Milica Gašić, and Steve Young. Sample-
efficient actor-critic reinforcement learning with supervised data for dialogue management. In
Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 147–
157, Saarbrücken, Germany, August 2017. Association for Computational Linguistics. doi:
10.18653/v1/W17-5518. URL https://www.aclweb.org/anthology/W17-5518.

Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press,
Cambridge, MA, USA, 1st edition, 1998. ISBN 0262193981. URL http://portal.acm.
org/citation.cfm?id=551283.

T. Tieleman and G. Hinton. Lecture 6.5—RmsProp: Divide the gradient by a running average of its
recent magnitude. COURSERA: Neural Networks for Machine Learning, 2012.

Stefan Ultes. Improving interaction quality estimation with BiLSTMs and the impact on dialogue
policy learning. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dia-
logue, pages 11–20, Stockholm, Sweden, September 2019. Association for Computational Lin-
guistics. doi: 10.18653/v1/W19-5902. URL https://www.aclweb.org/anthology/
W19-5902.

Stefan Ultes and Wolfgang Maier. Similarity scoring for dialogue behaviour comparison. In Pro-
ceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue,
pages 311–322, 1st virtual meeting, July 2020. Association for Computational Linguistics. URL
https://www.aclweb.org/anthology/2020.sigdial-1.38.

Stefan Ultes and Wolfgang Minker. Improving interaction quality recognition using error cor-
rection. In Proceedings of the SIGDIAL 2013 Conference, pages 122–126, Metz, France, Au-
gust 2013. Association for Computational Linguistics. URL https://www.aclweb.org/
anthology/W13-4018.

Stefan Ultes and Wolfgang Minker. Interaction quality estimation in spoken dialogue systems us-
ing hybrid-HMMs. In Proceedings of the 15th Annual Meeting of the Special Interest Group
on Discourse and Dialogue (SIGDIAL), pages 208–217, Philadelphia, PA, U.S.A., June 2014.
Association for Computational Linguistics. doi: 10.3115/v1/W14-4328. URL https://www.
aclweb.org/anthology/W14-4328.

Stefan Ultes, Tobias Heinroth, Alexander Schmitt, and Wolfgang Minker. A theoretical framework
for a user-centered spoken dialog manager. In Ramón López-Cózar and Tetsunori Kobayashi,
editors, Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue

110



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

Systems Workshop, pages 241–246, New York, NY, September 2011. Springer New York. ISBN
978-1-4614-1334-9. doi: 10.1007/978-1-4614-1335-6 24.

Stefan Ultes, Alexander Schmitt, and Wolfgang Minker. Towards quality-adaptive spoken dialogue
management. In NAACL-HLT Workshop on Future directions and needs in the Spoken Dialog
Community: Tools and Data (SDCTD 2012), pages 49–52, Montréal, Canada, June 2012. As-
sociation for Computational Linguistics. URL https://www.aclweb.org/anthology/
W12-1819.

Stefan Ultes, Alexander Schmitt, and Wolfgang Minker. On quality ratings for spoken dialogue
systems – experts vs. users. In Proceedings of the 2013 Conference of the North American
Chapter of the Association for Computational Linguistics: Human Language Technologies, pages
569–578, Atlanta, Georgia, June 2013. Association for Computational Linguistics. URL https:
//www.aclweb.org/anthology/N13-1064.

Stefan Ultes, Matthias Kraus, Alexander Schmitt, and Wolfgang Minker. Quality-adaptive spo-
ken dialogue initiative selection and implications on reward modelling. In Proceedings of the
16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 374–
383, Prague, Czech Republic, September 2015. Association for Computational Linguistics. doi:
10.18653/v1/W15-4649. URL https://www.aclweb.org/anthology/W15-4649.

Stefan Ultes, Paweł Budzianowski, Iñigo Casanueva, Nikola Mrkšić, Lina Rojas-Barahona, Pei-Hao
Su, Tsung-Hsien Wen, Milica Gašić, and Steve Young. Domain-independent user satisfaction
reward estimation for dialogue policy learning. In Proc. Interspeech 2017, pages 1721–1725.
ISCA, August 2017a. doi: 10.21437/Interspeech.2017-1032. URL http://dx.doi.org/
10.21437/Interspeech.2017-1032.

Stefan Ultes, Lina M. Rojas-Barahona, Pei-Hao Su, David Vandyke, Dongho Kim, Iñigo Casanueva,
Paweł Budzianowski, Nikola Mrkšić, Tsung-Hsien Wen, Milica Gašić, and Steve Young. Py-
Dial: A multi-domain statistical dialogue system toolkit. In Proceedings of ACL 2017, System
Demonstrations, pages 73–78, Vancouver, Canada, July 2017b. Association for Computational
Linguistics. URL https://www.aclweb.org/anthology/P17-4013.

Stefan Ultes, Paweł Budzianowski, Iñigo Casanueva, Lina M. Rojas-Barahona, Bo-Hsiang Tseng,
Yen-Chen Wu, Steve Young, and Milica Gašić. Addressing objects and their relations: The con-
versational entity dialogue model. In Proceedings of the 19th Annual SIGdial Meeting on Dis-
course and Dialogue, pages 273–283, Melbourne, Australia, July 2018. Association for Com-
putational Linguistics. doi: 10.18653/v1/W18-5032. URL https://www.aclweb.org/
anthology/W18-5032.

Stefan Ultes, Juliana Miehle, and Wolfgang Minker. On the Applicability of a User Satisfaction-
Based Reward for Dialogue Policy Learning, pages 211–217. Springer International Publishing,
Cham, 2019. ISBN 978-3-319-92108-2. doi: 10.1007/978-3-319-92108-2 22. URL https:
//doi.org/10.1007/978-3-319-92108-2_22.

David Vandyke, Pei-Hao Su, Milica Gašić, Nikola Mrkšić, Tsung-Hsien Wen, and Steve Young.
Multi-domain dialogue success classifiers for policy training. In 2015 IEEE Workshop on Au-
tomatic Speech Recognition and Understanding (ASRU), pages 763–770. IEEE, 2015. doi:
10.1109/ASRU.2015.7404865.

111



ULTES AND MAIER

Vladimir N. Vapnik. The nature of statistical learning theory. Springer-Verlag New York, Inc., New
York, NY, USA, 1995. ISBN 0-387-94559-8. doi: 10.1007/978-1-4757-3264-1.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez,
Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V.
Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Ad-
vances in Neural Information Processing Systems, volume 30, pages 5998–6008. Curran Asso-
ciates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/
3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.

Marilyn Walker. An application of reinforcement learning to dialogue strategy selection in a spoken
dialogue system for email. Journal of Artificial Intelligence Research, 12:387–416, 2000. doi:
https://doi.org/10.1613/jair.713.

Marilyn A. Walker, Diane J. Litman, Candace A. Kamm, and Alicia Abella. PARADISE: A frame-
work for evaluating spoken dialogue agents. In 35th Annual Meeting of the Association for
Computational Linguistics and 8th Conference of the European Chapter of the Association for
Computational Linguistics, pages 271–280, Madrid, Spain, July 1997. Association for Com-
putational Linguistics. doi: 10.3115/976909.979652. URL https://www.aclweb.org/
anthology/P97-1035.

Marilyn A. Walker, Jeanne C. Fromer, and Shrikanth Narayanan. Learning optimal dialogue
strategies: A case study of a spoken dialogue agent for email. In COLING 1998 Volume
2: The 17th International Conference on Computational Linguistics, 1998. URL https:
//www.aclweb.org/anthology/C98-2214.

Frank Wilcoxon. Individual comparisons by ranking methods. Biometrics bulletin, 1(6):80–83,
1945.

Jason D. Williams and Steve J. Young. Characterizing task-oriented dialog using a simulated asr
chanel. In Proceedings of the 8th International Conference on Spoken Language Processing
(Interspeech 2004), pages 185–188, 2004.

Steve J. Young, Milica Gašić, Blaise Thomson, and Jason D. Williams. POMDP-based statistical
spoken dialog systems: A review. Proceedings of the IEEE, 101(5):1160–1179, 2013. doi:
10.1109/JPROC.2012.2225812.

Guineng Zheng, Subhabrata Mukherjee, Xin Luna Dong, and Feifei Li. Opentag: Open attribute
value extraction from product profiles. In Proceedings of the 24th ACM SIGKDD International
Conference on Knowledge Discovery & Data Mining, pages 1049–1058. ACM, 2018.

112



USER SATISFACTION REWARD ESTIMATION ACROSS DOMAINS

Appendix A. Tables

Table 12: Full results showing the consistency of learned policies in terms of total match rate
(TMR), dialogue act match rate (DMR) and concept match rate (CMR) for three different
semantic error rates (SERs). Similarity scores are computed by comparing the learned
behaviour of the three policies originating from different random seeds for each policy
model and each reward model.

SER Policy
TMR DMR CMR

RTS Rs
IQ Rbi

IQ RTS Rs
IQ Rbi

IQ RTS Rs
IQ Rbi

IQ

0%
GP 0.48 0.44 0.47 0.72 0.70 0.74 0.55 0.26 0.51
DQN 0.42 0.46 0.39 0.76 0.71 0.70 0.51 0.53 0.48
eNAC 0.61 0.52 0.52 0.90 0.85 0.76 0.67 0.59 0.67

15%
GP 0.51 0.39 0.66 0.74 0.66 0.78 0.58 0.40 0.45
DQN 0.40 0.52 0.57 0.74 0.75 0.78 0.50 0.57 0.18
eNAC 0.56 0.52 0.39 0.82 0.72 0.63 0.63 0.64 0.60

30%
GP 0.36 0.52 0.46 0.62 0.61 0.70 0.45 0.47 0.63
DQN 0.51 0.54 0.50 0.69 0.73 0.66 0.59 0.49 0.27
eNAC 0.44 0.46 0.37 0.71 0.74 0.61 0.53 0.64 0.55

113



ULTES AND MAIER

Table 13: Full results showing the comparison of learned policies using different reward models
in terms of total match rate (TMR), dialogue act match rate (DMR) and concept match
rate (CMR) for three different semantic error rates (SERs). Similarity scores are com-
puted by pair-wise comparing the learned behaviour of the three policies of each reward
model originating from different random seeds with each trained policy of the other re-
ward model within each policy model.

SER Type
TMR DMR CMR

GP DQN eNAC GP DQN eNAC GP DQN eNAC

0%
RTS – Rs

IQ 0.31 0.33 0.39 0.57 0.55 0.53 0.40 0.40 0.44
RTS – Rbi

IQ 0.58 0.58 0.45 0.72 0.75 0.65 0.64 0.64 0.50
Rs

IQ – Rbi
IQ 0.38 0.45 0.27 0.55 0.65 0.42 0.36 0.50 0.23

15%
RTS – Rs

IQ 0.24 0.41 0.45 0.48 0.64 0.62 0.31 0.48 0.51
RTS – Rbi

IQ 0.49 0.53 0.37 0.78 0.74 0.59 0.57 0.60 0.41
Rs

IQ – Rbi
IQ 0.26 0.37 0.27 0.57 0.59 0.53 0.27 0.41 0.34

30%
RTS – Rs

IQ 0.22 0.36 0.45 0.30 0.55 0.57 0.29 0.43 0.49
RTS – Rbi

IQ 0.48 0.57 0.51 0.68 0.76 0.70 0.55 0.63 0.59
Rs

IQ – Rbi
IQ 0.37 0.51 0.24 0.58 0.70 0.41 0.40 0.59 0.26

Table 14: Full results showing the comparison of learned policies using different policy models
in terms of total match rate (TMR), dialogue act match rate (DMR) and concept match
rate (CMR) for three different semantic error rates (SERs). Similarity scores are com-
puted by pair-wise comparing the learned behaviour of the three policies of each policy
model originating from different random seeds with each trained policy of the other pol-
icy model within each reward model.

SER Type
TMR DMR CMR

RTS Rs
IQ Rbi

IQ RTS Rs
IQ Rbi

IQ RTS Rs
IQ Rbi

IQ

0%
GP – DQN 0.25 0.20 0.22 0.46 0.40 0.37 0.35 0.27 0.20
GP – eNAC 0.35 0.37 0.33 0.50 0.56 0.49 0.43 0.43 0.32
DQN – eNAC 0.28 0.33 0.38 0.48 0.53 0.59 0.36 0.40 0.46

15%
GP – DQN 0.24 0.24 0.33 0.45 0.45 0.61 0.34 0.31 0.39
GP – eNAC 0.30 0.35 0.25 0.57 0.53 0.52 0.39 0.43 0.32
DQN – eNAC 0.23 0.36 0.18 0.56 0.55 0.47 0.33 0.40 0.23

30%
GP – DQN 0.29 0.55 0.31 0.41 0.72 0.49 0.36 0.65 0.35
GP – eNAC 0.23 0.50 0.26 0.44 0.67 0.41 0.31 0.62 0.33
DQN – eNAC 0.25 0.49 0.29 0.45 0.66 0.42 0.35 0.58 0.24

114