CHEMICAL ENGINEERING TRANSACTIONS VOL. 79, 2020 A publication of The Italian Association of Chemical Engineering Online at www.cetjournal.it Guest Editors: Enrico Bardone, Antonio Marzocchella, Marco Bravi Copyright © 2020, AIDIC Servizi S.r.l. ISBN 978-88-95608-77-8; ISSN 2283-9216 Adaptive-Network-Based Fuzzy Inference (ANFIS) Modelling of Particle Image Velocimetry (PIV) Measurements in Stirred Tank Reactors Carlos E.Gómez-Camacho*, Leonardo Clemente, Giulia Moretti, Bernardo Ruggeri* Politecnico di Torino, Dep. of Applied Science and Technology (DISAT), C/so Duca degli Abruzzi 24, 10129 Torino, Italy carlos.gomezcamacho@polito.it and bernardo.ruggeri@polito.it Mixing represents an energy-intensive unit operation which can significantly influence the performance of the different industrial processes. Efficient mixing is necessary to reduce spatial inhomogeneities within reactors and bioreactors. Particularly, in bioreactors, shear stress on microorganisms and physicochemical gradients might affect the physiological state of the biotic phase and, hence, decrease bioreaction yields. The present work presents an innovative machine-learning modelling approach which uses the adaptive- network-based fuzzy inference system (ANFIS) on experimentally velocity fields data collected through Particle Image Velocimetry (PIV) in STR under different operating conditions. The calibration and optimization of the ANFIS system are performed by dividing the PIV data into two subsets: training and validation data. Then, a sensitivity analysis is carried out varying the percentage of training data and certain features of the membership functions (number and type). The fitness of the produced models was scored by means of the fuzzy Goodness Index (GI), which combines the correlation coefficient (R2), index of agreement (IA) and relative root mean square error (RRMSE). 1. Introduction Different industrial fields require distinct levels of modelling, due to the nature of the systems under analysis. Modelling is useful to understand the underlying physics of the systems and to identify the major governing phenomena, as well as to make assumptions and predictions following a realistic description of its behaviour. The Fuzzy Logic (FL) was introduced (Zadeh, 1965) as an alternative approach to model complex system based on the theory of possibility, rather than on large numbers approach. It addresses uncertainty based on expert knowledge, contrary to traditional models which rely on deterministic and stochastic models. FL use antecedents and consequents (independent and dependent variables in deterministic approach) described by using membership functions with defined form and shape to represent the degree of possession of an attribute estimated by numerical or verbal values in the continuous intervals [0,1], along with minimum/maximum operations for conjunction/disjunction (i.e. FL operations to aggregate/exclude antecedents and consequents). The quantity and quality of available information ultimately determine the robustness of mathematical models to describe and predict phenomena. Hence, the causes of uncertainty can be attributed first to the inaccurate definition of the modelling purpose and then to: lack of information, an excess of information (i.e. complexity), inconsistent evidence, ambiguity, the meaningfulness of experimental measurements, among others. Biological systems, for example, are intrinsically complex, which means that description of the whole system requires more than the sum of its individual parts or components and that these systems are typically characterized by large uncertainties. The study of highly non-linear phenomena requires effective complexity reduction approaches, as in the case of hydrodynamics. Typical reduction approaches in this field resort to compartmentalization and the estimation of certain characteristics of flow, such as the Reynolds number, mixing time, energy dissipation rate or gas hold-up among other key parameters. However, the biotechnological field currently faces different challenges associated to scale-up and scale-down operations which require a more DOI: 10.3303/CET2079001 Paper Received: 14 September 2019; Revised: 20 December 2019; Accepted: 3 March 2020 Please cite this article as: Gomez Camacho C.E., Clemente L., Moretti G., Ruggeri B., 2020, Adaptive-network-based Fuzzy Inference (anfis) Modelling of Particle Image Velocimetry (piv) Measurements in Stirred Tank Reactors, Chemical Engineering Transactions, 79, 1-6 DOI:10.3303/CET2079001 1 careful analysis of hydrodynamic properties, which are essential for improved designed and control strategies, particularly concerning the interactions of hydrodynamics with the biotic phase. Nowadays, the main computational techniques available for the study of velocity flow patterns on feasible spatial-temporal resolution are Computational Fluid Dynamics (CFD), using different approaches. These can be used as tools to recognize the existence of critical conditions as dead zones, imbalances in the pressure fields, self-similarities, among others. However, the use of these models has a strong sensitivity to the parameters and assumptions required to numerically solve a set of differential equations, which cannot be solved analytically in different cases. Notwithstanding, the development of experimental optic techniques has allowed getting more insights into hydrodynamic properties. These techniques employ tracer particles and imaging systems, which are post-processed to allow quantitative flow measurements in 2D or 3D, such as Particle Image Velocimetry (PIV) or Positron Emission Particle Tracking (PEPT). The uncertainties of the experimental techniques reverberate on the velocity magnitude in critical zones, particularly for viscous fluids and multiphase systems as semi-solid, liquid and gas (e.g. bioreactors). Coupling experimental measurements to soft-modelling techniques, such as fuzzy logic and neural networks, presents a great potential to process and model complex systems using expert knowledge or inference systems based on large experimental data sets. The present work is a study that seeks to model the velocity field data obtained through PIV in a stirred tank reactor, under different operating conditions using the adaptive-network-based fuzzy inference system (ANFIS). The optimization of the ANFIS model was performed by dividing the PIV velocity field output into two subsets: training and validation. A sensitivity analysis was conducted, varying the parameters considered for the ANFIS and comparing the modelled data with the validation fraction of PIV data. The fitness of the ANFIS models was scored using different indexes, such as the correlation coefficient (R2), index of agreement (IA) and the relative root mean square error (RRMSE). Lastly, a fuzzy-based index was calculated (i.e. Goodness Index, GI), which combines the previous three, in order to couple the information provided by the deterministic indicators and assess the overall goodness-of-fit in each case. 2. Particle Image Velocimetry (PIV) 2.1 Background on Particle Image Velocimetry (PIV) Particle Image Velocimetry (PIV) is a non-invasive optical technique that is used for the study and determination of the flow field of fluids in different vessels geometry or different impellers. In brief, flow seeding particles are suspended in a continuous phase to be examined by optical imaging analysis of pictures taken out of the reactor. A collimated pulsed light sheet is employed to illuminate the seeding particles, determining the flow cross- section within the tank vessel that is recorded with a digital camera. Since tracing particles must be able to reflect the incident light, flow seeding particles should be carefully chosen based on the characteristics of the continuous phase. These seeding particles are expected to faithfully follow the direction of the flow within the system, the concentration should not affect the continuous phase properties (e.g. density, viscosity). Operatively, seeding particles are illuminated twice by the pulsing light sheet, and the fluorescent print is captured by one or more digital camera(s), taking two consecutive pictures (Figure 1A). Then, the displacement of the particles is resolved in the time domain (Figure 1B) using the timespan between the two exposures (i.e. exposure time delay). The total area of the frames undergoes a structured grid partitioning, generating different interrogation windows (IW). The IWs of the first picture are then correlated to the corresponding windows of the following frame, which allows to find an average displacement within each IW and a distinct peak (Figure 1C). The resulting velocity field includes the all the vectors calculated for each IW and can be represented also by means of the cumulative distribution function (CDF). A) B) C) D) E) F) Figure 1. Key elements and working principle of Particle Image Velocimetry: (A) setup with optic system, (B) Grid partitioning and cross-correlation to estimate velocity vectors, (C) cumulative distribution function (CDF) of velocity vectors, (D) experimental set-up showing the vessel placed in the aquarium, (E) Imaging hardware (Imager Pro SX) and (F) Light sheet - modified from (Scarano, 2013) (Moretti, 2018). 2 2.2 Experimental set-up of PIV measurements The PIV measurements were performed placing the experimental setup in a dark laboratory room with black walls to prevent the reflection of the lasers and to enhance the quality of the measurements. The experimental setup (Figure 1D) consisted in: a stirred vessel placed within an aquarium, a stirrer gearbox, the impeller shaft, two impellers with different geometries, baffles, water as filling fluid, seeding particles, a high-resolution digital camera (Figure 1E), the laser pulser and the laser sheet (Figure 1F) and the corresponding software. The spherical-bottom vessel (Klöpper form) has an internal diameter of 160 mm and the working volume corresponds to a filling volume of 2.99 L of water, hence the filling height corresponds to 160 mm. The vessel is placed inside a water-filled aquarium (with a water level of about 26-29 cm), which holds the vessel and the stirrer gearbox, and is placed in front of the laser source and the camera. The baffles are four equally spaced flat plates supported by a ring that fits on the internal vessel wall and have a width JB = 12 mm, immersed length HB = 114 mm, thickness TB = 2 mm and distance of the baffle from the vessel wall of AB = 5 mm. The stirrer (IKA EUROSTAR 60, Staufen, Germany) has a control knob for the start/stop of the stirring and for the selection of the rotational speed, in the range 20 - 2000 rpm. Two different impellers geometry are tested in the present work: i) a Ring Propeller and ii) a Bionic-Loop impeller (BiLoop Impeller) (Table 1), inserted in the vessel with an off-bottom clearance hC = 0.33 D. The camera and the laser source are synchronized and connected to the software (DaVis 8.4 LaVision GmbH, Göttingen, Germany), following the calibration step. The light source for the experiments is a neodymium-doped yttrium aluminium garnet (Nd:YAG) laser (LaVision GmbH, Göttingen, Germany), the crystal solid material is yttrium aluminum garner (YAG) placed on a Y3Al5O12 matrix with neodymium ions Nd3+, of approx. 1 mm laser sheet thickness. The wavelength of Nd:YAG laser is 532 nm. A high-resolution digital camera was employed (Imager Pro SX LaVision GmbH, Göttingen, Germany), able to capture double-frame pictures (i.e. exposure time delay c. 1μs) at a frequency of 15 Hz. The flow field was extracted using the cross-correlation algorithm integrated in the software, with decreasing passes for the calculations (three times with passes of 48×48 pixels and subsequently three times with 24×24 pixels). Lastly, the continuous phase for the experiments is ultrapure water (Purelab Elga, Veolia, Italy), at a temperature of 21.6 °C, total organic carbon below 3 ppb and electric conductivity of c. 0.055 μS; experimental density of (997.66 kg/m3) and viscosity of (955.4 μPa/s). Table 1. Characteristics of the impellers employed in the present work 3. Adaptive-Network-Based Fuzzy Inference (ANFIS) modelling 3.1 Fuzzy Logic (FL) and Adaptive Neuro–Fuzzy Inference System (ANFIS) As introduced above, FL is a modelling approach which is particularly useful to formalize expert knowledge. In the basic form, it is applied using a structured inference system connecting antecedents and consequents constituted by a set of IF, OR, THEN logical operators (rules). The structure of this inference system is constituted by a set of rules, membership functions (MFs) and an inferring motor using the min-max criterion. The inputs are first transposed into the fuzzy dominium by projecting crisp situations into fuzzy-sets with a label (verbal variable) and a grade of possession of it in the interval [0,1], using a defined shape of the membership function. The output of the inference system in the fuzzy dominion undergoes then a defuzzification process to produce a quantifiable deterministic output value. Different shapes of MFs are available, such as triangular, trapezoidal, gaussian and generalised bell. The choice of numbers and shape of MFs are defined for each antecedent and consequent for the case under analysis. Hence, the user defines a desired threshold of uncertainty and consequently adapt the features of the inference system (i.e. increasing or decreasing the number of MFs or rules of the inference system or varying the shape of the MFs). However, building fuzzy inference systems for complex systems exclusively from expert knowledge can result in large numbers of rules, which cannot be handled in a simple way. An enhanced application of FL is represented by its combination with artificial neuronal networks (ANN), known as neuro-fuzzy or ANFIS. The approach consists in knowledge acquisition by building fuzzy inference systems, based on large sets of data (i.e. data obtained from high- throughput measurements such as PIV), operated with ANN algorithms. In this configuration, a Sugeno-type of inference is adopted, where inputs are fuzzified through MFs, while outputs are obtained through best-fitted zero- or first-order polynomial functions (ANN). ANFIS uses a two-passes hybrid learning approach that combines Back Propagation (BP) to learn antecedent parameters and least square estimator (LSE) to tune the coefficients of the outputs. Certain premise parameters, such as Gaussian curve coefficients (i.e. μ and σ) can Impeller Picture and flow pattern Impeller Picture and flow pattern 1. Ring Propeller 2. BiLoop Impeller 3 be classified as nonlinear, while the consequents or outputs can be considered linear functions. The hybrid training algorithm is obtained by alternatively modifying premise and consequent set of parameters, while maintaining fixed the other one. This is done in a two passes recursive algorithm: a forward pass, where premise parameters are kept fixed and consequent parameters are modified through LSE. In the backward pass, BP is applied following the gradient descent method, changing nonlinear premises while maintaining unvaried consequent parameters. Operatively, a raw first model is constructed specifying the quantity and shapes of MFs; secondly, a training process is performed on such initial model, adapting ANFIS parameters on a given share of PIV data. For the FL and ANFIS modelling, the Fuzzy Logic and ANFIS MATLAB® 2019 Toolboxes were used (Clemente, 2019). 3.2 Correlation Indexes: a combined approach to address uncertainty The fitness of correlation among the experimental PIV data and the ANFIS-model outputs was performed using three deterministic indicators: i) the coefficient of determination (R2), ii) the index of agreement (IA) and iii) the normalized root mean square error (RRMSE), as well as a fuzzy-based lumped indicator, the Goodness Index (GI) which combines in the fuzzy-domain the first three indicators (Table 2) (Di Addario, 2017). Table 2. Deterministic indicators (R2, IA and RRMSE) and the fuzzy indicator, Goodness Index (GI) Deterministic Indicators Fuzzy Goodness Index – GI i) 𝑅2 = (∑ (𝑂𝑖 − 𝑂𝑚)(𝑃𝑖 − 𝑃𝑚) 𝑛 𝑖=1 ) 2 ∑ (𝑂𝑖 − 𝑂𝑚) 2𝑛 𝑖=1 ∑ (𝑃𝑖 − 𝑃𝑚) 2𝑛 𝑖=1 Rule IF R2 IA RRMSE T H E N GI 1 Low Low High Very Low 2 Medium Medium Medium Medium 3 High High Low Very High 4 Low Low Low Medium 5 Low Low Medium Low 6 High Low High Medium 7 Medium Low High Low 8 Low High High Medium 9 Low Medium High Low 10 Low High Low Medium 11 Medium High Low High 12 High Low Low Medium 13 High Medium Low High 14 High High High Medium 15 High High Medium High ii) 𝐼𝐴 = 1 − ∑ (𝑂𝑖 − 𝑃𝑖) 2𝑛 𝑖=1 ∑ (|𝑂𝑖 − 𝑂𝑚| + |𝑃𝑖 − 𝑃𝑚|) 2𝑛 𝑖=1 iii) 𝑅𝑅𝑀𝑆𝐸 = ( 1 𝑛 ∑ (𝑂𝑖 − 𝑃𝑖) 2𝑛 𝑖=1 ) 0.5 𝑂𝑚 These deterministic indicators are calculated through the difference between observed (O) and predicted (P) values, using the mean value (m), considering up to the i-observation. R2 and IA indexes are dimensionless measures, ranging from 0 (poor predictive model) to 1 (optimal predictive model). RRMSE, although the base index (RMSE) is not dimensionless, here the normalized form was considered, by expressing it relatively to the mean observed value (Om). In this context, a good model implies R2 and IA close to 1 and RRMSE close to 0. The fuzzification of the deterministic indicators was done using three labels [Low, Medium, High] as the antecedents, while for the consequent (GI) five different labels were chosen [Very Low, Low, Medium, High, Very High] combined following the rules presented in Table 2. 4. Results and Discussion 4.1 Selecting best-performing Membership Functions (MF) The first modelling step regards the STR and the ring-propeller (Table 1). First, three antecedents are taken into consideration: the relative radial (r/R) and axial (h/H) coordinates and the different experimental values of power input tested (i.e. 20, 50, 100, 200 and 500 W/m3) to obtain the velocity field (output), normalized by the tip velocity in each case. These three antecedents are divided into various number of MFs; in the first formulation of the model, different types of fuzzy MFs were tested, while the output (PIV generated velocity field) datasets were split in two fractions: training and validation set (c. 3·103 points/each, in a ratio 50%:50%). For example, a simple one is the triangular MF, which is obtained by specifying three premises, the three modifiable vertices (e.g. a, b and c). Similar, a trapezoidal MF can be defined using four premises (a, b, c and d). Figure 2. Goodness index (GI) of ANFIS models using differently shaped membership functions. 4 Another type of MFs is based on distribution functions, gaussian or generalized bell curves, which are defined by two or three parameters: μ (mean value), σ (standard deviation) and, for the latter case, a shape parameter (c). Further classes include sigmoidal functions (closed and asymmetric) and Pi-functions, characterized by specific premises, all these MFs have been tested. The results are presented in terms of the GI (Figure 2), the different shape of MFs can help to tune uncertainty in the model. Gaussian and generalised bell MFs resulted as best-performing compared to the other tested shapes. Triangular MFs exhibited low fitness, probably due to the strong non-linear behaviour of the system under analysis. Furthermore, no marked differences were obtained in the GI for validation and training datasets, using equal fractions of the raw data for each. 4.2 Variation of the fraction of training data A sensitivity analysis was conducted to ascertain how the predictive power of the model varies due to the percentage of data used to train the model. Gaussian MFs were selected since it resulted as the best performing shape and are computationally lighter (only two parameters). Theoretically, ANFIS systems enhance its predictability power by increasing the amount of training data. However, large training batches could lead to very rigid models, thus reducing predictivity. In the present case, the percentage of training data on the total available set was varied from 5% to 50%, and the validation set corresponds to the complement values which are not used in the training step. Figure 3 shows the agreement behaviours vs. different deterministic indexes as well as the fuzzy GI index. Figure 3. Predictive power of ANFIS models: R2, IA, RRMSE and GI using different data training percentages. It is clear that a significant improvement of all correlation indexes is obtained in the passage from 5 to 10% of training data. This fact is confirmed by the GI, which reaches an asymptotic value of c. 0.95/1.00 using a training percentage of 10 %. The sensitivity of the GI can be tuned by the modification of the defined rules of the fuzzy index, when more a sensitive parameter is required to capture small variations of its antecedents. For the present case, a 10-20 % on the total PIV batch (i.e. one/two data points out of 10) subjected to ANFIS training can yield a satisfactory predictive model. The reasons for this fact are the adaptivity of ANFIS and a suitable batch of data representing the velocity field at a suitable grid. 4.3 Keeping track of computational burden in ANFIS Selecting an appropriate combination of MFs and antecedents is an essential part of ANFIS modelling. Considering exclusively the spatial coordinates as antecedents, at constant power input (P/V), as dimensional (r, h) and dimensionless (r/R, h/H) values result after training in 7 or 6 MFs, respectively. The inference system for each case consisted in 49 rules and 36 rules. The additional MFs for the case of spatial coordinates in their original dimension (mm) is an indication of the information regarding the total magnitude (in the case of dimensionless units is not necessary). A sensitivity analysis on the number of antecedents was conducted for the complete model with three antecedents (r/R, h/H, P/V). Three key features of the systems were studied: the number of rules, the time employed for each epoch and the fitness of the model (GI). This step is critical to reduce the computational cost by rationalizing the number of MFs of the antecedents, speeding up convergence during training and maintaining the predictive power (i.e. GI). Figure 4. Sensitivity Analysis of the number of MFs: a) number of rules and time/epoch and b) fitness (GI). 5 MFs were fixed at 5 for the P/V antecedent, while r/R and h/H MFs were scanned in the range 3 to 7 and using a 10% of the data in the training step. Figure 4A shows how the generated rules of the inference system tend to increase almost linearly as the number of antecedents MFs increase, while the time for epoch generation presents an exponential growth (Figure 4A). The predictive power of the model (Figure 4B), GI, presents a considerable improvement when the number of MFs is increased from 3 to 6. Hence, 6 MFs for the spatial antecedents provide an ANFIS model of PIV velocity fields, without representing a heavy computational burden. 4.4 Extending the approach for different impellers The optimization procedure reported in section 4.1 and 4.2 regards the ring propeller (Table 1); to include the BiLoop impeller, an ANFIS model was developed using the corresponding PIV data. BiLoop is characterized by a different distribution of power input in stirred tanks. For comparison purposes, in Figure 5, it is possible to see the experimental flow patterns (PIV) and the ANFIS modelling predictions for both cases. The ring propeller presents a more axially driven flow pattern and central distribution, while the BiLoop has also a radial component which distributes the velocity field upwards and downwards, albeit with less intensity than the ring propeller. Consequently, higher peaks of dimensionless velocity are visible for the ring-propeller, while the BiLoop dissipates the power more efficiently. Figure 5. Comparison between experimental PIV (upper) and modelled ANFIS (down) flow fields for each impeller at different P/V inputs. 5. Conclusions The present work shows the application of FL and ANN, and the combined ANFIS method as machine learning algorithms which can potentially improve hydrodynamics modelling. FL is used to incorporate expert knowledge to develop the GI, which proved useful to score the predictive power of ANFIS models by combining the statistic indicators (R2, IA and RRMSE), while ANFIS was used to model experimental PIV data. These models accurately follow the experimental velocity fields after a step-by-step optimization process, which included the selection of best-performing MFs for the antecedents (i.e. number and shape) and the choice of an adequate share of data for the training step. For both impeller geometries in the STR, a share of 10-20 % of the total data were sufficient for the ANFIS training, using 6 gaussian MFs for the radial and axial antecedents, while the power input data was included in the model using the 5 crisp experimentally tested conditions. The ANFIS model could be further enriched by combining additional high-throughput data and expert knowledge, which can be of interest in the bioreactors field to develop models with enhanced predictivity over other key flow properties of interest to assess hydrodynamic effects on the biotic phase (e.g. shear stress, local O2 transfer rate). Lastly, the use of ANFIS models can be a useful tool to perform scale-up and scale-down operations, including complex hydrodynamic information which is difficult using traditional modelling approaches. References Clemente L., 2019, ANFIS modelling of PIV tests, MSc. Thesis, Politecnico di Torino, Turin, Italy. Di Addario M., 2017, Bioreactor landfills: experimental simulations, full-scale monitoring and fuzzy modelling, PhD Thesis, Politecnico di Torino, Turin, Italy. Moretti G., 2018, Influence of Impeller Type and Geometry on Particle Stress in a Stirred Fermenter by Means of Particle Image Velocimetry, MSc. Thesis, Politecnico di Torino/Technische Universität Berlin, Turin, Italy. Scarano F., 2013, Tomographic PIV: Principles and practice, Measurement Science and Tech., 24(1), 1-28. Zadeh L.A., 1965, Fuzzy sets, Information and Control, 8(3), 338–353. 6