CHEMICAL ENGINEERING TRANSACTIONS VOL. 76, 2019 A publication of The Italian Association of Chemical Engineering Online at www.aidic.it/cet Guest Editors: Petar S. Varbanov, Timothy G. Walmsley, Jiří J. Klemeš, Panos Seferlis Copyright © 2019, AIDIC Servizi S.r.l. ISBN 978-88-95608-73-0; ISSN 2283-9216 Advanced Control of a Biochemical Reactor for Yeast Fermentation Monika Bakošová*, Juraj Oravec, Anna Vasičkaninová, Alajos Mészáros, Petra Artzová Slovak University of Technology in Bratislava, Faculty of Chemical and Food Technology, Institute of Information Engineering, Automation and Mathematics, Radlinského 9, 812 37 Bratislava, Slovak Republic monika.bakosova@stuba.sk Control of alcoholic fermentation is intensively studied in last decades as it is used in biofuel production. Two advanced control approaches for the yeast alcoholic fermentation running in a continuous-time biochemical reactor are studied in this paper with focus on maximizing product yield and minimizing energy consumption. The neural network predictive control uses a neural network (NN) process model in the optimizing model-based control strategy. A new approach to control of a biochemical reactor for yeast fermentation presented in the paper is a robust model-based predictive control with integral action (RMPC-IA). The RMPC-IA uses a discrete time state-space model for prediction of future outputs of the process with parametric uncertainty. The calculated control inputs are the results of an optimization strategy. The optimization problem to be solved is formulated as a convex optimization problem resolved via linear matrix inequalities. The RMPC-IA outperformed the NN predictive control of alcoholic fermentation as it improved performance indices, preserved the product yield, and ensured energy saving. 1. Introduction Renewable energy sector is rapidly expanding as the attention focusses on cleaner, reliable and sustainable resources. Effective and energy-efficient processing of the resources requires advanced control strategies. Between them, model predictive control (MPC) is popular because of its flexibility and ability to be used in industrial applications. MPC represents the state-of-art optimal strategy for complex systems. The optimal control action is computed with respect to the process variables constraints (Mayne, 2014). A systematic review on MPC in renewable energy applications was done by Sultana et al. (2017). One of main limitations of MPC is not considering of system uncertainties in the prediction model. The robust MPC (RMPC) has been developed to optimize control actions with respect to the bounded uncertain process parameters and bounded process variables. The milestone in RMPC research represented the work of Kothare et al. (1996), where the possibility to design RMPC using the convex optimization was proposed. Boyd et al. (2004) formulated the non-convex constraints in the form of the semidefinite programming and solved them via linear matrix inequalities (LMIs). RMPC is used in various applications leading to energy savings. Zhang et al. (2018) used a two-stage RMPC- based optimization for optimal energy management of island microgrids with uncertainty. Oravec et al. (2018) designed RMPC with integral action for the shell-and-tube heat exchangers with fouling and showed significant improvement of control performance and energy savings in comparison with conventional PID control. The growing energy consumption presses on the biofuels sector to develop technologies for an energy-efficient future (Darda et al., 2019). Replacement of petroleum by other energy sources is one of the most important engineering tasks. Promising substitute for petroleum is biomass that can be chemically converted into fuel. Nicodème et al. (2018) gave review of the different technologies currently available for biofuels production via a thermochemical pathway. Alcoholic yeast fermentation is one of the most important biochemical processes and importance of this fermentation has even increased recently because ethanol is an alternative source of energy. From the control point of view, the alcoholic fermentation is a process with significantly nonlinear behavior and the conventional control algorithms are not able to control the process efficiently. Moreover, the DOI: 10.3303/CET1976129 Paper Received: 16/03/2019; Revised: 31/05/2019; Accepted: 17/06/2019 Please cite this article as: Bakosova M., Oravec J., Vasickaninova A., Meszaros A., Artzova P., 2019, Advanced Control of a Biochemical Reactor for Yeast Fermentation, Chemical Engineering Transactions, 76, 769-774 DOI:10.3303/CET1976129 769 process is affected by various uncertainties and disturbances. Between various approaches, the neural network (NN) modelling is very popular for modelling of nonlinear, time-varying processes and processes having asymmetric dynamics and uncertainties. The details of the neural networks can be found e.g. in Kriesel (2005). NN models can be used in various control structures and one of them is MPC. Integrated advanced NN predictive control (NNPC) for production of bio-methanol from sugar cane via pyrolysis was investigated by Kasmuri et al. (2019). Nagy (2007) presented the use of feedforward neural networks for dynamic modeling and temperature control of a continuous yeast fermentation bioreactor. The resulting ANNs were introduced in a MPC scheme and better results were obtained than using linear model predictive control (LMPC) and PID control. Ławryńczuk (2008) developed ANN model of a yeast fermentation and used it in a computationally efficient nonlinear MPC algorithm with nonlinear prediction and linearization (MPC-NPL). The control accuracy and the disturbance rejection ability were also demonstrated. Kumar et al. (2019) designed internal model control based proportional-integral-derivative (IMC-PID) controller for the alcoholic fermentation. The controller was successfully tested to control bioreactor temperature. The novelty of this paper is design of the robust model predictive control with integral action (RMPC-IA) for yeast fermentation. This approach for yeast fermentation control has not been published yet. The main objective of this paper is to show that RMPC-IA is able to ensure higher product yield and energy saving in comparison with the NNPC. The comparison is based on the simulation results. 2. Biochemical reactor for yeast fermentation Yeast fermentation is one of the most important biochemical processes. The considered reactor for yeast fermentation is taken from Nagy (2007). The mathematical model describes the kinetics of yeast fermentation and mass and energy balances of the reactor. The kinetic equations are modifications of the Monod equations based on the Michaelis - Menten kinetics. The reactor is modelled as a continuous-time stirred tank reactor with constant substratum feed flow. The reactor outlet flow contains the product, the substrate and the biomass. The biomass is a yeast suspension fed into the reactor and continuously evacuated. The substrate is the glucose solution feeding the micro-organism saccharomyces cerevisiae and it is continuously fed into the reactor. The product is ethanol, which is evacuated along with other components. Inorganic salts are added with yeast because of the formation of coenzymes. Because a low dilution rate is required, the dynamic properties of the process are very slow. The model of the biochemical reactor has 6 state variables: Vr – the volume of the reacting mixture, cx, cp, cs – the biomass, product and substratum concentrations, respectively, cO2 – the oxygen concentration in the liquid phase, Tr – the temperature in the reactor, Tag – the temperature of the cooling agent in the jacket. The continuous-time fundamental model of the reactor contains 7 nonlinear ordinary differential equations and algebraic kinetics equations. The manipulated variable is the flow rate of the cooling agent in the jacket, Fag, and the controlled output is the temperature in the reactor, Tr. The steady-state values of these variables for inputs given in Ławryńczuk (2008) are Fags = 18 L/h and Trs = 29.6 °C. This model was used as a controlled system in simulation experiments. 3. Neural network-based model predictive control The neural network-based (NN) predictive controller uses a NN model of the controlled process. The advantage is that the process can be nonlinear, with asymmetric behavior or affected by various uncertainties and disturbances. The NN model is used to predict future behavior of the process. The NN predictive controller calculates the predicted control input that optimizes the process performance over a specified time horizon (Hagan et al., 2002). The predictions are used in a numerical optimization procedure to calculate the control signal minimizing the cost function (2): 𝐼(𝑘) = ∑ (𝑟(𝑘 + 𝑗) − 𝑦𝑚(𝑘 + 𝑗)) 2 𝑁2 𝑗=𝑁1 +  ∑(𝑢(𝑘 + 𝑗 − 1))2 𝑁𝑢 𝑗=1 (1) In Eq(1), N1, N2, are the prediction horizons, Nu is the control horizon, k is the time in the discrete time domain. The parameter  represents contribution of the sum of the squares of the control increments u(k+j+i) to the cost function I, r is the reference signal, ym is the NN model response, and u is the sequence of the future control increments that are calculated in the optimization procedure. The neural network process model is a very important component in the NNPC structure and the choice of the number of layers and neurons in the individual layers is a challenging task. The prediction error between the process output and the neural network output is used as the NN training signal. The NN process model uses previous process inputs and previous process outputs to predict future values of the process outputs. The first step in NNPC design is the training of a neural network to represent the feedforward dynamics of the process. 770 The network can be trained off line in the batch mode using data collected from the process operation. The Levenberg-Marquardt (LM) algorithm is efficient for training (Amini and Rostami, 2016) and is often used. 4. Robust model-based predictive control with integral action Advanced optimization-based RMPC uses a polytopic linear process model in the MPC structure (Liu et al., 2016). The model can have the form of a polytopic linear discrete-time state-space system in Eq(2) 𝑥(𝑘 + 1) = 𝐴𝑣𝑥(𝑘) + 𝐵𝑣𝑢(𝑘), 𝑦(𝑘) = 𝐶𝑣𝑥(𝑘), 𝑥(0) = 𝑥0 (2) where k is the discrete time, x(k) is the vector of system states, u(k) is the vector of control inputs, and y(k) is the vector of system outputs. The system matrices Av, Bv, Cv have appropriate dimensions. The considered parametric polytopic uncertainty of the controlled system Eq(2) has the form 𝔸 = convhull([𝐴𝑣, 𝐵𝑣, 𝐶𝑣], ∀𝑣 = 1, … , 𝑛) (3) where 𝔸 is the convex hull of the system vertices v. The nominal system is represented by the set of matrices A0, B0, C0 that can be obtained for the mean values of the system interval parameters. To assure offset-free control response, the RMPC with integral action (RMPC-IA) was designed (Oravec et al., 2018). Therefore, it was necessary to define an augmented vector of the system states �̂�(𝑘) = [ 𝑥(𝑘) ∑ 𝑒(𝑖) 𝑘 𝑖=0 ] (4) where 𝑒(𝑘) = 𝑤(𝑘) − 𝑦(𝑘) is the control error calculated as the difference between the reference value 𝑤 and the system output 𝑦. Using the augmented states Eq(4), the system Eq(2) was transformed to the discrete-time state space model �̂�(𝑘 + 1) = �̂�𝑣𝑥(𝑘) + �̂�𝑣𝑢(𝑘), 𝑦(𝑘) = �̂�𝑣�̂�(𝑘), �̂�(0) = �̂�0 (5) where the augmented matrices for the sampling time ts are �̂�𝑣 = [ 𝐴𝑣 0 −𝑡s𝐶𝑣 𝐼 ] , �̂�𝑣 = [ 𝐵𝑣 0 ] , �̂�𝑣 = [𝐶𝑣 0] (6) The linear state feedback control law in the RMPC-IA has the form 𝑢(𝑘) = 𝐹(𝑘)�̂�(𝑘) (7) The control input Eq(7) has to ensure robust stability of the closed-loop and to minimize the quadratic cost function 𝐽 = ∑ (𝑥(𝑘)T𝑄P𝑥(𝑘) + (∑ 𝑒(𝑖)) 𝑘 𝑖=0 T 𝑄I(∑ 𝑒(𝑖)) 𝑘 𝑖=0 + 𝑢(𝑘) T𝑅𝑢(𝑘))𝑁𝑘=0 , (8) where 𝑄P is the state weighting matrix in the proportional part, 𝑄I is the error weighting matrix in the integration part, and 𝑅 is the control input weighting matrix. The control inputs and the controlled outputs must satisfy the symmetric constraints that follow from requirements in practice −𝑢max ≼ 𝑢(𝑘) ≼ 𝑢max, −𝑦max ≼ 𝑦(𝑘) ≼ 𝑦max. (9) The task of finding the state feedback controller F(k) in Eq(7) can be formulated using LMIs as the semidefinite programming (SDP) problem of convex optimization having the form min(𝛾) (10) [ 1 ∗ 𝑥(𝑘) 𝑋 ] ≽ 0, [ 𝑋 ∗ �̂�0𝑋 + �̂�0𝐻 𝑋 ] ≽ 0, [ 𝑋 ∗ �̂�𝑣𝑋 + �̂�𝑣𝐻 𝑋 ∗ ∗ ∗ ∗ �̂� 1 2⁄ 𝑋 0 𝑅 1 2⁄ 𝑌 0 𝛾𝐼 ∗ 0 𝛾𝐼] ≽ 0, (11) [ 𝑋 ∗ 𝑍 𝑈max ] ≽ 0, [ 𝑋 ∗ 𝐶[�̂�𝑣𝑋 + �̂�𝑉𝐻] 𝑌max ] ≽ 0, (12) where 𝑋 is the symmetric positive definite inverse Lyapunov matrix, 𝑌, 𝐻 are the auxiliary controller tuning matrices, 𝑈max, 𝑌max are the matrices representing the input and output constraints, respectively. Matrices 771 �̂� = [ 𝑄P 0 0 𝑄I ] , 𝑅 are the weighting matrices from the cost function Eq(8). In SDP Eq(10)–Eq(12), the symbol * denotes the symmetric structure of LMIs. The gain matrix of the controller in Eq(7) is calculated from solutions of SDP as 𝐹 = 𝑌𝑋−1. (13) 5. Results and discussion 5.1 NNPC of yeast fermentation The first step in the NNPC design was NN model identification. The neural network used for the model of the yeast fermentation had the input layer with 6 neurons, the output layer with 2 neurons and one hidden layer with 8 neurons. The sigmoid transfer functions were used in the hidden layer and the linear transfer functions were used in the output layer. The Levenberg-Marquardt backpropagation algorithm was used for training. The NN model of the yeast fermentation (YF) was trained off-line using data obtained from the nonlinear model. The uncertainty was introduced in the NN model by obtaining the training data for a large operation range of the biochemical reactor. 500 training samples were used for training, validation, and testing. The parameters in the cost function Eq(1) used for the NNPC design were: the prediction and the control horizons N1 = 1, N2 = 18, Nu = 3, the weighting parameter λ = 0.001. The controlled output was the temperature in the reactor, Tr. The manipulated variable was the cooling agent flow rate, Fag. The controlled process was the nonlinear mathematical model of the reactor. The constraints on the control inputs were chosen: the minimum control input Fag = 6 L/h, the maximum control input Fag = 36 L/h. The constraints on the controlled outputs were chosen: the minimum reactor temperature Tr = 25 °C, the maximum reactor temperature Tr = 36 °C. 5.2 RMCP-IA of yeast fermentation For RMPC-IA design, it was necessary to consider the normalized state-space model, i.e., the model with the steady-state values shifted to the origin. The model of the biochemical reactor in the form of a polytopic linear state-space system was obtained using data of several step-responses. As the process behavior is significantly non-linear and asymmetric, several step responses were measured. The step changes of Fag were within interval [6, 30] L/h. The reactor was identified using a step-response-based method (Mikleš and Fikar, 2007) in the form of the first-order transfer function 𝐺𝑆 = 𝐾 𝑇𝑠 + 1 e−𝐷𝑠 (14) where GS is the process transfer function, K is the gain, T is the time constant, and D is the time delay. Because of the strongly nonlinear reactor behavior, the transfer function parameters were identified as interval parameters. The limit and mean values of the interval parameters are in Table 1. The mean values represent the nominal system, i.e., an idealized system without uncertainties. Table 1: Identified parameters of the process transfer function Value K [°C L-1 h] T [h] D [h] Minimum – 0.7235 9.890 0 Maximum Nominal – 0.1570 – 0.4403 45.580 27.735 1.18 0.59 Table 2: Vertex systems of the polytopic state-space system Vertex system Av Bv Cv 1 0.8169 – 0.1325 1 2 0.9571 – 0.0067 1 3 0.8169 – 0.0287 1 4 0.9571 – 0.0311 1 As RMPC-IA design required model in the form Eq(2) - Eq(3), the identified transfer functions were transformed to the polytopic state-space system considering the sampling time ts = 2 h. The parameters of 4 vertex systems are presented in Table 2. The weighting matrices in Eq(8) were set QP = 20, QI = 20, R = 1. 772 5.3 Simulation results Simulation results were obtained using MATLAB/Simulink R2018b. The Neural Network Toolbox was used for NNPC design. MUP toolbox (Oravec, 2014) was used for RMPC-IA design. NNPC and RMPC-IA of the reactor for yeast fermentation were compared in reference tracking problem without and with measurement noise. The reference temperature Tref,1 = 32.0 °C changed to Tref,2 = 33.0 °C at 200 h and then to Tref,3 = 32.0 °C at 400 h. The simulation results in reference tracking without measurement noise are presented in Figure 1a, Figure 1b and with measurement noise in Figure 2a, Figure 2b. Figure 1a: Controlled reactor temperature without measurement noise in NNPC (green line) and RMPC-IA (red line) Figure 1b: Cooling agent flow rate generated in NNPC (green line) and RMPC-IA (red line) without presence of measurement noise Figure 2a: Controlled reactor temperature with measurement noise in NNPC (green line) and RMPC-IA (red line) Figure 2b: Cooling agent flow rate generated in NNPC (green line) and RMPC-IA (red line) in the presence of measurement noise Table 3: Values of P, Vag, settling time, steady-state error, IAE, and ISE Control Measurement noise P [kg] Vag [L] Settling time [h] SSE [°C] IAE [°C h] ISE [°C2 h] NNPC no 424.285 6,354.3 82 1.7 272.2 249.8 RMPC-IA no 424.845 6,281.2 62 0 141.7 147.1 NNPC yes 424.034 6,375.9 82 1.7 262.5 249.5 RMPC-IA yes 424.102 6,347.8 62 ≈0 141.2 147.2 In all figures, the black solid line represents the reference value, the black dashed line represents the allowed error, the green line represents the variable in NNPC and the red line represents the variable in RMPC-IA. Table 3 summarizes the obtained numerical results and performance indices. The simulation results were compared according to the total production of ethanol P, the total cooling agent consumption Vag consumed during control, the average settling time, the average value of the steady-state error SSE, the integral quality criteria ISE (integrated squared error) and IAE (integrated absolute error) (Ogunnaike and Ray, 1994). The RMPC-IA ensured offset-free reference tracking without overshoots in all situations. The NNPC led to small offset for Tref,2, but the offset was smaller than the allowed error when no measurement noise was present. The average settling time of the control responses was shorter in RMPC-IA. NNPC led to worse values of IAE and ISE. The total production of ethanol P was higher when RMPC was used and RMPC-IA assured also lower 773 consumption of cooling agent then NNPC. According to the data presented in Table 3, RMPC-IA outperformed NNPC. 6. Conclusions NNPC and RMPC-IA were used for control of the reactor for yeast fermentation. RMPC-IA improved control performance in comparison to the NNPC, as it removed the steady-state error and shortened the average settling time. The settling time was reduced in 32 % in average. The value of IAE decreased by 86 % in the presence of measurement noise and by 92% without measurement noise, and the value of ISE decreased by 69.5 % in the presence of measurement noise and by 69.8 % without measurement noise. Using RMPC-IA, the control responses were offset-free and without overshoots. RMPC-IA ensured also higher ethanol production and lower coolant consumption, but the improvement was small. Both strategies were able to assure reference tracking with required accuracy ± 0.2 ºC. According to the simulation results, RMPC-IA represents a promising strategy that can lead to maximizing of production and minimizing of energy consumption. The future research will be focused on incorporation of stochastic signals into the controller design and on implementation and validation of RMPC-IA on a laboratory fermenter. Acknowledgments The authors gratefully acknowledge the contribution of the Scientific Grant Agency of the Slovak Republic under the grant 1/0112/16. References Amini K., Rostami F., 2016, Three-steps modified Levenberg–Marquardt method with a new line search for systems of nonlinear equations, Journal of Computational and Applied Mathematics, 300, 30–42. Boyd S., Vandenberghe L., 2004, Convex Optimization, Cambridge University Press, New York, USA. Darda S., Papalas T., Zabaniotou A., 2019, Biofuels journey in Europe: Currently the way to low carbon economy sustainability is still a challenge, Journal of Cleaner Production, 208, 575–588. Hagan M.T., Demuth H.B., De Jesús O., 2002, An introduction to the use of neural networks in control systems, Int. J. Robust Nonlinear Control, 12, 959–985. Kasmuri, N.H., Kamarudin S.K., Abdullah S.R.S., Hasan H.A., Md Som A., 2019, Integrated advanced nonlinear neural network-simulink control system for production of bio-methanol from sugar cane bagasse via pyrolysis, Energy, 168, 261–272. Kothare M. V., Balakrishnan V., Morari M., 1996, Robust Constrained Model Predictive Control Using Linear Matrix Inequalities, Automatica, 32, 1361–1379. Kriesel D., 2016, A Brief Introduction to Neural Networks accessed 30.04.2019. Kumar M., Prasad D., Giri B.S., Singh R.S., 2019, Temperature control of fermentation bioreactor for ethanol production using IMC-PID controller, Biotechnology Reports, 22, e00319. Ławryńczuk M., 2008, Modelling and nonlinear predictive control of a yeast fermentation biochemical reactor using neural networks, Chemical Engineering Journal, 145, 290 – 307. Liu X., Feng S., Ma M., 2016, RMPC for the constrained system with polytopic uncertainty, International Journal of Systems Science, 43, 248-258. Mayne D.Q., 2014, Model predictive control: Recent developments and future promise, Automatica, 50, 2967 – 2986. Nagy Z. K., 2007, Model based control of a yeast fermentation bioreactor using optimally designed artificial neural networks, Chemical Engineering Journal, 127, 95 – 109. Nicodème T., Berchem T., Jacquet N., Richel A., 2018, Thermochemical conversion of sugar industry by- products to biofuels, Renewable and Sustainable Energy Reviews, 88, 151–159. Ogunnaike B.A., Ray W.H., 1994, Process Dynamics, Modelling, and Control, Oxford University Press, New York, USA, 1260 ps. ISBN: 978-0195091199. Oravec, J., MUP, < bitbucket.org/oravec/mup/wiki/Home > accessed 15.03.2019. Oravec J., Bakošová M., Trafczynski M., Vasičkaninová A., Mészáros A., Markowski, M., 2018, Robust model predictive control and PID control of shell-and-tube heat exchangers, Energy, 159, 1–10. Sultana W.R., Sahoo S.K., Sukchai S., Yamuna S., Venkatesh D., 2017, A review on state of art development of model predictive control for renewable energy applications, Renewable and Sustainable Energy Reviews, 76, 391–406. Zhang Y., Fu L., Zhu W., Bao X., Liu C., 2018, Robust model predictive control for optimal energy management of island microgrids with uncertainties, Energy, 164, 1229–1241. 774