www.biomathforum.org/biomath/index.php/biomath ORIGINAL ARTICLE A new class of activation functions. Some related problems and applications Nikolay Kyurkchiev1,2 1Faculty of Mathematics and Informatics University of Plovdiv Paisii Hilendarski 24 Tzar Asen Str., 4000 Plovdiv, Bulgaria nkyurk@uni-plovdiv.bg 2Institute of Mathematics and Informatics Bulgarian Academy of Sciences Acad. G. Bonchev Str., Bl. 8, 1113 Sofia, Bulgaria Received: 21 January 2020, accepted: 3 May 2020, published: 17 May 2020 Abstract—The cumulative distribution function (cdf) of the discrete two–parameter bathtub hazard distribution has important role in the fields of population dynamics, reliability analysis and life testing experiments. Also of interest to the specialists is the task of approximating the Heaviside function by new (cdf) in Hausdorff sense. We define new activation function and family of new recurrence generated functions and study the ”saturation” by these families. In this paper we analyze some in- trinsic properties of the new Topp–Leone–G–Family with baseline ”deterministic–type” (cdf) – (NTLG– DT). Some numerical examples with real data from Biostatistics, Population dynamics and Signal the- ory, illustrating our results are given. It is shown that the study of the two characteristics - ”confiden- tial curves” and ”super saturation” is a must when choosing the right model. Some related problems are discussed, as an example to the Approximation Theory. Keywords-two–parameter bathtub hazard dis- tribution; ”saturation” by: new activation func- tion and family of new recurrence generated functions; Topp–Leone–G–Family with baseline ”deterministic–type” (cdf) – (NTLG–DT); Heavi- side function; Hausdorff distance; upper and lower bounds I. INTRODUCTION AND PRELIMINARIES Definition 1. Define the following deterministic (cdf) based on two–parameter bathtub hazard dis- tribution [2]: Mβ(t) = 1 −qe tβ−1, (1) where 0 < q < 1; β > 0, t > 0. Definition 2. The shifted Heaviside step function Copyright: c© 2020 Kyurkchiev. This article is distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Citation: Nikolay Kyurkchiev, A new class of activation functions. Some related problems and applications, Biomath 9 (2020), 2005033, http://dx.doi.org/10.11145/j.biomath.2020.05.033 Page 1 of 10 http://www.biomathforum.org/biomath/index.php/biomath https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/ http://dx.doi.org/10.11145/j.biomath.2020.05.033 Nikolay Kyurkchiev, A new class of activation functions. Some related problems and applications is defined by ht0 (t) =   0, if t < t0, [0, 1], if t = t0, 1, if t > t0 (2) Definition 3. [3] The Hausdorff distance (the H– distance) ρ(f,g) between two interval functions f,g on Ω ⊆ R, is the distance between their completed graphs F(f) and F(g) considered as closed subsets of Ω ×R. More precisely, ρ(f,g) = max{ sup A∈F(f) inf B∈F(g) ||A−B||, sup B∈F(g) inf A∈F(f) ||A−B||}, wherein ||.|| is any norm in R2, e. g. the maximum norm ||(t,x)|| = max{|t|, |x|}; hence the distance between the points A = (tA,xA), B = (tB,xB) in R2 is ||A−B|| = max(|tA − tB|, |xA −xB|). Definition 4. We define the following activation function: A(t; β) = qe −tβ −qe tβ qe −tβ + qe tβ . (3) Definition 5. Define the following family of new recurrence generated functions Ai+1(t; β) = Ai(t + Ai(t; β); β), i = 0, 1, 2, . . . ; A0(t; β) = A(t; β). (4) based on the function A(t; β). In [1] Bantan, Jamal, Chesneau and Elgarhy introduced a new power Topp–Leone–G–Family (NTL–G) of distribution with (cdf) F(t) = e αβ ( 1− 1 G(t) ) ( 2−eβ ( 1− 1 G(t) ))α (5) where α,β ∈ R+ and G(t) is a (cdf) of a baseline continuous distribution. The following result shows some inequalities involving F(t) (see, Proposition 1 [1]): e αβ ( 1− 1 G(t) )( 2 −G(t)β )α ≤F(t)≤2αeαβ ( 1− 1 G(t) ) . (6) In this paper we study some properties of the new Topp–Leone–G–Family with baseline ”deterministic–type” (cdf) – (NTLG–DT); G(t) = 1 −qe t−1, where 0 < q < 1. Definition 6. We define the following correspond- ing (cdf): Q(t) = e αβ ( 1− 1 1−qet−1 ) ( 2 −eβ ( 1− 1 1−qet−1 ))α (7) where α,β ∈ R+ and 0 < q < 1. II. MAIN RESULTS When studying the intrinsic properties of the family Mβ(t), it is also appropriate to study the ”saturation” to the horizontal asymptote. In this Section we give upper and lower esti- mates for the one–sided Hausdorff approximation of the Heaviside step–function ht0 (t) by means of family (1), where t0 is the level of the ”median”. A. The case β = 1. Let t0 is the unique positive root of the nonlinear equation M1(t0) − 12 = 0. The one–sided Hausdorff distance d between ht0 (t) and the function (1) satisfies the relation M1(t0 + d) = 1 −qe (t0+d)−1 = 1 −d. (8) The following theorem gives upper and lower bounds for d Theorem 1. Let β = 1, q < 2 e 2( e1.052.1 −1) ≈ 0.971975. (9) Then, for the one–sided Hausdorff distance d between ht0 (t) and the (cdf) – (1) the following inequalities hold: dl = 1 2.1(1+ 1 2 ln 2 q ) 0 we conclude that function f(d) is strictly monotonically increasing. Consider then the function g(d) = − 1 2 + (1 + 1 2 ln 2 q )d, which approximates function f with d → 0 as O(d2) (see, Fig. 1). In addition g′(d) > 0. We look for two reals dl and dr such that g(dl) < 0 and g(dr) > 0 (leading to g(dl) < d < g(dr)). From (9) we have g ( dl = 1 2.1(1 + 1 2 ln 2 q ) ) < 0, g ( dr = ln(2.1(1 + 1 2 ln 2 q )) 2.1(1 + 1 2 ln 2 q ) ) > 0 proving the estimates (10). For example, for β = 1, q = 0.1 we have dl = 0.190639 < d = 0.230226 < 0.31596 = dr and for β = 1, q = 0.9 we have dl = 0.340317 < d = 0.355551 < 0.36682 = dr. B. The case β 6= 1. For given β 6= 1 the one–sided Hausdorff distance d satisfies the relation Mβ(t0 + d) = 1 −qe (t0+d) β−1 = 1 −d. (11) The reader may formulate the corresponding approximation problem following the ideas given in Theorem 1, and will be omitted. We illustrate the ”saturation” with the (cdf) – (1) for various β and fixed q = 0.1 (see, Fig. 2) Fig. 1. The functions f(d) and g(d) for a) β = 1, q = 0.1; b) β = 1, q = 0.9. Fig. 2. a) β = 1, q = 0.1; t0 = 0.263156; Hausdorff distance d = 0.230226; b) β = 2, q = 0.1; t0 = 0.512988; Hausdorff distance d = 0.208046; c) β = 3, q = 0.1; t0 = 0.640823; Hausdorff distance d = 0.181048; d) β = 6, q = 0.1; t0 = 0.800514; Hausdorff distance d = 0.127635. III. SOME APPLICATIONS. It is well known that in many cases the existing modifications to the classical logistic and Gom- pertz models do not give very reliable results in approximating ”specific data”. We examine the following ”specific datasets”: Biomath 9 (2020), 2005033, http://dx.doi.org/10.11145/j.biomath.2020.05.033 Page 3 of 10 http://dx.doi.org/10.11145/j.biomath.2020.05.033 Nikolay Kyurkchiev, A new class of activation functions. Some related problems and applications Fig. 3. The fitted model (1). Example 1. We analyze the following data [4] data Communication := {{0.584, 0.027}, {0.649, 0.147},{0.909, 0.187},{1.039, 0.303}, {1.558, 0.453},{2.208, 0.527},{2.792, 0.580}, {3.052, 0.627},{3.312, 0.657},{4.091, 0.707}, {4.740, 0.753},{5, 0.780},{5.390, 0.827}, {7.078, 0.853},{7.597, 0.877},{8.961, 0.903}, {9.091, 0.927},{10.195, 0.950},{22.078, 0.980}, {24.610, 1}}; The cdf Mβ(t) for β = 0.484411 and q = 0.82547 is visualized on Fig. 3. Example 2. Analysis of ”data Nicotine” [5] data Nicotine := {{0.11, 0.021},{0.21, 0.053},{0.31, 0.063}, {0.41, 0.105},{0.51, 0.2},{0.61, 0.274}, {0.71, 0.358},{0.81, 0.495},{0.91, 0.632}, {1.01, 0.726},{1.11, 0.832},{1.21, 0.905}, {1.31, 0.942},{1.41, 0.958},{1.51, 0.974}, {1.61, 0.979},{1.71, 0.989},{1.81, 1}, {1.9, 1},{2, 1}}; After that using the model Mβ(t) for β = 1.98567 and q = 0.485475 we obtain the fitted model (see, Fig. 4). Example 3. Analysis of data ”Biomass pro- duced by Paesilomyces lilacinus 6029” [6]. After that using the model M∗β(t) = ωMβ(t) for ω = 10.521, β = 0.805824 and q = 0.97915 we obtain the fitted model (see, Fig. 5). Fig. 4. The fitted model (1). Fig. 5. The fitted model. The new activation function. We define the following activation function: A(t; β) = qe −tβ −qe tβ qe −tβ + qe tβ . (12) In antenna-feeder technique most often occurred signals are of types shown on Fig. 6 – Fig. 7. For β even, the corresponding approximation using model (7) is shown in Fig. 6. For β odd, the corresponding approximation using new activation function A(t; β) is shown in Fig. 7. A family of recurrence generated functions based on the A(t; β). Let us consider the following family of recur- rence generated functions Ai+1(t; β) = Ai(t + Ai(t; β); β), i = 0, 1, 2, . . . ; A0(t; β) = A(t; β), (13) based on the function A(t; β). Let for instance β = 1. Biomath 9 (2020), 2005033, http://dx.doi.org/10.11145/j.biomath.2020.05.033 Page 4 of 10 http://dx.doi.org/10.11145/j.biomath.2020.05.033 Nikolay Kyurkchiev, A new class of activation functions. Some related problems and applications Fig. 6. The function A(t;β); β = 4, q = 0.01, t0 = 0.587335; Hausdorff distance d = 0.138899; β = 6, q = 0.01, t0 = 0.701333; Hausdorff distance d = 0.111603; β = 8, q = 0.01, t0 = 0.766378; Hausdorff distance d = 0.0992629; β = 10, q = 0.01, t0 = 0.808266; Hausdorff distance d = 0.0867535; β = 16, q = 0.01, t0 = 0.87543; Hausdorff distance d = 0.0632673. Fig. 7. The function A(t;β); β = 3, q = 0.01, t0 = 0.491867; Hausdorff distance d = 0.152538; β = 7, q = 0.01, t0 = 0.737794; Hausdorff distance d = 0.107003; β = 13, q = 0.01, t0 = 0.848962; Hausdorff distance d = 0.073086. Fig. 8. The recurrence generated family: A0(t) (blue), A1(t) (red) and A2(t) (dashed). The recurrence generated family: A0(t),A1(t) and A2(t) is visualized on Fig. 8. Some properties of the new Topp–Leone–G– Family with baseline ”deterministic–type” (cdf) – (NTLG–DT) Q(t)(7). We study the Hausdorff approximation of the Heaviside step function ht0 (t) where t0 is the ”median” by families of the new Topp–Leone–G– Family with baseline ”deterministic–type” (cdf) – (NTLG–DT). The obtained two-sides estimations (see Propo- sition 1. [1] ) in particular case with usage of the baseline ”deterministic–type” (cdf) for α = 0.9; β = 0.3; q = 0.1 e αβ ( 1− 1 1−qet−1 ) ( 2 − ( 1 −qe t−1 )β)α ≤ Q(t) (14) ≤ 2αeαβ ( 1− 1 1−qet−1 ) are given in Fig. 9 a. Let t0 is the value for which Q(t0) = 12 . The Hausdorff distance d between the function ht0 (t) and Q(t) satisfies the relation Q(t0 + d) = 1 −d. (15) Biomath 9 (2020), 2005033, http://dx.doi.org/10.11145/j.biomath.2020.05.033 Page 5 of 10 http://dx.doi.org/10.11145/j.biomath.2020.05.033 Nikolay Kyurkchiev, A new class of activation functions. Some related problems and applications Fig. 9. a) The two-sides estimations (14) for α = 0.9; β = 0.3; q = 0.1; b) The model Q(t) for α = 0.9; β = 0.3; q = 0.1, t0 = 0.0852097; H–distance d = 0.116811 For fixed α = 0.9; β = 0.3; q = 0.1 we find t0 = 0.0852097 and from the nonlinear equation (15) we have d = 0.116811 (see, Fig. 9 b). From Fig. 9 it can be seen that these estimations can be used as ”confidence bounds”, which are extremely useful for the specialists in the choice of model for cumulative data approximating in areas of Biostatistics, Population dynamics, Growth the- ory, Debugging and Test theory, Computer viruses propagation, Financial and Insurance mathematics. For other results, see [8]–[53], [59]. IV. CONCLUDING REMARKS. The results obtained in this article can be suc- cessfully continued. 1. For example, we study the Hausdorff ap- proximation of the Heaviside step function ht0 (t) where t0 is the ”median” by families of the new Topp–Leone–G–Family Q1(t) with baseline ”deterministic–inverse–type” (cdf) – (NTLG–DIT) G(t) = qe 1 t −1, where 0 < q < 1, Q1(t) = e αβ ( 1− 1 qe 1 t −1 )  2 −eβ ( 1− 1 qe 1 t −1 )  α (16) Fig. 10. a) The two-sided bounds (17) for α = 0.6; β = 0.1; q = 0.4; b) The model Q1(t) for α = 0.6; β = 0.1; q = 0.4, t0 = 0.698075; H–distance d = 0.153113 The obtained two-sided bounds (see Proposition 1. [1] ) in particular case with usage of the base- line ”deterministic–inverse–type” (cdf) for α = 0.6, β = 0.1, q = 0.4, e αβ ( 1− 1 qe 1 t −1 ) ( 2 −qβ ( e 1 t −1 ))α (17) ≤ Q1(t) ≤ 2αe αβ ( 1− 1 qe 1 t −1 ) are given in Fig. 10 a. Example 4. Storm worm one of the most biggest cyber threats of 2008. We analyze the following data [7] data Storm IDs := {{1, 0.843}, {4, 0.926},{5, 0.954},{6, 0.967}, {7, 0.976},{8, 0.981},{9, 0.985}, {10, 0.991},{22, 0.995},{38, 0.997}, {51, 0.998},{64, 0.9985},{74, 0.999}, {83, 1},{100, 1},{367, 1}} Biomath 9 (2020), 2005033, http://dx.doi.org/10.11145/j.biomath.2020.05.033 Page 6 of 10 http://dx.doi.org/10.11145/j.biomath.2020.05.033 Nikolay Kyurkchiev, A new class of activation functions. Some related problems and applications Fig. 11. The fitted model Q1(t). The cdf Q1(t) for α = 0.14146, β = 161.078891, q = 0.9 is visualized on Fig. 11. Exploring both features - ”confidential curves” and ”super saturation” is a must when choosing the right model. 2. Following the ideas given in [54]–[56] we consider the following new differential model:  dy(t) dt = ky(t)s(t) = ky(t)qe t−1 y(t0) = y0 (18) where k > 0 and 0 < q < 1. The general solution of the differential equation (18) is of the following form: y(t) = y0e k q Ei(et lnq)−k q Ei(lnq) (19) where Ei(.) is the traditional exponential integral. The new ”growth” function y(t) and the ”input function” s(t) = qe t−1 are visualized on Fig. 12– Fig. 13. Example 4. We will analyze a sample of exper- imental data obtained by the biologist T. Carlson in 1913 about the development of Saccharomyces culture in nutrient medium (see, for example [58], [57]). After that using the model M∗(t) = ωe k q Ei(et lnq)−k q Ei(lnq) for k = 0.293574, Fig. 12. The ”growth” function y(t)–(red) and s(t)–(green) for k = 12.6; q = 0.14; y0 = 0.01. Fig. 13. The ”growth” function y(t)–(red) and s(t)–(green) for k = 1.1; q = 0.906; y0 = 0.1. Fig. 14. The fitted model M∗(t). Biomath 9 (2020), 2005033, http://dx.doi.org/10.11145/j.biomath.2020.05.033 Page 7 of 10 http://dx.doi.org/10.11145/j.biomath.2020.05.033 Nikolay Kyurkchiev, A new class of activation functions. Some related problems and applications Fig. 15. The fitted model M∗(t). q = 0.999983 and ω = 30.114 we obtain the fitted model (see, Fig. 14). Example 5. Analysis of data ”Biomass pro- duced by Paesilomyces sinclairi ascomycota”. After that using the model M∗(t) for ω = 0.305247, k = 3.01914 and q = 0.83 we obtain the fitted model (see, Fig. 15). The general solution y(t) has been applied widely in life testing experiments and debugging theory. ACKNOWLEDGMENT The author would like to thank the anonymous referees for their valuable comments. This paper is supported by the National Sci- entific Program ”Information and Communica- tion Technologies for a Single Digital Market in Science, Education and Security (ICTinSES)”, financed by the Ministry of Education and Science. REFERENCES [1] R. Bantan, F. Jamal, Ch. Chesneau, M. Elgarhy, A New Power Topp-Leone Generated Family of Distributions with Applications, Entropy, 21, 12, 1177, 2019. [2] A. Sarhan, Bayesian analysis of the discrete two– parameter bathtub hazard distribution, J. of Math. Sci. and Modelling, 2, 3, 183–192, 2019. [3] B. Sendov, Hausdorff Approximations, Kluwer, Boston, 1990. [4] P. E. Oguntunde, M. A. Khaleel, H. I. Okagbue, O. A. Odetunmibi, The Topp-Leone Lomax (TLLo) Distribution with Applications to Airbone Communication Transceiver Dataset, Wireless Personal Communications, 109, 349– 360, 2019. [5] A. Afify, A. Abdellatif, The extended Burr XII distribu- tion: properties and applications, J. of Nonlinear Sci. and Appl., 13, 133–146, 2020. [6] A. Sharma, S. Sharma, A. Mittal, S. Naik, Statistical optimization of growth media for Paecilomyces lilacinus 6029 using non–edible oil cakes, Ann. of Microbiology, 64, 515–520, 2014. [7] S. Sarat, A. Terzis, HiNRG Technical Report: 01-10-2007 Measuring the Storm Worm Network, 2007. [8] B. Gompertz, On the nature of the function expressive of the law of human mortality, and on a new mode of determining the 62 value of life contingencies, Philos Trans R Soc London, 115, 513-585, 1825. [9] S. Markov, Reaction networks reveal new links between Gompertz and Verhulst growth functions, Biomath, 8, 1, 2019. [10] L. Rogers-Bennett, D. W. Rogers, A Two-Step Growth Curve: Approach to the von Bertalanffy and Gompertz Equations, Advances in Pure Mathematics, 6, 321-330, 2016. [11] K. Tjrve, E. Tjrve, The use of Gompertz models in growth analyses, and new Gompertz-model approach: An addition to the Unified-Richards family, PLoS ONE, 12, 6, 2017. [12] M. Tahir, G. Cordeiro, Compounding of distributions: A survey and new generalized classes, J. of Stat. Distr. and Appl., 3, 13, 1–35, 2016. [13] D. Satoh, Property of logistic data exposed with Gom- pertz model and resistance to noise in actual data, Japan J. of Ind. and Appl. Math., 2019. [14] S. Yamada, A stochastic software reliability growth model with Gompertz curve, Trans. IPSJ, 33, 964–969, 1992. (in Japanese) [15] D. Satoh, S. Yamada, Discrete equations and software reliability growth models, in: Proc. 12th Int. Symp. on Software Reliab. and Eng., 176–184, 2001. [16] H. Okamura, T. Dohi, S. Osaki, EM algorithms for logistic software reliability models, In: Proc. 7th IASTED Int. Conf. on Software Eng., 263–268, 2004. [17] S. Yamada, M. Ohba, S. Osaki, S–shaped reliability growth modeling for software error detection, IEEE Trans, Reliab., R–32, 475–478, 1983. [18] M. Xie, Software Reliability Modelling, World Scien- tific, Singapore, 1991. [19] H. Pham, Software Reliability, Springer, Singapore, 2000. [20] N. Kyurkchiev, On a sigmoidal growth function gen- erated by reaction networks. Some extensions and ap- plications, Communications in Applied Analysis, 23, 3, 383–400, 2019. [21] D. Costarelli, R. Spigler, Constructive Approximation by Superposition of Sigmoidal Functions, Anal. Theory Appl., 29, 169–196, 2013. [22] D. Costarelli, G. Vinti, Pointwise and uniform approx- imation by multivariate neural network operators of the max-product type, Neural Networks, 81, 81–90, 2016. [23] N. Guliyev, V. Ismailov, A single hidden layer feed- Biomath 9 (2020), 2005033, http://dx.doi.org/10.11145/j.biomath.2020.05.033 Page 8 of 10 http://dx.doi.org/10.11145/j.biomath.2020.05.033 Nikolay Kyurkchiev, A new class of activation functions. Some related problems and applications forward network with only one neuron in the hidden layer san approximate any univariate function, Neural Computation, 28, 1289–1304, 2016. [24] N. Kyurkchiev, S. Markov, On the Hausdorff distance between the Heaviside step function and Verhulst logistic function, J. Math. Chem., 54, 1, 109–119, 2016. [25] N. Kyurkchiev, S. Markov, Sigmoid functions: Some Approximation and Modelling Aspects, LAP LAMBERT Academic Publishing, Saarbrucken, 2015, ISBN 978-3- 659-76045-7. [26] N. Kyurkchiev, A. Iliev, S. Markov, Some Techniques for Recurrence Generating of Activation Functions: Some Modeling and Approximation Aspects, LAP LAMBERT Academic Publishing, 2017, ISBN: 978-3-330-33143-3. [27] A. Iliev, N. Kyurkchiev, S. Markov, A Note on the New Activation Function of Gompertz Type, Biomath Communications, 4, 2, 20 pp., 2017. [28] R. Anguelov, M. Borisov, A. Iliev, N. Kyurkchiev, S. Markov, On the chemical meaning of some growth mod- els possessing Gompertzian-type property, Math. Meth. Appl. Sci., 1–12, 2017. [29] R. Anguelov, N. Kyurkchiev, S. Markov, Some proper- ties of the Blumberg’s hyper-log-logistic curve, Biomath, 7, 1, 8 pp., 2018. [30] S. Markov, A. Iliev, A. Rahnev, N. Kyurkchiev, On the exponential–generalized extended Compertz cumulative sigmoid, International Journal of Pure and Applied Math- ematics, 120, 4, 555–562, 2018. [31] N. Pavlov, A. Iliev, A. Rahnev, N. Kyurkchiev, Some software reliability models: Approximation and modeling aspects, LAP LAMBERT Academic Publishing, 2018, ISBN: 978-613-9-82805-0. [32] A. Iliev, N. Kyurkchiev, A. Rahnev, T. Terzieva, Some models in the theory of computer viruses propagation, LAP LAMBERT Academic Publishing, 2019, ISBN: 978-620-0-00826-8. [33] A. Iliev, N. Kyurkchiev, S. Markov, On the approx- imation of the cut and step functions by logistic and Gompertz functions, Biomath, 4, 2–13, 2015. [34] N. Kyurkchiev, A. Iliev, A. Rahnev, A new class of activation functions based on the correcting amendments of Gompertz-Makeham type, Dynamic Systems and Ap- plications, 28, 2, 243–257, 2019. [35] N. Kyurkchiev, A. Iliev, A. Rahnev, Some Families of Sigmoid Functions: Applications to Growth Theory, LAP LAMBERT Academic Publishing, 2019, ISBN: 978-613- 9-45608-6. [36] N. Pavlov, A. Iliev, A. Rahnev, N. Kyurkchiev, Nontriv- ial Models in Debugging Theory (Part 2), LAP LAM- BERT Academic Publishing, 2018, ISBN: 978-613-9- 87794-2. [37] O. Rahneva, T. Terzieva, A. Golev, Investigations on the Zubair-family with baseline Ghosh-Bourguignon’s ex- tended Burr XII cumulative sigmoid. Some applications, Neural, Parallel, and Scientific Computations, 27, 1, 11– 22, 2019. [38] A. Malinova, O. Rahneva, A. Golev, V. Kyurkchiev, Investigations on the Odd-Burr-III-Weibull cumulative sigmoid. Some applications, Neural, Parallel, and Scien- tific Computations, 27, 1, 35–44, 2019. [39] N. Kyurkchiev, Investigations on a hyper-logistic model. Some applications, Dynamic Systems and Applications, 28, 2, 351–369, 2019. [40] S. Markov, A. Iliev, A. Rahnev, N. Kyurkchiev, A note on the n-stage growth model. Overview, Biomath Communications, 5, 2, 79–100, 2018. [41] E. Angelova, A. Golev, T. Terzieva, O. Rahneva, A study on a hyper-power-logistic model. Some applica- tions, Neural, Parallel, and Scientific Computations, 27, 1, 45–57, 2019. [42] N. Kyurkchiev, A. Iliev, A. Rahnev, On a special choice of nutrient supply for cell growth in a continuous bioreac- tor. Some modeling and approximation aspects, Dynamic Systems and Applications, 28, 3, 587–606, 2019. [43] N. Kyurkchiev, A. Iliev, Extension of Gompertz-type equation in modern science. 240 Anniversary of the birth of B. Gompertz, LAP LAMBERT Academic Publishing, 2018, ISBN: 978-613-9-90569-0. [44] N. Kyurkchiev, A. Iliev, A. Rahnev, T. Terzieva, Proper- ties of a power Topp–Leone G–family with baseline Gom- pertz cumulative distribution function, 2020. (in print) [45] B. I. Yun, A Neural Network Approximation Based on a Parametric Sigmoidal Function, Mathematics, 7, 262, 2019. [46] N. Kyurkchiev, Comments on the Yun’s algebraic ac- tivation function. Some extensions in the trigonometric case, Dynamic Systems and Applications, 28, 3, 533– 543, 2019. [47] N. Kyurkchiev, A. Andreev, Approximation and Antenna and Filters synthesis. Some Moduli in Programming En- vironment MATHEMATICA, LAP LAMBERT Academic Publishing, Saarbrucken, 2014, ISBN 978-3-659-53322- 8. [48] Bl. Sendov, H. Schinev, N. Kjurkchiev, Hausdorff- synthesis of aerial grids in scanning the directive di- agram, Electropromishlenost i Priboroostroene, 16, 5, 203–205, 1981. (in Bulgarian) [49] N. Kyurkchiev, Synthesis of slot aerial grids with Haus- dorff type directive patterns, PhD Thesis, Department of Radio-Electronics, VMEI, Sofia, 1979. (in Bulgarian) [50] N. Kyurkchiev, Bl. Sendov, Approximation of a class of functions by algebraic polynomials with respect to Hausdorff distance, Ann. Univ. Sofia, Fac. Math., 67, 573–579, 1975. (in Bulgarian) [51] N. Kyurkchiev, S. Markov, On the numerical approxi- mation of the ”cross” set, Ann. Univ. Sofia, Fac. Math., 66, 19–25, 1974. (in Bulgarian) [52] J. Peters, Foundations of Computer Vision: Computa- tional Geometry, Visual Image Structures and Object Shape Detection, Springer, Cham, 2017. [53] N. Kyurkchiev, G. Nikolov, Comments on some new classes of sigmoidal and activation functions. Applica- tions, Dynamic Systems and Applications, 28, 4, 789– 808, 2019. Biomath 9 (2020), 2005033, http://dx.doi.org/10.11145/j.biomath.2020.05.033 Page 9 of 10 http://dx.doi.org/10.11145/j.biomath.2020.05.033 Nikolay Kyurkchiev, A new class of activation functions. Some related problems and applications [54] N. Kyurkchiev, S. Markov, On a logistic differential model. Some applications, Biomath Communications, 6, 1, 34–50, 2019. [55] N. Kyurkchiev, A. Iliev, A. Rahnev, A special choice of nutrient supply for cell growth in logistic differential model. Some applications, Conference Proceedings of AIP, 2019. [56] N. Kyurkchiev, A. Iliev, A. Golev, A. Rahnev, On a Special Choice of Nutrient Supply with Marshall-Olkin Correction. Some Applications, Communications in Ap- plied Analysis, 23, 3, 401–419, 2019. [57] Bl. Sendov, R. Maleev, S. Markov, S. Tashev, Mathe- matics for Biologists, University Publishing House ”St. Kliment Ohridski”, Sofia, 1991. [58] E. Bohl, Mathematik in der Biologie, 4., vollständig überarbeitete und erweiterte Auflage, Springer, Berlin, 2006. [59] N. Kyurkchiev, Selected Topics in Mathematical Modeling: Some New Trends (Dedicated to Academician Blagovest Sendov (1932-2020)), LAP LAMBERT Academic Publishing, 2020; ISBN: 978-620-2-51403-3. Biomath 9 (2020), 2005033, http://dx.doi.org/10.11145/j.biomath.2020.05.033 Page 10 of 10 http://dx.doi.org/10.11145/j.biomath.2020.05.033 Introduction and Preliminaries Main Results The case =1. The case =1. Some Applications. Concluding remarks. References