EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS Vol. 17, No. 1, 2024, 105-115 ISSN 1307-5543 – ejpam.com Published by New York Business Global Fekete-Szegö Functional of a Subclass of Bi-Univalent Functions Associated with Gegenbauer Polynomials Waleed Al-Rawashdeh Department of Mathematics, Faculty of Science, Zarqa University, 2000 Zarqa, 13110 Jordan Abstract. In this paper, we introduce and investigate a class of bi-univalent functions, denoted by F(n, α, β), that depends on the Ruscheweyh operator and defined by the use of Gegenbauer Polynomials. For functions in this class, we derive the estimations for the initial Taylor-Maclaurin coefficients |a2| and |a3|. Moreover, we obtain the classical Fekete-Szegö inequality of functions belonging to this class. 2020 Mathematics Subject Classifications: 30C45, 30C50, 33C45, 33C05, 11B39 Key Words and Phrases: Analytic Functions, Taylor-Maclaurin Series, Univalent and Bi- Univalent Functions, Principle of Subordination, Hadamard Product, Ruscheweyh Operator, Ruscheweyh Derivative, Gegenbauer Polynomials, Chebyshev polynomials, Coefficient estimates, Fekete-Szegö Inequality 1. Introduction Let A be the family of all analytic functions f that are defined on the open unit disk D = {z ∈ C : |z| < 1} and normalized by the conditions f(0) = 0 and f ′(0) = 1. Any function f ∈ A has the following Taylor-Maclarin series expansion: f(z) = z + ∞∑ n=2 anz n, where z ∈ D. (1) Let S denote the class of all functions f ∈ A that are univalent in D. Let the functions f and g be analytic in D, we say the function f is subordinate by the function g in D, denoted by f(z) ≺ g(z) for all z ∈ D, if there exists a Schwartz function w, with w(0) = 0 and |w(z)| < 1 for all z ∈ D, such that f(z) = g(w(z)) for all z ∈ D. In particular, if the function g is univalent over D then f(z) ≺ g(z) equivalent to f(0) = g(0) and f(D) ⊂ g(D. For more information about the Subordination Principle we refer the readers to to the monographs [9], [23] and [24]. DOI: https://doi.org/10.29020/nybg.ejpam.v17i1.5004 Email address: walrawashdeh@zu.edu.jo (W. Al-Rawashdeh) https://www.ejpam.com 105 © 2024 EJPAM All rights reserved. W. Al-Rawashdeh / Eur. J. Pure Appl. Math, 17 (1) (2024), 105-115 106 As known univalent functions are injective (one-to-one) functions. Hence, they are invertible and the inverse functions may not be defined on the entire unit disk D. In fact, the Koebe one-quarter Theorem tells us that the image of D under any function f ∈ S contains the disk D(0, 1/4) of center 0 and radius 1/4. Accordingly, every function f ∈ S has an inverse f−1 = g which is defined as g(f(z)) = z, z ∈ D f(g(w)) = w, |w| < r(f); r(f) ≥ 1/4. Moreover, the inverse function is given by g(w) = w − a2w 2 + (2a22 − a3)w 3 − (5a32 − 5a2a3 + a4)w 4 + · · ·· (2) For this reason, we define the class Σ as follows. A function f ∈ A is said to be bi-univalent if both f and f−1 are univalent in D. Therefore, let Σ denote the class of all bi-univalent functions in A which are given by equation (1). For example, the following functions belong to the class Σ: z 1− z , − log(1− z), log √ 1 + z 1− z . However, Koebe function, 2z − z2 2 and z 1− z2 do not belong to the class Σ. For more information about univalent and bi-univalent functions we refer the readers to the articles [19], [22], [25], the monograph [10], [12] and the references therein. In the year 1784, Legendre [18] introduced and studied the orthogonal polynomials. Traditionally, orthogonal polynomials are crucial in approximation theory where are used in polynomial interpolation. Moreover, under specific restrictions, orthogonal polynomials are frequently used in the study of differential equations. In particular in some special cases of Sturm-Liouville differential equation. An example of orthogonal polynomials is a Gegenbauer polynomial. Special cases of Gegenbauer polynomials are Legendre polyno- mials and the Chebyshev polynomials of the first and second kind. For more information about orthogonal polynomials we refer the readers to the monograph [8]. We define Gegen- bauer polynomials in the next section. The subject of the geometric function theory in complex analysis has been investi- gated by many researchers in recent years, the typical problem in this field is studying a functional made up of combinations of the initial coefficients of the functions f ∈ A. For a function in the class S, it is well-known that |an| is bounded by n. Moreover, the coefficient bounds give information about the geometric properties of those functions. For instance, the bound for the second coefficients of the class S gives the growth and distor- tion bounds for the class. In addition, the Fekete-Szegö functional arises naturally in the investigation of univalency of analytic functions. In the year 1933, Fekete and Szegö [11] W. Al-Rawashdeh / Eur. J. Pure Appl. Math, 17 (1) (2024), 105-115 107 found the maximum value of |a3−λa22|, as a function of the real parameter 0 ≤ λ ≤ 1 for a univalent function f . Since then, the problem of dealing with the Fekete-Szegö functional for f ∈ A with any complex λ is known as the classical Fekete-Szegö problem. There are many researchers investigated the Fekete-Szegö functional and the other coefficient estimates problems, for example see the articles [20], [15], [21], [17], [11], [22], [14] and the references therein. 2. Preliminaries In this section we present some information that are curial for the main results of this paper. we start by defining our subclass. In the year 1994, Szynal [28] introduced and studied a subclass F(α) of the class A consisting of functions of the form f(z) = ∫ 1 −1 K(z, x) dσ(x), (3) where K(z, x) = z (z2 − 2xz + 1)α , α ≥ 0, −1 ≤ x ≤ 1, and σ is the probability measure on [−1, 1]. Moreover, the functionK(z, x) has the following Taylor-Maclaurin series expansion K(z, x) = z + Cα 1 (x)z 2 + Cα 2 (x)z 3 + Cα 3 (x)z 4 + · · ·, where Cα n (x) denotes the Gegenbauer polynomials of order α and degree n in x. Fur- thermore, for any real numbers α, x ∈ R ,with α ≥ 0 and −1 ≤ x ≤ 1, and z ∈ D the generating function of Gegenbauer polynomials is given by Hα(z, x) = (z2 − 2xz + 1)−α. Moreover, for any fixed x the function Hα is analytic on the unit disk D and its Taylor-Maclaurin series is given by Hα(z, x) = ∞∑ n=0 Cα n (x)z n. In addition, if f ∈ F(α) that is given by (3), the nth coefficient can be written as an = ∫ 1 −1 Cα n−1(x) dσ(x). In addition, Gegenbauer polynomials can be defined in terms of the following recurrence relation: Cα n (x) = 2x(n+ α− 1)Cα n−1(x)− (n+ 2α− 2)Cα n−1(x) n , (4) with initial values Cα 0 (x) = 1, Cα 1 (x) = 2αx, and Cα 2 (x) = 2α(α+ 1)x2 − α. W. Al-Rawashdeh / Eur. J. Pure Appl. Math, 17 (1) (2024), 105-115 108 It is well-known that the Gegenbauer polynomials and their special cases such as Legendre polynomials Ln(x) and the Chebyshev polynomials of the second kind Tn(x), are orthogonal polynomials, where the values of α are α = 1/2 and α = 1 respectively, more precisely Ln(x) = C1/2 n (x), and Tn(x) = C1 n(x). For more information about the Gegenbauer polynomials and their special cases, we refer the readers to the articles [4], [3], [7], [6], [5], [21], [16], [22], [13], [14], the monograph [10], [12], [27], and the references therein. In the year 1975, Ruscheweyh [26] introduced the operator R which defined, using the Hadamard product, as follows Rλf(z) = f(z) ∗ z (1− z)1−λ , where f ∈ A, z ∈ D and real number λ ≥ −1. For λ = n ∈ N0 = N ∪ {0}, we get the Rscheweyh derivative Rn of order n of the function f : Rnf(z) = z ( zn−1f(z) )(n) n! . Moreover, the Taylor-Maclaurin series of Rnf is given by Rnf(z) = z + ∞∑ k=2 σ(n, k)akz k, σ(n, k) = Γ(n+ k) (k − 1)!Γ(n+ 1) . (5) We say that a function f ∈ Σ in the subclass F(n, α, β) if it satisfies the following subordination conditions, associated with the Gegenbauer Polynomials, for all z, w ∈ D: (Rnf(z))′ + βz(Rnf(z))′′ ≺ Hα(z, x) (6) and (Rng(w))′ + βw(Rng(w))′′ ≺ Hα(w, x), (7) where α > 0, β > 0, n ∈ N0, x ∈ (12 , 1] and g(w) is defined by equation (2). The following lemma (see[17]) is a well-known fact, so we omit its proof. Lemma 1. Let K, L ∈ R and p, q ∈ C. If |p| < R and |q| < R, |(K + L)p+ (K − L)q| ≤ { 2|K|R, if |K| ≥ |L| 2|L|R, if |K| ≤ |L| W. Al-Rawashdeh / Eur. J. Pure Appl. Math, 17 (1) (2024), 105-115 109 Our investigation in this paper is motivated by the work of the researchers presented in the papers [1], [2], and [14]. In this presenting paper, we investigate a subclass of bi- univalent functions Σ in the open unit disk D, which we denote by F(n, α, β) with α > 0, β > 0 and n ∈ N0. For functions in this subclass, we obtain the estimates for the initial Taylor-Maclarin coefficients |a2| and |a3|. Furthermore, we examine the corresponding Fekete-Szegö functional problem for functions in this subclass. 3. Initial Coefficient estimates for the function class F(n, α, β) In this section, we provide bounds for the initial Taylor-Maclaurin coefficients for the functions belong to the class F(n, α, β) which are given by equation (1). Theorem 1. Let the function f given by (1) be in the class F(n, α, β). Then |a2| ≤ 2αx √ x(n!)√ |(3α(n+ 2)!(1 + 2β)x2 − 4(1 + β)2(n+ 1)(n+ 1)!{(2 + 2α)x2 − 1}| (8) and |a3| ≤ 4αx(n!) 3(1 + 2β)(n+ 2)! + α2x2 (1 + β)2(n+ 1)2 (9) Proof. Let f belong to the class F(n, α, β). Then Using (6) and (7) we can find two analytic functions p and q on the unit disk D such that (Rnf(z))′ + βz(Rnf(z))′′ ≺ Hα(x, p(z)), (10) and (Rng(w))′ + βw(Rng(w))′′ ≺ Hα(x, q(w)). (11) where the analytic functions p and q are given by p(z) = 1 + p1z + p2z 2 + p3z 3 + · · · where z ∈ D, and q(w) = 1 + q1w + q2w 2 + q3w 3 + · · · where w ∈ D Such that p(0) = q(0) = 0, and for all z, w ∈ D |p(z)| < 1 and |q(z)| < 1. Moreover, it is well-known that (see, for details [10]) for all j ∈ N |pj | ≤ 1 and |qj | ≤ 1. W. Al-Rawashdeh / Eur. J. Pure Appl. Math, 17 (1) (2024), 105-115 110 Now, upon comparing the coefficients in both sides of (10) and (11) we obtain the following 2(1 + β)σ(n, 2)a2 = Cα 1 (x)p1, (12) 3(1 + 2β)σ(n, 3)a3 = Cα 1 (x)p2 + Cα 2 (x)p 2 1, (13) −2(1 + β)σ(n, 2)a2 = Cα 1 (x)q1, (14) and 3(1 + 2β)σ(n, 3)(2a22 − a3) = Cα 1 (x)q2 + Cα 2 (x)q 2 1 (15) Using equations (12) and (14) we get p1 = −q1 (16) Moreover, adding the square of equations (12) and (14) we get 8(1 + β)2[σ(n, 2)]2a22 = [Cα 1 (x)] 2(p21 + q21) (17) By adding equations (13) and (15) we get 6(1 + 2β)σ(n, 3)a22 = [Cα 1 (x)](p2 + q2) + [Cα 2 (x)](p 2 1 + q21) (18) In view of equation (17), equation (18) can be written as ( 6(1 + 2β)σ(n, 3)[Cα 1 (x)] 2 − 8(1 + β)2[Cα 2 (x)][σ(n, 2)] 2 ) a22 = [Cα 1 (x)] 3(p2 + q2) (19) Using equation (5), equation (19) becomes a22 = 4α2x3(p2 + q2) 3(1 + 2β)(n+ 1)(n+ 2)αx2 − 4(1 + β)2(n+ 1)2[2(α+ 1)x2 − 1] . Using the facts |p2| ≤ 1 and |q2| ≤ 1, we get the desired estimate of a2: |a2| ≤ 2αx √ x(n+ 1)√ (n+ 1)2|(3α(1 + 2β)(n+ 2)x2 − 4(1 + β)2(n+ 1){(2 + 2α)x2 − 1}| . Next, we look for the bound of |a3|. Subtracting equation (15) from equation (13) and using equation (16), we get a3 = Cα 1 (x)(p2 − q2) 6(1 + 2β)σ(n, 3) + a22. (20) W. Al-Rawashdeh / Eur. J. Pure Appl. Math, 17 (1) (2024), 105-115 111 In view of equation (17), we obtain a3 = Cα 1 (x)(p2 − q2) 6(1 + 2β)σ(n, 3) + [Cα 1 (x)] 2p21 4(1 + β)2[σ(n, 2)]2 . Hence, Using Equation (5) and the facts |p2| ≤ 1 and |q2| ≤ 1, we get the desired estimate of a3: |a3| ≤ 4αx 3(1 + 2β)(n+ 2)(n+ 1) + α2x2 (1 + β)2(n+ 1)2 . This completes the proof of Theorem 1. Taking α = 1, we get the following interesting corollary of Theorem 1. These initial coefficient estimates are related to Chebyshev polynomials of the second kind. The prove is similar to the proof of previous theorem, so we omit the proof’s details. Corollary 1. Let the function f given by (1) be in the class F(n, 1, β). Then |a2| ≤ 2x √ x(n!)√ |(3(n+ 2)!(1 + 2β)x2 − 4(1 + β)2(n+ 1)(n+ 1)!(4x2 − 1)| , and |a3| ≤ 4x(n!) 3(1 + 2β)(n+ 2)! + x2 (1 + β)2(n+ 1)2 . On the other hand, taking β = 1, we get the following corollary. Corollary 2. Let the function f given by (1) be in the class F(n, α, 0). Then |a2| ≤ 2αx √ x(n!)√ |(9α(n+ 2)!x2 − 16(n+ 1)(n+ 1)!{(2 + 2α)x2 − 1}| , and |a3| ≤ 4αx(n!) 9(n+ 2)! + α2x2 4(n+ 1)2 . 4. Fekete-Szegö problem for the function class F(n, α, β) In this section, we consider the classical Fekete-Szegö problem for our presenting class F(n, α, β). Theorem 2. Let the function f given by (1) be in the class F(n, α, β). Then for some ζ ∈ R, |a3 − ζa22| ≤ { 4αx B , if |1− ζ| ≤ ∆(α,n,β) 4Bα2x2 16α3x3|1−ζ| ∆(α,n,β) , if |1− ζ| ≥ ∆(α,n,β) 4Bα2x2 , (21) where ∆(α, n, β) = 4α[B − 4(1 + β)2(n+ 1)2(α+ 1)]x2 − 8α(1 + β)2(n+ 1)2, W. Al-Rawashdeh / Eur. J. Pure Appl. Math, 17 (1) (2024), 105-115 112 and B = 3(1 + 2β)(n+ 2)(n+ 1) Proof. For some real number ζ, using equation (20) we have a3 − ζa22 = Cα 1 (x)(p2 − q2) 6(1 + 2β)σ(n, 3) + (1− ζ)a22. In view of equation (19), we obtain a3 − ζa22 = Cα 1 (x)(p2 − q2) 6(1 + 2β)σ(n, 3) + (1− ζ)[Cα 1 (x)] 3(p2 + q2) 6(1 + 2β)σ(n, 3)[Cα 1 (x)] 2 − 8(1 + β)2[Cα 2 (x)][σ(n, 2)] 2 . The last expression can be written as: a3 − ζa22 = Cα 1 (x)[(K − L)p2 + (K + L)q2], where K = 1 6(1 + aβ)σ(n, 3) , and L = (1− ζ)[Cα 1 (x)] 2 △(α, n, β) . Using Lemma 1, we get the following |a3 − ζa22| ≤ { 2| Cα 1 (x) 6(1+aβ)σ(n,3) |, if |K| ≥ |L| 2| (1−ζ)[Cα 1 (x)]3 △(α,n,β) |, if |K| ≤ |L| . Using the initial values (4) and equation (5), we get the desired inequality (21). This completes the proof of Theorem 2. The following corollaries are just consequences of Theorem 2. Taking α = 1, we get the Fekete-Szegö inequality that is related to Chebyshev polynomials of the second kind. Corollary 3. Let the function f given by (1) be in the class F(n, 1, β). Then for some ζ ∈ R, |a3 − ζa22| ≤ { 4x B , if |1− ζ| ≤ G 16x3|1−ζ| 4B(n+2)(n+1)x2−8(1+β)(n+1)2(4x2−1) , if |1− ζ| ≥ G, (22) where G = 4B(n+ 2)(n+ 1)x2 − 8(1 + β)(n+ 1)2(4x2 − 1) 2Bx2 . Taking β = 1, we get the following corollary. Corollary 4. Let the function f given by (1) be in the class F(n, α, 0). Then for some ζ ∈ R, |a3 − ζa22| ≤ { 4(n!)αx 9(n+2)! , if |1− ζ| ≤ (n!)H(n,α) 36(n+2)!α2x2 16α3x3|1−ζ| H(n,α) , if |1− ζ| ≥ (n!)H(n,α) 36(n+2)!α2x2 , (23) where H(n, α) = 4αx2(n+ 1) (9(n+ 2)− 16(n+ 1)(α+ 1))− 32α(n+ 1)2. REFERENCES 113 5. Conclusion This research paper has investigated a new subclass of bi-univalent functions, defined in terms of the Ruscheweyh derivative Rn of order n, by the means of Gegenbauer polyno- mials. For functions belong to this function class, the author has derived estimates for the Taylor-Maclaurin initial coefficients and Fekete-Szegö functional problem. The work pre- sented in this paper will lead to many different results for subclasses defined by the means of Legendre polynomials Ln(x) = C 1/2 n (x) and the Chebyshev polynomials of the second kind Tn(x) = C1 n(x). Moreover, the presented work in this paper will inspire researchers to extend its concepts to harmonic functions and symmetric q-calculus. Acknowledgements The author would like to express his sincerest thanks to the referees for their valuable comments and various useful suggestions. This research is partially funded by Zarqa University. The author would like to express his sincerest thanks to Zarqa University for the financial support. References [1] K. I. Abdullah and N. H. Mohammed. Bounds For the Coefficients of Two New Subclasses of Bi-Univalent Functions. Science Journal of University of Zakho, 10:66– 69, 2022. [2] W. Al-Rawashdeh. Horadam Polynomials and a Class of Bi-Univalent Functions Defined by Ruscheweyh Operator. preprint. [3] W. Al-Rawashdeh. Coefficient Bounds of a class of Bi-Univalent Functions Related to Gegenbauer Polynomials. International Journal of Mathematics and Mathematical Sciences, Article ID 2573044:7 pages, 2023. [4] W. Al-Rawashdeh. Applications of Gegenbauer Polynomials on a Class of Non- Bazilevic Functions. International Journal of Mathematics and Computer Science, 19:635–642, 2024. [5] C. Cesarano. Identities and generating functions on Chebyshev polynomials. Georgian Mathematical Journal, 19, 2012. [6] C. Cesarano. Integral representations and new generating functions of Chebyshev polynomials. Hacettpe Journal of Mathematics and Statistics, 44, 2015. [7] C. Cesarano. Multi-dimensional Chebyshev polynomials: a non-conventional ap- proach. Communications in Applied Industrial Mathematics, 10, 2019. [8] B. Doman. The Classical Orthogonal Polynomials. World Scientific, Singapore, 2015. REFERENCES 114 [9] P. Duren. Subordination in Complex Analysis, Lecture Notes in Mathematics. Springer, Berlin, Germany, 599, 1977. [10] P. Duren. Univalent functions. Grundlehren der Mathematischen Wissenschaften 259, Springer-Verlag, New York, 1983. [11] M. Fekete and G. Szegö. Eine Bemerkung Über ungerade Schlichte Funktionen. Journal of London Mathematical Society, s1-8, 1933. [12] A. W. Goodman. Univalent functions. Mariner Publishing Co. Inc., Boston, 1983. [13] N. Magesh H. Orhan and V. Balaji. Second Hankel determinant for certain class of bi-univalent functions defined by Chebyshev polynomials. Asian-European Journal of Mathematics, 12(2):1950017, 2019. [14] M. Kamali H.M. Srivastava and A. Urdaletova. A study of the Fekete-Szegö func- tional and coefficient estimates forvsubclasses of analytic functions satisfying a certain subordination conditionv and associated with the Gegenbauer polynomials. AIMS Mathematics, 7(2):2568–2584, 2021. [15] Y.C. Kim J.H. Choi and T. Sugawa. A general approach to the Fekete-Szegö problem. Journal of the Mathematical Society of Japan, 59, 2007. [16] I. Naraniecka K. Kiepiela and J. Szynal. The Gegenbauer polynomials and typically real functions. Journal of Computational and Applied Mathematics, 153(1-2):273–282, 2003. [17] F.R. Keogh and E.P. Merkes. A Coefficient inequality for certain classes of analytic functions. Proceedings of the American Mathematical Society, 20, 1969. [18] A. Legendre. Recherches sur Laattraction des Sphéroides Homogénes; Mémoires Présentes par Divers Savants a laAcadémie des Sciences de laInstitut de France. Goethe Universitat, 10, 1785. [19] M. Lewin. On a coefficient problem for bi-univalent functions. Proceedings of the American Mathematical Society, 18(1):63–68, 1967. [20] H. Orhan M. Cağlar and M. Kamali. Fekete-Szegö problem for a subclass of analytic functions associated with Chebyshev polynomials. Bol. Soc. Paran. Mat., 40, 2022. [21] E. Deniz M. Kamali, M. Cağlar and M. Turabaev. Fekete Szegö problem for a new subclass of analytic functions satisfying subordinate condition associated with Cheby- shev polynomials. Turkish J. Math., 45, 2012. [22] N. Magesh and S. Bulut. Chebyshev polynomial coefficient estimates for a class of an- alytic bi-univalent functions related to pseudo-starlike functions. Afrika Matematika, 29(1-2):203–209, 2018. REFERENCES 115 [23] S. Miller and P. Mocabu. Differential Subordination: Theory and Applications. CRC Press, New York, 2000. [24] Z. Nehari. Conformal Mappings. McGraw-Hill, New York, 1952. [25] E. Netanyahu. The minimal distance of the image boundary from the origin and the second coefficient of a univalent function in |z| < 1. Archive for Rational Mechanics and Analysis, 32(2):100–112, 1969. [26] S. Ruscheweyh. New criteria for univalent functions. Proceedings of the American Mathematical Society, 49, 1975. [27] H.M. Srivastava and H.L. Manocha. A treatise on generating functions. Halsted Press, John Wiley and Sons, New York, Chichester, Brisbane and Toronto, 1984. [28] J. Szynal. An extension of typically real functions. Ann. Univ. Mariae Curie- Sko lodowska Sect., 48, 1994.