Academic Journal of Science and Technology ISSN: 2771-3032 | Vol. 10, No. 1, 2024 364 Multi‐label Feature Selection based on Label‐specific features and Manifold Learning Wanzhu Wang, Yong Liu Henan University of Science and Technology, 471000, Luoyang, China Abstract: Each instance in multi-label data is associated with multiple labels, and there are irrelevant or redundant features in its feature space, which leads to the performance degradation of multi-label learning algorithms. Multi-label feature selection selects representative features from the feature space to improve the accuracy of the model. Due to the high cost of labels and the difficulty of data collection, there will be some missing labels in the data set, which affects the accuracy of feature selection. To solve this problem, a multi-label feature selection algorithm based on label-specific features and manifold learning is proposed. The algorithm uses the linear relationship between the features and labels in known label samples to build a linear regression model for learning label-specific features. By using the nonlinear relation between instances and the nonlinear relation between features, we can precisely learn the label-specific features. We use the Laplacian feature mapping method to construct the instance manifold model and the feature manifold model, which are also used as the regular term constraint weight matrix. The final model can not only complete the missing labels, but also select sparse and representative features. The feature selection is carried out by analyzing the weight of the feature given by the final model. Experiments were conducted to verify the effectiveness of the proposed algorithm under different label deletion rates on four evaluation indexes. Keywords: Multi-label learning; Manifold learning; Label-specific features; Feature selection. 1. Introduction Multi-label learning is characterized by high efficiency and flexibility, and has shown wide application prospects in many application fields such as machine learning, data mining and pattern recognition [1-3], and is also widely used in various practical tasks, such as text classification, music classification and gene function prediction. For example, in the text category, a news story can be classified as both social and technological; In the music category, a piece of music may belong to both light music and English songs. Multiple concepts can be expressed by labeling each instance differently. However, in such cases where multi-label classification is required, attaching only one label to each instance does not express the underlying complex semantics. Therefore, multi-label learning is achieved by assigning multiple labels to an instance at the same time. Like single-label learning, the feature space of multi-label learning is often a set containing a large number of features, which may have up to thousands of dimensions [4]. In text classification, the complex semantic information of a text can be represented in detail by thousands of features. By extracting and utilizing these features effectively, we can understand the intrinsic meaning and context of the text more deeply. However, in the feature set, most of the features often present redundant or irrelevant features. These redundant and uncorrelated features not only increase the computational complexity, but also may negatively affect the performance of multi-label learning algorithms, and also lead to overfitting and other problems. In order to solve this problem, researchers proposed a multi-label feature selection algorithm, which is mainly used for dimensionality reduction of multi- label data. Literature [5] proposed a filter-based feature selection method for multi-label classification. Based on random forest as a learner, a filter feature selection method based on statistical integration method and label space division based on integration method were used. Literature [6] proposes a multi-label feature selection based on coarse- grained balls and label distribution. By using the pellet computing model and its proposed label enhancement method, logical labels are transformed into label distribution by exploring the similarities between instances and coarse- grained balls. The method measures feature significance by the consistency of labels in samples within the same information granularity. Based on mutual information theory, literature [7] designed a new multi-label feature selection method combining dynamic correlation change, label redundancy and interactive information. In the model training of multi-label feature selection algorithm, the correlation between labels is a key information, which can be effectively used to optimize the feature selection process. By considering the correlation between labels, the algorithm can more accurately capture the correlation and dependency between different labels, so as to select the features that are closely related to multiple labels. Literature [8] uses the co-occurrence relationship between labels to evaluate the similarity relationship between samples. According to literature [9], label correlation and label-specific features are two important features of multi-label learning. The global and local label correlation are calculated by label co-occurrence and neighborhood information respectively, and the label-specific features is learned by l_1 norm. Literature [10] considers the influence of two types of label dependencies on the algorithm. This method considers that second-order label dependencies and higher-order label dependencies are both important and complementary to capture label information, so second-order label dependencies and higher-order label dependencies are considered at the same time. Compared with traditional machine learning, multi-label labeling is very difficult. With the increasing complexity of multi-label tasks, there are also many challenges, among which the increase of sample dimension and data volume will significantly affect the labeling cost. First, because of the 365 large number of sample instances, the process of labeling them one by one is time-consuming and labor-intensive. Therefore, in the labeling process, manual labeling tends to select labels that are more important to label and discard other labels. The second is that human taggers can only label what they know according to cognitive differences. For example, an article is labeled machine learning, Data mining, and artificial neural networks, and the human tagger does not know about the category of artificial neural networks and only labels the article with the remaining two labels. A variety of circumstances have led to the creation of missing labels. However, the accuracy of the model or the efficiency of classification can be affected by the multi-label feature selection or multi-label classification on the multi-label data set with missing labels. In order to solve the impact of missing labels, many multi-label feature selection algorithms have been proposed to deal with missing labels based on theories such as multi-label information entropy [11], linear regression [12] and label correlation. He et al. [13] simultaneously considered label relevance, missing labels, and feature selection through a multi-classification framework. Literature [14] proposes an embedded packaging approach that combines the generation of label-specific features with subsequent model induction to solve the multi-label classification problem of missing labels. At the same time, it is more efficient to use the specific features of each label than to use all the features to classify. Label-specific features, refer to the subset of features most closely associated with each label. In the multi-label learning scenario, the use of label-specific features for feature selection can reveal the deep correlation between features and labels more efficiently. At the same time, from the point of view of the nonlinear relationship between data, consider the nonlinear relationship between instances and the nonlinear relationship between features, and study their influence on the relationship between labels and features. Based on the above considerations, this paper proposes a multi-label feature selection algorithm (ML-LSF) based on label-specific features and manifold learning that can deal with missing labels. 2. Proposed Models In multi-label learning, 𝑋 π‘₯ ,π‘₯ οΌŒβ€¦οΌŒπ‘₯ ∈ 𝑅 represents a multi-label data set with n instances, where d represents the feature dimension; π‘Œ 𝑦 οΌŒπ‘¦ οΌŒβ€¦οΌŒπ‘¦ ∈ 𝑅 represents the set of labels corresponding to n instances, where l represents the number of labels. The label values in set Y are set to 1, 0, and -1.When the label value is 1, the instance has the label. A label value of -1 indicates that the instance does not have the label. When the label value is 0, the instance is missing the label. 2.1. Build a label-specific feature model In a multi-label classification task, each label in the label space has the most relevant and discriminating features for the label, which are called label-specific feature. First, the basic label-specific feature model is constructed by linear regression method. The purpose of adding 𝑙 norm is to ensure that the features selected by each label are sparse and representative. The objective function is as follows: min 1 2 β€–π‘‹π‘Š π‘Œβ€– πœ†β€–π‘Šβ€– 1 Where X is the eigenmatrix of the instance, Y is the label matrix containing the missing label, and Ξ» is the parameter. W is the coefficient matrix, 𝑀 represents the i row in the matrix W, the row vector represents the feature vector, and the column vector represents the label vector. If the value of a certain place in W is 0, it means that there is no correlation between the feature and the label at that place. 2.2. Build the instance manifold and feature manifold models The core idea of manifold learning is that data in high- dimensional space is actually distributed on a low dimensional manifold structure, that is, although data may appear complex and difficult to analyze in the original high- dimensional space, their essential structure can often be characterized by a low dimensional manifold. The characteristic makes manifold learning suitable for handling nonlinear relationships between data. The goal of manifold learning is to effectively map points on manifold M in an N- dimensional space to a low dimensional n-dimensional space, where n