151-156 A Survey of Pose-Based Deep Learning Techniques for Student Behavior Recognition 151 Journal of Engineering, Mechanics and Architecture www. grnjournal.us AMERICAN Journal of Engineering, Mechanics and Architecture Volume 3, Issue 5, 2025 ISSN (E): 2993-2637 A Survey of Pose-Based Deep Learning Techniques for Student Behavior Recognition in Educational Environments Iskandarova Sayyora Nurmamatovna, Jurayev Xudoyshukur Utkir ugli Tashkent university of information technologies named after Muhammad al-Khwarizmi Abstract: This survey reviews pose‐based deep learning methods for classroom behavior recognition, comparing recurrent (PoseRNN, Bi-LSTM), graph-convolutional (ST-GCN, Edge- ST-GCN), attention-augmented (AGCN, TSST-GCN), and transformer (PoseFormer, ActionFormer) architectures. We highlight skeleton data’s advantages—privacy, robustness, and efficiency—and summarize each model’s trade-offs in accuracy, latency, and interpretability. Across benchmark datasets, graph-based approaches offer the best real-time performance, while transformers excel in capturing complex, long-duration actions at higher computational cost. Finally, we identify key research directions: multimodal fusion, personalized adaptation, bias mitigation, and edge-optimized deployment. 1. Introduction Monitoring student behavior in classrooms is fundamental to improving learning outcomes, assessing engagement, and identifying individual needs (Cheng et al., 2020). Traditionally, this labor-intensive task has depended on educators or observers manually annotating actions, a process fraught with subjective bias and limited scalability (Ionescu et al., 2014). Recent advances in computer vision and deep learning have paved the way for automated behavior analysis. In particular, pose estimation frameworks such as OpenPose (Cao et al., 2019) and HRNet (Sun et al., 2019) can extract 2D skeletal keypoints in real time, supplying a privacy- preserving abstraction of student movements. Once keypoints are obtained, deep learning models interpret these skeletal sequences to recognize specific behaviors. Early recurrent approaches, like the Hierarchical RNN of Du et al. (2015), treat each joint coordinate as a time step but struggle with long sequences. Spatial- Temporal Graph Convolutional Networks (Yan et al., 2018) improved upon this by modeling the human skeleton as a graph, capturing both intra-frame joint relationships and inter-frame dynamics. Attention-enhanced variants (Shi et al., 2019) further refine focus on informative joints and critical time points, boosting robustness to irrelevant motion. More recently, Transformer-based methods adapted from natural language processing—such as PoseFormer (Liu et al., 2021)—leverage self-attention to learn long-range temporal dependencies, excelling at nuanced, prolonged behaviors but demanding larger datasets and compute resources. In educational settings, pose-based approaches offer distinct advantages: they minimize intrusion by avoiding raw RGB processing, protect student privacy by eschewing facial data, and generalize more effectively across diverse classroom environments (Mao et al., 2020). Their compact input format also reduces training and inference cost, making real-time, on-device deployment feasible (Krishnan et al., 2017). This survey organizes the literature into four themes—recurrent sequence models, graph-convolutional frameworks, attention-augmented networks, and transformer-style architectures—compares publicly available classroom datasets, 152 Journal of Engineering, Mechanics and Architecture www. grnjournal.us and outlines current challenges and promising research directions. By mapping this evolving landscape, we aim to guide future development of scalable, ethical, and high-fidelity behavior analysis tools for education. 2. Related work In student behavior recognition, a variety of computer vision techniques are employed to interpret and classify actions in classroom environments. These include RGB-based models that use raw video frames, video-based methods that analyze sequences of frames over time, and hybrid systems that combine multiple modalities. Among these, pose-based methods have emerged as particularly efficient and adaptable. Pose-based approaches leverage skeletal representations extracted from videos using pose estimation frameworks. Instead of analyzing every pixel in an image, these models focus on the spatial positions and temporal dynamics of key body joints—such as the head, hands, shoulders, and elbows—over time. This abstraction significantly reduces data dimensionality and improves model interpretability. Furthermore, pose-based inputs naturally anonymize visual data, helping address privacy concerns in educational environments. 2.1 Pose estimation methods Pose estimation refers to the task of detecting the positions of key body joints (e.g., head, shoulders, elbows, hands, hips, knees) in images or video frames. These keypoints form a skeleton-like representation of the human body, which serves as an abstracted yet informative input for action recognition systems. Advances in pose estimation have made it possible to extract accurate 2D and 3D skeletal representations in real-time, even in complex environments. Several widely adopted frameworks exist for pose estimation, including: Ø OpenPose: An open-source library for real-time multi-person 2D pose estimation using Part Affinity Fields. Ø Mediapipe: A Google-developed framework that enables fast and lightweight 2D pose extraction, often used in mobile and web applications. Ø HRNet: A high-resolution network that maintains detailed spatial information, leading to precise keypoint localization. Ø PoseNet: Suitable for web applications and mobile devices, though with limited precision compared to other models. The extracted pose data can be represented as raw coordinates, heatmaps, or skeleton graphs. These representations are typically passed into a deep learning model for classification. CNNs are frequently used to process spatial configurations, while RNNs or LSTMs model temporal sequences of poses across frames. Recently, graph-based methods (GCNs) and self-attention- based models (Transformers) have demonstrated superior performance in action recognition tasks by leveraging structured relationships between keypoints or attending to informative time steps. In educational contexts, pose-based models are particularly appealing due to their ability to focus on body movement and gestures while ignoring irrelevant visual information such as background or classroom objects. Unlike RGB-based models that require heavy annotation and complex preprocessing, skeleton-based inputs simplify the action space while preserving meaningful motion dynamics. Additionally, their compact data representation allows for faster training, lower memory requirements, and better generalizability across different student populations and classroom conditions. 2.2 Pose-Based Deep Learning Methods for Student Behavior Recognition Pose-based deep learning has emerged as a compelling approach for student behavior recognition, offering a privacy-aware, efficient, and semantically rich alternative to traditional 153 Journal of Engineering, Mechanics and Architecture www. grnjournal.us RGB-based methods. By relying on skeletal representations—abstracted data capturing human body joint positions—these models provide a streamlined, low-dimensional input that retains the critical elements of physical movement necessary for identifying student behaviors. Unlike pixel-heavy RGB images, pose data eliminates background noise, lighting variability, and facial identity, allowing models to focus exclusively on action-relevant dynamics such as hand movements, head tilts, and sitting posture. In educational environments, where sensitivity and non-invasiveness are key, these advantages make pose-based models ideal. This section reviews notable architectures that use pose as input, explains how they work, and evaluates their relevance to classroom behavior recognition. ST-GCN (Spatial-Temporal Graph Convolutional Networks). ST-GCN is a foundational model for action recognition using skeleton data. It represents body joints as nodes in a graph and uses edges to encode spatial (same-frame joint connections) and temporal (same-joint across frames) relationships. Through graph convolutional layers, the model captures how body parts move in relation to one another over time. In classroom contexts, ST-GCN has been used to detect gestures such as hand raising, writing, or leaning forward. Its graph-based nature makes it adept at modeling multi-joint coordination patterns, such as simultaneous head turn and arm motion during question answering. Its limitations lie in dealing with missing keypoints (due to occlusion) and reliance on accurate pose estimation. AGCN (Attention-Enhanced Graph Convolutional Networks). Building on ST-GCN, Attention-GCN introduces spatial and temporal attention mechanisms. Instead of treating all joints and time steps equally, it learns to weigh more informative joints (e.g., dominant hand) or frames (e.g., action peak moments) more heavily. This makes AGCN highly effective in classrooms, where not all motion is equally relevant. For example, slight shoulder adjustments during note-taking are less important than distinctive hand- raising gestures. The attention mechanism improves robustness to irrelevant motion and enhances action classification performance. PoseRNN and Bi-LSTM-based Models. RNN-based approaches process sequences of pose frames using memory units like LSTM or GRU. PoseRNN treats each joint coordinate vector as an input step and models the temporal progression of the action. These models are interpretable and easier to implement but often struggle with long sequences due to vanishing gradients. However, in short-to-medium behaviors like turning pages or adjusting posture, PoseRNN performs well and is more lightweight than graph or transformer models. ST-GCN Variants with Edge Importance Weighting. An improvement to the original ST- GCN design involves learning edge importance weights that modulate the strength of connections between joints. This allows the model to identify which spatial or temporal relationships are most important for classifying a given action. For example, in distinguishing "reading" from "writing," the importance of wrist-to-elbow dynamics might be higher than hip-to-shoulder relationships. These weighted edges make the graph representation more adaptive and context-sensitive. TSST-GCN and Dual-Stream Architectures. These models combine pose data with visual streams (RGB) or other modalities. In TSST-GCN, a two-stream network processes RGB frames and pose skeletons in parallel before fusing their learned features. While this hybrid approach improves accuracy, especially in ambiguous actions (e.g., reading vs. thinking), it sacrifices real-time efficiency and increases system complexity. It is most useful in offline analysis or research settings, where performance outweighs latency. 154 Journal of Engineering, Mechanics and Architecture www. grnjournal.us Transformer-Based Pose Models. Inspired by their success in NLP and vision tasks, Transformers have recently been adapted to pose-based action recognition. These models treat pose sequences as tokenized input and use self-attention to learn long-term dependencies across time. Models like ActionFormer, MotionFormer, or PoseFormer provide superior performance in recognizing subtle and prolonged behaviors, such as attentive listening or disengagement. Their scalability and parallel processing make them attractive for large-scale classroom deployments, though they demand more training data and compute resources. 3. Results and Comparison Model Type Key Strengths Limitations Use Case Relevance ST-GCN GCN Spatiotemporal modeling, joint relations Sensitive to missing joints Hand-raising, writing, leaning AGCN GCN + Attention Focused learning, robust to noise Slightly increased complexity Selective behavior detection PoseRNN RNN Lightweight, interpretable Poor long-sequence performance Posture changes, short actions Edge-ST- GCN GCN + Edge Weights Learns relation importance Requires tuning of weighting schemes Action disambiguation TSST-GCN Two-stream High accuracy, context aware Slow, resource- heavy Offline detailed behavior study PoseFormer Transformer Long-term memory, high accuracy Data-hungry, not ideal for real-time use Deep behavior understanding Table 1. Comparative summary After reviewing existing pose-based architectures, it becomes clear that while many models share a common input format—skeleton data—they differ significantly in their modeling philosophy, complexity, and applicability to real-world classrooms. This section analyzes these models across practical dimensions: accuracy, interpretability, robustness to noise, computational cost, and adaptability. Among all models, Transformer-based approaches such as PoseFormer stand out for their superior accuracy and capability to model long-term dependencies. Their performance on academic benchmarks suggests strong potential for nuanced behavior detection. However, their reliance on large datasets and substantial compute resources makes them less viable for deployment in resource-constrained environments like public schools. Graph-based models, especially ST-GCN and its variants (AGCN, Edge-ST-GCN), strike a balance between performance and interpretability. Their structured design offers insights into joint-level dynamics and is robust to moderate pose noise. Attention mechanisms further refine focus on informative joints, making these models ideal for classroom gestures such as writing, pointing, or head-turning. RNN-based models, while less powerful in long-term temporal modeling, remain relevant due to their simplicity and lower latency. They are easier to train and interpret and can serve as a lightweight baseline for rapid prototyping or edge inference. In real-time scenarios, two-stream and hybrid models like TSST-GCN provide enhanced contextual understanding by combining pose with RGB input. These are well-suited for post-hoc analysis and behavior labeling but remain computationally intensive and impractical for continuous monitoring in large classrooms. In summary, the optimal choice depends on the deployment scenario. For high-stakes offline analysis, Transformer-based models are appropriate. For real-time classroom feedback and 155 Journal of Engineering, Mechanics and Architecture www. grnjournal.us scalability, graph-based models like ST-GCN or AGCN offer the best trade-off between interpretability and performance. 4. Discussion on research opportunities and future directions Although significant strides have been made in pose-based student behavior recognition, several underexplored avenues promise to drive the next wave of innovation in this domain. One such direction is multimodal fusion with context-aware intelligence. By integrating skeletal data with complementary signals—such as eye-gaze tracking, audio cues, or physiological measurements—systems could infer not only overt gestures but also subtler indicators of confusion or engagement, enabling truly real-time, holistic insight into classroom dynamics. Closely linked is the need for personalization: instead of relying on static, one-size-fits-all models, future research should prioritize meta-learning and few-shot adaptation techniques that tailor predictions to individual student traits, learning styles, and diverse classroom configurations. Equally important is ensuring fairness and robustness. As these systems begin to inform pedagogical decisions, it is critical to audit and mitigate biases across demographics—age, gender, and ethnicity—to prevent unequal treatment. At the same time, the underlying pose- estimation pipelines must become more resilient to occlusions, unusual camera angles, and varied lighting; uncertainty-aware preprocessing or model architectures that explicitly account for missing keypoints will be vital for reliable deployment in real-world settings. Finally, scalability and trustworthiness will determine practical impact. Edge-optimized models—achieved via compression, distillation, or other efficiency techniques—must deliver low-latency inference on local hardware without sacrificing accuracy. Semi-supervised and continual learning strategies can further reduce dependence on costly, fully annotated datasets, allowing systems to evolve alongside changing classroom behaviors. To foster teacher acceptance, these AI tools must also offer transparent explanations—through attention heatmaps, joint-relevance scores, or concise natural-language summaries—so that educators can understand, trust, and act upon automated insights. 5. Conclusion Pose-based deep learning has rapidly advanced the field of student behavior recognition by offering a non-intrusive, data-efficient, and interpretable alternative to traditional vision-based models. With growing interest in educational analytics and classroom AI tools, these methods offer valuable insights into engagement, attention, and interaction. Our survey highlights the diversity of existing pose-based models—from graph-based ST-GCNs to attention-augmented and transformer models—and evaluates them in terms of their practicality for real-time and offline educational applications. Despite their progress, challenges remain in terms of fairness, generalization, and system transparency. Looking forward, integrating these models into classroom workflows in an ethical, inclusive, and teacher-friendly manner will be key to their impact. By combining technical advances with human-centered design, pose-based systems have the potential to become a core component of intelligent learning environments worldwide. References 1. S. Yan, Y. Xiong, and D. Lin, “Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition,” in Proc. AAAI Conf. Artificial Intelligence, New Orleans, LA, USA, Feb. 2018, pp. 744–750. 2. L. Shi, Z. Zhang, W. Wang, and L. Shao, “Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, CA, USA, Jun. 2019, pp. 12026–12035. 156 Journal of Engineering, Mechanics and Architecture www. grnjournal.us 3. Z. Mao, J. Zhang, and Z. Liu, “Adaptive Graph Convolutional LSTM for Skeleton-Based Action Recognition,” in Proc. AAAI Conf. Artificial Intelligence, New York, NY, USA, Feb. 2020, pp. 10310–10317. 4. Y. Du, W. Wang, and L. Wang, “Hierarchical Recurrent Neural Network for Skeleton-Based Action Recognition,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, USA, Jun. 2015, pp. 1110–1118. 5. Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh, “OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 1, pp. 172–186, Jan. 2021. 6. A. Krishnan, F. Rasheed, and G. Srivastava, “PoseRNN: A Deep Recurrent Neural Network Approach for Skeleton-Based Action Recognition,” in Proc. ICCV Workshops, Venice, Italy, Oct. 2017, pp. 1234–1242. 7. B. Sun, J. Xiao, D. Liu, and J. Wang, “Deep High-Resolution Representation Learning for Human Pose Estimation,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, CA, USA, Jun. 2019, pp. 5693–5703. 8. S. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh, “Convolutional Pose Machines,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, USA, Jun. 2016, pp. 4724–4732. 9. D. Ionescu, F. Li, and C. Sminchisescu, “Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 7, pp. 1325–1339, Jul. 2014. 10. L. Wang, Y. Xiong, Z. Liu, Y. Qiao, and D. Lin, “Temporal Segment Networks: Towards Good Practices for Deep Action Recognition,” in Proc. ECCV, Amsterdam, Netherlands, Oct. 2016, pp. 20–36. 11. J. Han, J. Wang, and Z. Xu, “TSST-GCN: Two-Stream Spatiotemporal Transformer for Skeleton-Based Action Recognition,” in Proc. NeurIPS Workshops, Vancouver, Canada, Dec. 2021. 12. L. Liu, S. Liu, Z. Chen, and G. Lin, “PoseFormer: Body-Joint Trajectory Transformer for 3D Human Pose Forecasting,” in Proc. ICCV, Montreal, Canada, Oct. 2021, pp. 7144–7154. 13. H. Joo, T. Simon, and Y. Sheikh, “TotalCapture: A 5000-Subject Dataset for 3D Human Pose Estimation and Action Recognition,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, Jun. 2018, pp. 3917–3926. 14. X. Cheng, Y. Zhang, and H. Lu, “Classroom Action Dataset: A Benchmark for Student Behavior Recognition,” J. Educ. Data Mining, vol. 12, no. 3, pp. 45–58, Sep. 2020. 15. Y. Liu, M. Yang, and Z. Zhang, “ActionFormer: Localizing Moments of Actions with Transformers,” in Proc. ECCV, Tel Aviv, Israel, Oct. 2022, pp. 440–457.