№1, 2023

Kamala Sh. Gurbanova, Fargana J. Abdullayeva

The dynamic development of computing techniques and communication tools and the improvement of network technology have increased the role of information as the main resource in society. The application of information communication technologies has stimulated the development of intellectual and scientific potential all over the world and has been successfully applied to all fields. Gestures are the only means of communication for hearing and speech impaired people. Automatic recognition of gestures in order to facilitate communication through gestures is an urgent issue from both scientific and practical point of view. This article highlights the static and dynamic gestures. The process of gesture recognition collects data by means of various sensor technologies. The article analyzes the image-based and non-image-based technologies and presents their advantages and disadvantages. It also comparatively analyzes the working principle of existing methods proposed for the gestures identification, explores their advantages and disadvantages and interprets the performance of the software which localizes the hand showing the gesture in the video frame. As a result, it develops a machine learning method based on neural networks for high accuracy identification of gestures. The developed method testing on database open for research shows high performance (pp.43-52).

Keywords:Sign language, Dynamic gestures, Gesture recording technologies, Hidden Markov model, Artificial neural network method, Color segmentation algorithm, Viola-Jones algorithm
  • World Hearing Day on March 3, 2022 will be held under the slogan “To hear all your life, take care of your hearing!”. 25 February 2022, https://kurgan-med.ru/news/vsemirnyy-den-slukha-3-marta-2022-goda-proydet-pod-lozungom-chtoby-slyshat-vsyu-zhizn-beregite-svoy-/
  • Yeo, H.S., Lee, B.G., Lim, H. (2015) Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware. Multimedia Tools and Applications, 74(8), 2687-2715.
  • Prozorova, E.V. (2007) Russian sign language as a subject of linguistic research. Questions of Linguistics, 1, 44–61. (Russian)
  • Kataev, M.Yu., Shirokov, L.V. (2013) Method for determining hand gestures observed with a video camera. Reports of TUSUR, 1(27), 45-49. (Russian)
  • Sign Language Lessons. 23 November 2022, https://www.pinterest.com/pin/742460688548321728/?mt=login
  • Muratov, E.R., Nikiforov, M.B., Skachkov, A.M. (2020). Artificial intelligence in the tasks of recognizing dynamic gestures. Modern information technologies and IT education, 16(4), 883-892. (Russian)
  • Bauer, A.V. (2015). Real-time gesture recognition implementation. Microsoft Technologies in the Theory and Practice of Programming: Proceedings of the 12th All-Russian Scientific and Practical Conference of Students, Postgraduates and Young Scientist, Tomsk, March 25-26, (pp. 56-58) (Russian)
  • Gesture recognition for interaction with AI: from theory to the latest achievements. 23 November 2022, integral-russia.ru
  • Abdullayeva, F.J., Gurbanova, K.S. (2022). Sign Language Hand Gesture Recognition Method based on Machine Learning, The 16th IEEE International Conference on Application of Information and Communication Technologies, Washington DC, 12-14 october 2022, 1-5.
  • Nguyen, T.T., Bolotova, Yu.A., Spitsyn, V.G. (2012). Real-time processing of video sequence data based on a hierarchical time network. Scientific Bulletin of NSTU, 2(47),33-43. (Russian)
  • Chudnovsky, M.M. (2014). An algorithm for recognizing human hand gestures on a video sequence in real time for the implementation of human-machine interaction interfaces. Siberian Journal of Science and Technology, 3(55), 162-167. (Russian)
  • Muratov, E.R., Nikiforov, M.B., Skachkov, A.M. (2020). Artificial intelligence in the tasks of recognizing dynamic gestures. Modern information technologies and IT education, 16(4), 883-892. (Russian)
  • Chuikov, A.V., Vulfin, A.M. (2017). Gesture recognition system based on neural network Technologies. Bulletin of Ugatu, 3 (77), 113–122. (Russian)
  • Rustamov, G.A. (2012). Automatic regulation theory. Textbook for higher technical schools, 729. (Azerbaijan)
  • Potopakhin, A.A. (2017). Analysis of software tools and methods for dynamic gestures recognition. Computer science, control systems, mathematical and computer modeling: Proceedings of the 8th International Scientific and Technical Conference within the framework of the Third International Scientific Forum of the Donetsk People’s Republic, Donetsk, May 25, 2, (pp.416-419) (Russian)
  • Yu, S. (2010). Hidden semi-Markov models. Artificial Intelligence, 174, 215-243.
  • Bauer, A.V. (2014). Dynamic Hand Gesture Recognition Based on Hidden Markov Models. Modern Techniques and Technologies: Proceedings of the 10th International Scientific and Practical Conference of Students, Postgraduate Students and Young Scientists, Tomsk, April 14-18, 2, (pp. 145-146) (Russian)
  • Gefke, D.A., Zatsepin, P.M. (2012). Application of hidden Markov models for recognition of sound sequences. Scientific journal “Izvestiya ASU”, 2(73), 72-76. (Russian)
  • Nikolaev, E.I., Zakharov, V.V., Zakharova, N.I. (2017). Technology for training deep models on a synthesized data set for recognition and detection of gestures. Vestnik SevKavGTI, 4, 237-240. (Russian)
  • Dolgov, G.A. (2021). Choosing the optimal neural network architecture for gesture recognition. MNSK-2021: Proceedings of the 59th International Scientific Student Conference, Novosibirsk, April 12–23, (pp. 87) (Russian)
  • Matveev, A.S., Shestakova, V.V., Prokhorov, A.V. (2018). Tasks of professional orientation in the context of changing the educational system of undergraduate studies on the example of Tomsk Polytechnic University. Power industry through the eyes of youth: Proceedings of the 9th International Youth Scientific and Technical Conference, Kazan, 1, (pp. 69–72) (Russian)
  • Arindam, M., Abe, T., Takayuki, O., Koichiro, D. (2011). Hand Gesture Recognition using Histogram of Oriented Gradients and Partial Least Squares Regression. MVA 2011 IAPR Conference on Machine Vision Applications, Nara, Japan, June 13-15, (pp. 479-482)
  • Fan, N.Kh., Buy, T.T., Spitsyn, V.G. (2013). Real-time gesture recognition on a video sequence based on the application of the Viola-Jones method, the Camshift algorithm, the Wavelet transform and the principal component method. Tomsk State University Bulletin, Management, Computer Engineering and Informatics, 2(23), 102-111. ( Russian)
  • Yang, J., Xu, Y. (1994). Hidden Markov Model for Gesture. Recognition: Technical Report CMURI-TR-94-10. – Pittsburgh, PA: Carnegie Mellon University, Robotics Institute, 27.
  • Nosov, A.V. (2019). Algorithm for localizing an image of a hand from a video sequence.REDS: Telecommunication Devices and Systems, 9(1), 22-26. (Russian)
  • Favorskaya, M.N., Zotin, A.G. (2012). Methods for motion estimation in video sequences and their software implementation. Textbook allowance; Sib. state aerospace un-t. Krasnoyarsk, 132. (Russian)
  • Bolotova, Yu.A., Fedotova, L.S., Spitsyn, V.G. (2014). Algorithm for detecting areas of faces and hands in an image based on the Viola-Jones method and the color segmentation algorithm. Fundamental Research, 11(10), 2130-2134. (Russian)
  • Kozyr, M.V. (2017). Algorithm for detecting areas of faces and hands in an image based on the Viola-Jones method. Proceedings of the Second Regional Scientific and Technical Conference, Omsk, June 06–07, (pp. 26-30) (Russian)
  • ASL Alphabet: Image data set for alphabets in the American Sign Language. 23 November 2022, https://www.kaggle.com/datasets/grassknoted/asl-alphabet