Mostrar el registro sencillo del ítem

Human-Computer Interface Based on Facial Gestures Oriented to WhatsApp for Persons with Upper-Limb Motor Impairments

Interfaz humano-computador basada en gestos faciales y orientada a la aplicación WhatsApp para personas con limitación motriz de miembros superiores

dc.creatorFerrín-Bolaños, Carlos
dc.creatorMosquera-DeLaCruz, José
dc.creatorPino-Murcia, John
dc.creatorMoctezuma-Ruiz, Luis
dc.creatorBurgos-Martínez, Jonathan
dc.creatorAragón-Valencia, Luis
dc.creatorLoaiza-Correa, Humberto
dc.date2021-01-30
dc.identifierhttps://revistas.itm.edu.co/index.php/tecnologicas/article/view/1722
dc.identifier10.22430/22565337.1722
dc.descriptionPeople with reduced upper-limb mobility depend mainly on facial gestures to communicate with the world; nonetheless, current facial gesture-based interfaces do not take into account the reduction in mobility that most people with motor limitations experience during recovery periods. This study presents an alternative to overcome this limitation, a human-computer interface based on computer vision techniques over two types of images: images of the user’s face captured by a webcam and screenshots of a desktop application running on the foreground. The first type is used to detect, track, and estimate gestures, facial patterns in order to move and execute commands with the cursor, while the second one is used to ensure that the cursor moves to specific interaction areas of the desktop application. The interface was fully programmed in Python 3.6 using open source libraries and runs in the background in Windows operating systems. The performance of the interface was evaluated with videos of people using four interaction commands in WhatsApp Desktop. We conclude that the interface can operate with various types of lighting, backgrounds, camera distances, body postures, and movement speeds; and the location and size of the WhatsApp window does not affect its effectiveness. The interface operates at a speed of 1 Hz and uses 35 % of the capacity a desktop computer with an Intel Core i5 processor and 1.5 GB of RAM for its execution; therefore, this solution can be implemented in ordinary, low-end personal computers.en-US
dc.descriptionEn el caso de personas con limitación motriz de miembros superiores, los gestos faciales son la principal forma de comunicarse con el mundo. Sin embargo, las interfaces actuales basadas en gestos no tienen en cuenta la reducción de movilidad que la mayoría de las personas con limitación motriz experimentan durante sus periodos de recuperación. Como alternativa para superar esta limitación, se presenta una interfaz humana-computador basada en técnicas de visión por computador sobre dos tipos de imagen: la imagen del rostro capturada mediante webcam y la captura de pantalla de una aplicación de escritorio en primer plano. La primera imagen es utilizada para detectar, seguir y estimar la pose del rostro con el fin de desplazar y ejecutar comandos con el cursor; la segunda imagen es utilizada para lograr que los desplazamientos del cursor sean realizados a zonas específicas de interacción de la aplicación de escritorio. La interfaz es programada totalmente en Python 3.6 utilizando bibliotecas de código abierto y se ejecuta en segundo plano dentro del sistema operativo Windows. El desempeño de la interfaz se evalúa con videos de personas utilizando cuatro comandos de interacción con la aplicación WhatsApp versión de escritorio. Se encontró que la interfaz puede operar con varios tipos de iluminación, fondos, distancias a la cámara, posturas y velocidades de movimiento; la ubicación y el tamaño de la ventana de WhatsApp no afecta la efectividad de la interfaz. La interfaz opera a una velocidad de 1 Hz y utiliza el 35 % de la capacidad de un procesador Intel Core i5 y 1,5 GB de RAM para su ejecución lo que permite concebir esta solución en equipos de cómputo personales.es-ES
dc.formatapplication/pdf
dc.formattext/xml
dc.formattext/html
dc.formatapplication/zip
dc.languagespa
dc.publisherInstituto Tecnológico Metropolitano (ITM)en-US
dc.relationhttps://revistas.itm.edu.co/index.php/tecnologicas/article/view/1722/1812
dc.relationhttps://revistas.itm.edu.co/index.php/tecnologicas/article/view/1722/1818
dc.relationhttps://revistas.itm.edu.co/index.php/tecnologicas/article/view/1722/1829
dc.relationhttps://revistas.itm.edu.co/index.php/tecnologicas/article/view/1722/1925
dc.relation/*ref*/J. H. Mosquera-DeLaCruz; H. Loaiza-Correa; S. E. Nope-Rodríguez; A. D. Restrepo-Giró, “Human-computer multimodal interface to internet navigation,” Disabil. Rehabil. Assist. Technol., pp. 1–14, Jul. 2020. https://doi.org/10.1080/17483107.2020.1799440
dc.relation/*ref*/C. Ferrin-Bolaños; H. Loaiza-Correa; J. Pierre-Diaz; P. Vélez-Ángel, “Evaluación del aporte de la covarianza de las señales electroencefalográficas a las interfaces cerebro-computador de imaginación motora para pacientes con lesiones de médula espinal,” TecnoLógicas, vol. 22, no. 46, pp. 213–231, Sep.. 2019. https://doi.org/https://doi.org/10.22430/22565337.1392
dc.relation/*ref*/Ministerio de Salud y Protección Social Oficina de Promoción Social de Colombia, “Sala Situacional Situación de las Personas con Discapacidad,” Junio. 2018. https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/PS/sala-situacional-discapacidad-junio-2018.pdf
dc.relation/*ref*/L. Cortés-Rico; G. Piedrahita-Solórzano, “Interacciones basadas en gestos: revisión crítica,” TecnoLógicas, vol. 22, pp. 119–132, Dic. 2019, https://doi.org/10.22430/22565337.1512
dc.relation/*ref*/N. Balsero; D. Botero; J. Zuluaga; C. Parra Rodríguez, “Interacción hombre-máquina usando gestos manuales en texto real,” Ing. y Univ., vol. 9, no. 2, pp. 101–112, 2005. https://www.javeriana.edu.co/Facultades/ingenieria/revista/DefRev0902Interaccion.pdf
dc.relation/*ref*/W. A. Castrillón Herrera, “Implementación de una Interfaz Hombre-Máquina para el Control de un Brazo Robótico Mediante Posturas Labiales,” (Trabajo de Grado), Universidad Nacional de Colombia, Manizales, 2009. https://repositorio.unal.edu.co/bitstream/handle/unal/7090/williamalfredocastrillonh.2009.pdf?sequence=1&isAllowed=y
dc.relation/*ref*/J. H. Mosquera; H. Loaiza; S. E. Nope; A. D. Restrepo, “Identifying facial gestures to emulate a mouse: navigation application on Facebook.,” IEEE Lat. Am. Trans., vol. 15, no. 1, pp. 121–128, Jan. 2017, https://doi.org/10.1109/TLA.2017.7827915
dc.relation/*ref*/C. Mauri, T. Granollers; J. Lorés; M. García “Computer vision interaction for people with severe movement restriction,” An Interdiscip. J. Humans ICT Environ., vol. 2, pp. 38–54, Apr. 2006. https://core.ac.uk/download/pdf/70291032.pdf
dc.relation/*ref*/E. Perini; S. Soria; A. Prati; R. Cucchiara, “FaceMouse: A Human-Computer Interface for Tetraplegic People,” in Lecture Notes in Computer Science, Springer-Verlag Berlin Heidelberg, pp. 99–108, 2006. https://aimagelab.ing.unimore.it/imagelab/pubblicazioni/Published_LNCS.pdf
dc.relation/*ref*/J. Varona; C. Manresa-Yee; F. J. Perales, “Hands-free vision-based interface for computer accessibility,” J. Netw. Comput. Appl., vol. 31, no. 4, pp. 357–374, Nov. 2008, https://doi.org/10.1016/j.jnca.2008.03.003
dc.relation/*ref*/M. Betke; J. Gips; P. Fleming, “The Camera Mouse: visual tracking of body features to provide computer access for people with severe disabilities,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 10, no. 1, pp. 1–10, Mar. 2002. https://doi.org/10.1109/TNSRE.2002.1021581
dc.relation/*ref*/S. S. Khan; M. S. H. Sunny; M. S. Hossain; E. Hossain; M. Ahmad, “Nose tracking cursor control for the people with disabilities: An improved HCI,” en 2017 3rd International Conference on Electrical Information and Communication Technology (EICT), Khulna 2017, pp. 1–5. https://doi.org/10.1109/EICT.2017.8275178
dc.relation/*ref*/A. Matos; V. Filipe; P. Couto, “Human-computer interaction based on facial expression recognition: A case study in degenerative neuromuscular disease,” ACM Int. Conf. Proceeding Ser., pp. 8–12, Dec. 2016. https://doi.org/10.1145/3019943.3019945
dc.relation/*ref*/A. Rabhi; A. Sadiq; A. Mouloudi, “Face tracking: state of the art,” in 2015 Third World Conference on Complex Systems (WCCS), Marrakech. 2015, pp. 1–8. https://doi.org/10.1109/ICoCS.2015.7483308
dc.relation/*ref*/P. Premaratne, Human Computer Interaction Using Hand Gestures. Singapore: Springer Singapore, 2014.
dc.relation/*ref*/L. Nanni; S. Brahnam; A. Lumini, “Face Detection Ensemble with Methods Using Depth Information to Filter False Positives,” Sensors, vol. 19, no. 23, p. 5242, Nov. 2019, https://doi.org/10.3390/s19235242
dc.relation/*ref*/M. W. Ni, “Facial image registration,” (Tesis Doctoral), Electrotechnique, Automatique et ´Traitement du Signal, l’universite de Grenoble, 2017. https://tel.archives-ouvertes.fr/tel-01557731/document
dc.relation/*ref*/M. H. Teja, “Real-time live face detection using face template matching and DCT energy analysis,” in 2011 International Conference of Soft Computing and Pattern Recognition (SoCPaR), Dalian. 2011, pp. 342–346. https://doi.org/10.1109/SoCPaR.2011.6089267
dc.relation/*ref*/A. Aldhahab; T. Alobaidi; A. Q. Althahab; W. B. Mikhael, “Applying Multiresolution Analysis to Vector Quantization Features for Face Recognition,” in 2019 IEEE 62nd International Midwest Symposium on Circuits and Systems (MWSCAS), Dallas 2019, pp. 598–601. https://doi.org/10.1109/MWSCAS.2019.8885188
dc.relation/*ref*/S. Zafeiriou; C. Zhang; Z. Zhang, “A survey on face detection in the wild: Past, present and future,” Comput. Vis. Image Underst., vol. 138, pp. 1–24, Sep. 2015. https://doi.org/10.1016/j.cviu.2015.03.015
dc.relation/*ref*/A. Kumar; A. Kaur; M. Kumar, “Face detection techniques: a review,” Artif. Intell. Rev., vol. 52, pp. 927–948, Agu.2019. https://doi.org/10.1007/s10462-018-9650-2
dc.relation/*ref*/F. Pujol; M. Pujol; A. Jimeno-Morenilla; M. Pujol, “Face Detection Based on Skin Color Segmentation Using Fuzzy Entropy,” Entropy, vol. 19, no. 1, p. 26, Jan. 2017. https://doi.org/10.3390/e19010026
dc.relation/*ref*/E. Perini; S. Soria; A. Prati; R. Cucchiara, “FaceMouse: A human-computer interface for tetraplegic people,” Lect. Notes Comput. Sci, Berlin, 2006, pp. 99–108. https://doi.org/10.1007/11754336_10
dc.relation/*ref*/R. Brunelli, Template Matching Techniques in Computer Vision: Theory and Practice. JohnWiley & sons ltda. 2009
dc.relation/*ref*/V. S. R. Middi, K. J. Thomas; T. A. Harris, “Facial Keypoint Detection Using Deep Learning and Computer Vision,” Springer International Publishing, 2020, pp. 493–502. https://doi.org/10.1007/978-3-030-16660-1_48
dc.relation/*ref*/A. Divya; K. B. Raja; K. R. Venugopal, “Face Recognition Based on Windowing Technique Using DCT, Average Covariance and Artificial Neural Network,” in 2018 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), Bangkok 2018, pp. 335–342. https://doi.org/10.1109/ICIIBMS.2018.8549981
dc.relation/*ref*/J. Huang; Y. Shang; H. Chen, “Improved Viola-Jones face detection algorithm based on HoloLens,” Eurasip J. Image Video Process., vol. 2019, no. 1, 2019. https://doi.org/10.1186/s13640-019-0435-6
dc.relation/*ref*/Y. Freund; R. E. Schapire, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,” J. Comput. Syst. Sci., vol. 55, no. 1, pp. 119–139, Aug. 1997, https://doi.org/10.1006/jcss.1997.1504
dc.relation/*ref*/I. Culjak; D. Abram; T. Pribanic; H. Dzapo; M. Cifrek M, “A brief introduction to OpenCV,” en Proceedings of the 35th International Convention MIPRO, Opatija 2012. https://ieeexplore.ieee.org/document/6240859
dc.relation/*ref*/J. H. Mosquera; E. Oliveros, “Interacción Humano-Máquina Audiovisual,” (Trabajo de grado), Universidad del Valle, Santiago de Cali, 2011. https://bibliotecadigital.univalle.edu.co/bitstream/handle/10893/8979/CB-0441157.pdf;jsessionid=7DE19B6C73D98F390B5D6A9B0C55D34A?sequence=1
dc.relation/*ref*/V. Londoño-Osorio; J. Marín-Pineda; E. I. Arango-Zuluaga, “Introducción a la Visión Artificial mediante Prácticas de Laboratorio Diseñadas en Matlab,” TecnoLógicas, edición especial, pp. 591-603, Nov. 2013. https://doi.org/10.22430/22565337.350
dc.relation/*ref*/C. Sagonas; G. Tzimiropoulos; S. Zafeiriou; M. Pantic, “300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge,” in 2013 IEEE International Conference on Computer Vision Workshops, 2013, pp. 397–403. https://doi.org/10.1109/ICCVW.2013.59
dc.relation/*ref*/X. Ren; J. Ding; J. Sun; Q. Sui, “Face modeling process based on Dlib,” in 2017 Chinese Automation Congress (CAC), Jinan 2017, pp. 1969–1972. https://doi.org/10.1109/CAC.2017.8243093
dc.relation/*ref*/A. Sweigart, “Welcome to PyAutoGUI’s documentation!,” Read the Docs, 2020. https://pyautogui.readthedocs.io/en/latest/
dc.relation/*ref*/C. Ferrin; J. Pino, “hciVisualGesture,” 2020. https://github.com/cdfbdex/hciVisualGesture
dc.relation/*ref*/ISO 9241-940:2017, Ergonomics of human-system interaction - Part 940:Evaluation of tactile and haptic interactions. Switzerland, 2017. https://www.iso.org/standard/61362.html
dc.relation/*ref*/Z. Ali; S. B. Bhaskar, “Basic statistical tools in research and data analysis,” Indian J. Anaesth., vol. 60, no. 9, pp. 662-669, 2016. https://doi.org/10.4103/0019-5049.190623
dc.rightsCopyright (c) 2020 TecnoLógicasen-US
dc.rightshttp://creativecommons.org/licenses/by-nc-sa/4.0en-US
dc.sourceTecnoLógicas; Vol. 24 No. 50 (2021); e1722en-US
dc.sourceTecnoLógicas; Vol. 24 Núm. 50 (2021); e1722es-ES
dc.source2256-5337
dc.source0123-7799
dc.subjectHuman-computer interfaceen-US
dc.subjectface detectionen-US
dc.subjectcomputer visionen-US
dc.subjectassistive technologyen-US
dc.subjectInterfaz humano-computadores-ES
dc.subjectdetección de rostroses-ES
dc.subjectvisión por computadores-ES
dc.subjecttecnología de asistenciaes-ES
dc.titleHuman-Computer Interface Based on Facial Gestures Oriented to WhatsApp for Persons with Upper-Limb Motor Impairmentsen-US
dc.titleInterfaz humano-computador basada en gestos faciales y orientada a la aplicación WhatsApp para personas con limitación motriz de miembros superioreses-ES
dc.typeinfo:eu-repo/semantics/article
dc.typeinfo:eu-repo/semantics/publishedVersion
dc.typeResearch Papersen-US
dc.typeArtículos de investigaciónes-ES


Ficheros en el ítem

FicherosTamañoFormatoVer

No hay ficheros asociados a este ítem.

Este ítem aparece en la(s) siguiente(s) colección(ones)

Mostrar el registro sencillo del ítem