Achievements: A novel framework for human Activity Recognition (AR) is addressed, where a probabilistic ensemble of classifiers called Dynamic Mixture Model (DBMM) was proposed. The DBMM relies on the computed confidence belief from multiple base classifiers, combining these likelihoods into a single form by assigning weights from an uncertainty measure to counterbalance the posterior probability. Discriminative spatio-temporal features are extracted from human skeleton given RGB-D data. Assessment on well-known human daily activity datasets CAD60: Cornell Activity Dataset; Univeristy of Texas: UTKinect; Microsoft Research: MSR-Action3D and MSR-DailyActivity, and also using a mobile robot for assisted living were successfully carried out with overall accuracy greater than 90%. A real time application for AR including risk situations was implemented in ROS (Robot Operating System) useful for robot-assisted living.
Video 1: Human daily activity recognition for robot-assisted Living
Video 2: Activity recognition - anticipating human trajectory to avoid collision
General Objective: Interdisciplinary work involving experts from robotics and psychology applied to the context of child-robot interaction towards assisting the facilitation of adaptive health-related coping and improved quality of life outcomes in pediatric settings: robotherapy.
Achievements: We have started to program the humanoid robot NAO (Aldebaran robotics) to endow the robot with enough skills to interact with children. Initially, we have prepared a script to control the robot actions and reactions during the child-robot interaction online by tele-operation, where an expert in robotics selects appropriate reactions given the inputs and feedback of the child (talks and gestures), however, still with some autonomous decisions for reactions given the child feedback. Experiments with six children (e.g. boys and girls between 5 and 8 years old) were carried out. We followed the strategy of covering different types of interaction such as: verbal, gestural and physical interaction. These experiments were followed by a psychologist to assess the child's performance, reactions and acceptance of the child and their parents during the CRI with the NAO robot. Questionnaires were applied to the children’s parents to quantify the aforementioned parameters.
How can we endow an artificial system with appropriate cognitive skills (i.e., advanced perception capabilities) in order to grasp and manipulate everyday objects in the most autonomous and natural way possible?
In order to answer this question, this research was based on the fact that humans excel when dealing with everyday manipulation tasks, and are also able to learn new skills to adapt to different complex environments. Human abilities result from lifelong learning, and also from the observation of other skilled humans. To obtain similar dexterity with robotic hands, cognitive capacity is needed to deal with uncertainty. By extracting relevant multi-sensor information from the environment (objects), knowledge from previous grasping tasks can be generalized to be applied within different contexts. In examining this strategy, my research has shown that learning from human experiences is an alternative to accomplishing the goal of robot grasp synthesis for unknown objects. During my Ph.D. research different subtopics of the grasping area were studied; the interrelation between them is demonstrated in Figure 1. Applications for in-hand exploration of objects to represent the object shape by using a probabilistic volumetric map was proposed and also object identification by in-hand exploration using a probabilistic approach (Gaussian Mixture Models for learning and signatures acquired from Gaussian Mixture Regression). Task modeling (at trajectory level: 3D movements) and recognition were developed using Bayesian techniques.
Interrelation between grasp topics.
Some topics addressed in the past:
Publications (PDFs are available at publications page, accessing the main menu)
Diego R. Faria, Pedro Trindade, Jorge Lobo, Jorge Dias - "Knowledge-based Reasoning from Human Grasp Demonstrations for Robot Grasp Synthesis". RAS Journal, Elsevier - Robotics and Autonomous Systems, 2014.
Diego R. Faria, Ricardo Martins, Jorge Lobo, Jorge Dias. "Extracting Data from Human Manipulation of Objects Towards Improving Autonomous Robotic Grasping". Robotics and Autonomous Systems, Elsevier, Volume 60, Issue 3, Pages 396-410, March 2012.
Diego R. Faria, Jorge Lobo, Jorge Dias - "Identifying Objects from Hand Configurations during In-hand Exploration". In proceedings of the 2012 IEEE International Conference on Multisensor Fusion and Information Integration (IEEE MFI 2012), pp. 132-137, Hamburg, September, 2012.
Diego R. Faria, Ricardo Martins, Jorge Lobo, Jorge Dias. "A probabilistic framework to detect suitable grasping regions on objects". In 10th IFAC Symposium on robot Control (SYROCO'12), pp. 247-252, Dubrovinik, Croatia, Sept, 2012.
Ricardo Martins, Diego R. Faria, Jorge Dias. "Representation framework of perceived object softness characteristics for active robotic hand exploration". In Proceedings of 7th ACM/IEEE HRI'2012 - Workshop on Advances in Tactile Sensing and Touch based Human-Robot Interaction, Boston, USA, March 5-8, 2012.
Jafar Hosseini, Diego R. Faria, Jorge Lobo, Jorge Dias. "Probabilistic Classification of Grasping Behaviours using Visuo-hapitc Perception". In Proceeding of 3rd Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS'12), Technological Innovation for Value Creation, IFIP Advances in Information and Communication Technology Volume 372, pp. 241-248, Springer 2012.
Diego R. Faria, Ricardo Martins, Jorge Lobo, Jorge Dias. "Manipulative Tasks Identification by Learning and Generalizing Hand Motions". to appear in the Proceedings of DoCEIS'11 - 2nd Doctoral Conference on Computing, Electrical and Industrial Systems. Springer - Technological Innovation for Sustainability, IFIP Advances in Information and Communication Technology Volume 349, pp 173-180, Springer 2011.
Diego R. Faria, Ricardo Martins, Jorge Lobo, Jorge Dias. "Probabilistic Representation of 3D Object Shape by In-Hand Exploration". Proceedings of The 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS'10 - Taipei, Taiwan, pp. 1560 - 1565, October 2010.
Diego R. Faria, Ricardo Martins, Jorge Dias. "Learning Motion Patterns from Multiple Observations along the Action Phases of Manipulative Tasks". Proceedings of Workshop on Grasping Planning and Task Learning by Imitation: IEEE/RSJ IROS'2010, - Taipei, Taiwan - October 2010.
Ricardo Martins, Diego R. Faria, Jorge Dias. "Symbolic Level Generalization of In-hand Manipulation Tasks from Human Demonstrations using Tactile Data Information". Proceedings of Workshop on Grasping Planning and Task Learning by Imitation: IEEE/RSJ IROS'2010, - Taipei, Taiwan - October 2010.
Diego R. Faria, Ricardo Martins, Jorge Dias. "Grasp Exploration for 3D Object Shape Representation using Probabilistic Map". Proceedings of DoCEIS'10 - Doctoral Conference on Computing, Electrical and Industrial Systems. Emerging Trends in Technological Innovation, IFIP Advances in Information and Communication Technology Volume 314, pp 215-222, 2010. Springer - ISBN: 978-3-642-11627-8.
Diego R. Faria, Jorge Dias. "3D Hand Trajectory Segmentation by Curvatures and Hand Orientation for Classification through a Probabilistic Approach". In Proceedings of The 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS'09, St. Louis, MO, USA, pp 1284-1289 October 2009.
Diego R. Faria, Jose Prado, Paulo Drews Jr., Jorge Dias. "Object Shape Retrieval through Grasping Exploration". 4th European Conference on Mobile Robots, ECMR'09, Mlini/Dubrovnik, Croatia, pp.43-48, September 2009.
Diego R. Faria, Hadi Aliakbarpour, Jorge Dias, "Grasping Movements Recognition in 3D Space Using a Bayesian Approach", in Proceedings of ICAR'2009 - The 14th International Conference on Advanced Robotics - Munich, Germany, June 22-26, 2009. Print ISBN: 978-1-4244-4855-5
Diego R. Faria, Ricardo Martins, Jorge Dias, "Human reach-to-grasp generalization strategies: a Bayesian approach" - Workshop at Robotics Science and Systems 2009: "Understanding the Human Hand for Advancing Robotic Manipulation" - July 28, 2009 - Dillon Eng Seattle, WA, USA
Diego R. Faria, Jorge Dias, "Bayesian Techniques for Hand Trajectory Classification", RECPAD 2008 - 14th Portuguese Conference on Pattern Recognition, Coimbra-Portgal, 31st October, 2008.
Diego R. Faria, Jorge Dias, "Hand Trajectory Segmentation and Classification Using Bayesian Techniques", Workshop on "Grasp and Task Learning by Imitation", 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, Acropolis Convention Center, Nice, France Sept, 22-26, 2008, pp.44-49.
Ph.D. Thesis: Diego R. Faria, "Probabilistic Learning of Human Manipulation of Objects towards Autonomous Robotic Grasping". Ph.D. in Electrical and Computer Engineering. Faculty of Science and Technology, University of Coimbra, Portugal. The research was carried out at the Institute of Systems and Robotics, DEEC, University of Coimbra.
Work in Collaboration with Dr Cristiano Premebida
General Objectives: Address the problem of semantic place categorization in mobile robotics by considering a time-based probabilistic approach called Dynamic Bayesian Mixture Model (DBMM), which is an improved variation of the Dynamic Bayesian Network (DBN). More specifically, multi-class semantic classification is performed by a DBMM composed of a mixture of heterogeneous base classifiers, using geometrical features computed from 2D laser scanner data, where the sensor is mounted on-board a moving robot operating indoors. Besides its capability to combine different probabilistic classifiers, the DBMM approach also incorporates time-based (dynamic) inferences in the form of previous class-conditional probabilities and priors.
Achievements: Extensive experiments were carried out on publicly available benchmark datasets (Image Database for rObot Localization: IDOL and COLD Saarbrücken dataset, both were acquired using a mobile robot)highlighting the influence of the number of time-slices and the effect of additive smoothing on the classification performance of the proposed approach. Reported results, under different scenarios and conditions, show the effectiveness and competitive performance of the DBMM using different time slices. The accuracy of classification was measured by the Fmeasure (F1 score: using the precision and recall measures), and the overall performance was greater than 90%.
Illustrative representation of the DBMM approach with time-slices. The posterior depends on the priors P(Ck), the combined probabilities from the base-classifiers and the normalization factor (beta).
Results show the evolution of the Fmeasure per values of alpha and timeslices T = [0; ... ;4] for the four experiments on the IDOL dataset (see at publications). The curves on the graphic clearly demonstrate improvement on the performance of the DBMM when the ‘dynamic’ part is considered. -please check out the publications to see the results.
For More Details about this research and results check out the publications listed below.
C. Premebida, D. R. Faria, U. Nunes. "Dynamic Bayesian Network for Semantic Place Classification in Mobile Robotics", AURO Springer: Autonomous Robotics, 2015. (Under Review)
C. Premebida, D. R. Faria, F. A. de Souza, U. Nunes, "Applying Probabilistic Mixture Models to Semantic Place Classification in Mobile Robotics". Proceedings of IEEE IROS'15: IEEE International Conference on Intelligent Robots and Systems. Hamburg, Germany, 2015.