Supervision : Bernadette BOUCHON-MEUNIER

Co-supervision : RIFQI Maria

Modeling Affective Systems through Physiology and their Application in Design of Video Games

As emotions play a significant role in normal human relations, there is need to develop methodologies for assessing user's emotional experience while interacting with computers. More especially, it has a particular interest in the field of gameplay technology. Especially in the recent years, video games have enjoyed a dramatic increase in popularity, not only as a form of entertainment but also as one of the most powerful educative channels of our times both at home and formal education systems. Indeed, video games are now part of society culture and its impact to form and in uence the future generations cannot be ignored. With this ourishing technological and theoretical efforts, there is urgent need to think of new paradigms to optimize user's positive experiences during these human computer interactions. Yet, quantifying such experiences is venture that is clouded with ambiguity. In this context, among a vast range of possible ways to access a user's emotional responses such as facial gesture or voice recognition, physiological measures have a key advantage as they grant an access to non conscious and non reportable processes. In fact, it is very inter- esting that physiological recordings provide an insight into human feelings that the subjects themselves may not even be consciously aware of. Nevertheless, to map physiological pat- terns to user psychological states still remains an extremely intricate task, rising critical questions of viability of such paradigms as real-life emotionally intelligent applications. In this work, we addressed the possibility of developing an adequate model to tackle some of these fundamental issues.
Consequently, we conducted experiments in which we recorded physiological measures on players involved in an action game with an aim of discovering typical physiological signatures associated with various gaming experiences. Foremost, in order to guarantee the viability of the developed models in real life application, efforts were put to ensure that the experimental setup was as close to normal human situations as possible. Subsequently, we employ a machine learning model that is best suited for this peculiar task. While some of the current models may have proved to be eifficient in classification (discriminating cognitive states), they do not seem robust enough to tackle the problem of psychophysiological characterization i.e the induced knowledge is often difficult to exploit. In this work, to exploit the experimental results, we consider two frameworks: machine learning by fuzzy decision trees, and drawing from prototype-based reasoning approaches we formulate a framework that employs typicality degrees to discover pertinent psychophys- iological characteristics. Each of these frameworks has its peculiar properties appropriate to bring out the typical psychophysiological relations. In actual fact, these two approaches can be compared in two main ways. One, as characterization tools: while cardinal properties can be summarized by the fuzzy prototypes thanks to typicality degrees, decision trees also represent induced knowledge in a very explicitly understandable rules. Two, as in the case typicality-based fuzzy prototypes, the induced knowledge in fuzzy decision trees is modeled in a fuzzy sets based context to represent continuous transitions.
To sum it up, as a real life application, we aim to enhance the user's experiences with video games through a continuous renewal of the player's interest by adapting to user's cognitive states. We frame this in two main experimental studies, one, to extensively study characterization properties of various physiological features in relation to cognitive states, and another, to test the constructed framework by assessing player's psychological states in relation to various game events. Thanks to our framework, we developed a fuzzy psychophysiological controller that was able to correctly recognize different psychological levels of the player's enjoyment. We found our study worthy for a real life application in true affective systems, able to continuously measure the user's cognitive states.

2009-2019 Publications

• 2019
  • D. Machanje, J. Orero, Ch. Marsala : “Distress Recognition from Speech Analysis: A Pairwise Association Rules-Based Approach”, IEEE Symposium Series on Computational Intelligence (SSCI) - Computational Intelligence for Engineering Solutions (CIES), Xiamen, China (2019)
• 2018
  • D. Machanje, J. Orero, Ch. Marsala : “A 2D-Approach Towards the Detection of Distress Using Fuzzy K-Nearest Neighbor”, Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations. IPMU 2018, vol. 853, Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations, Cadix, Spain, pp. 762-773, (Springer International Publishing) (2018)
• 2011
  • J. Orero : “Modélisation de systèmes émotionnels à partir de signaux physiologiques et application dans la conception de jeux vidéo”, thesis, phd defence 05/31/2011, supervision Bouchon-meunier, Bernadette, co-supervision : Rifqi, Maria (2011)
• 2010
  • F. Levillain, J. Orero, M. Rifqi, B. Bouchon‑Meunier : “Characterizing Player’s Experience From Physiological Signals Using Fuzzy Decision Trees”, CIG 2010 - IEEE Symposium on Computational Intelligence and Games, Copenhagen, Denmark, pp. 75-82, (IEEE) (2010)
  • J. Orero, F. Levillain, M. Damez‑Fontaine, M. Rifqi, B. Bouchon‑Meunier : “ASSESSING GAMEPLAY EMOTIONS FROM PHYSIOLOGICAL SIGNALS: A FUZZY DECISION TREES BASED MODEL”, INTERNATIONAL CONFERENCE ON KANSEI ENGINEERING AND EMOTION RESEARCH 2010, Paris, France, pp. 1684-1693 (2010)
• 2009
  • Th. Baccino, Ch. Tijus, F. Jouen, F. Levillain, F. Lallemand, M. Damez, B. Bouchon‑Meunier, M.‑J. Lesot, J. Orero, M. Rifqi : “Couplage de données multisources à l’oculomotricité”, Journée de l’oculomotricité, Paris, France (2009)