attention & running commentaries in unknown environments

Fiora Pirri, Sapienza, University of Rome, Italy

We concentrate on attention in very large and essentially not known a priori environments, where the subject might also have several difficulties in moving within it, such as rubble, pits, collapsed stairs, and always unexpected scenarios/situations. In such environments, we study attention with running commentaries, namely how visual attention couples with linguistic observations. For example the subject wearing the gaze machine utters the word referring to what she/he thinks is an observed object and the fixation algorithm would tell whether the observation coincide with a fixation or not (this includes the subject observing her/his hands or foot, as this is often the case in such difficult environments). To obtain this parallel construction the gaze machine is registered not only to ensure corrected projection of gaze in the scene, but also to ensure contemporaneity with utterance. It turns out that this requires a specific short denotation for observations, in order to ensure that the running commentaries are uttered at approximately the same speed of scene acquisition (20/30 Hz) while pupil frame rate acquisition is about (100 Hz). The running commentary returns the effort of recognition, in an unknown environment and also pseudo motivations for large saccades.