Michael Sülflow, Stefan Jarolimek & Pablo B. Jost (Johannes Gutenberg University, Mainz, Germany)
Summary of the Project
Moving images have become an integral part of various communication channels, such as social networks, video platforms or online news sites. Companies and political parties perceive the potential and produce appealing videos to present themselves, recruit new employees or members and gain support for their products, ideas or goals. The rising importance of audiovisual displays not only opens up new possibilities for attracting people’s awareness but also poses new challenges for social sciences. Research still lacks in-depth knowledge about perception and evaluation of moving images in particular. Although there is a bulk of research concerning the effects of visual stimuli, only analysis of eye movements provide results of actual attention distribution during the reception process. There is a long tradition of eye-tracking studies on static material, like newspaper reception (Holsanova et al., 2006), faces (Sekiguchi, 2011) or election posters (Geise, 2011). However, only few studies to date analyze moving images (e.g. Brasel & Gips, 2008; Scherer et al., 2012), as methodological challenges hindered yet new research impulses in this area (Papenmeier & Huff, 2010). However, eye-tracking only gives us information about where people look, but not about how they evaluate what they see. Often participants are interviewed after the reception of videos to detect media effects, however, follow-up interviews provide general impressions and hence cannot generate valid information about the effects of individual elements during the movie reception and are susceptible to several biases (e.g. forgetting information, primacy-recency effects). Therefore, we argue in this paper that in addition to content analysis and interviews, the combination of eye-tracking and Real-Time-Response Measurement (RTR) is required in order to gain valid insights into perception, processing and evaluation of moving visual stimuli. As the combination of these methods has not been done yet to our knowledge, our main approach is to examine the applicability and test the potential of this study design for further research.
This paper asks for methodological possibilities in analyzing effects of moving images by applying eye-tracking and RTR Measurement. We consider and reflect possible mutual interferences of the two methods as well as methodological challenges of participants’ stress situations and reactivity. More specific, the question arises whether participants feel subjectively overstrained by simultaneously evaluating moving images (RTR), being eyetracked as well as following a task allocation. Furthermore, we want to test possible interactions between the handling of an RTR-dial, gaze behavior and task performance.
RQ1: Do participants feel subjectively overstrained by simultaneously evaluating moving images (RTR), being eyetracked and following a task allocation?
RQ2: Does the combination and possible interaction of different real-time-measurement methods (RTR, eye-tracking) and task allocation cause differences in gaze behavior between groups?
Awareness has long been the subject of debate in mass communication research (e.g. McQuail, 2010) as it constitutes the basis for subsequent cognitive processing. The observation method of eye movement analysis is able to measure awareness distribution by number and duration of image content fixations (Duchowski, 2007). However, eye movement analysis as a single method does not allow valid assumptions about perception of content. RTR Measurement enables researchers to gain insight into spontaneous impression formation (Maier et al., 2006). It can be applied to test the effects of different kind of information, such as nonverbal and verbal cues (Nagel et al., 2012). But literature review shows that the use of an RTR-dial might have influence on the attention and the evaluation process itself (Fahr & Fahr, 2009). Besides, this method allows no assumptions about where people actually look at during the evaluating process. Consequently, it is not possible to draw conclusions about which specific elements of an audiovisual dynamic stimuli cause different evaluations. Therefore, a combination of these two methods is recommended to fully assess individual processing of audiovisual contents. The synchronization of eye-tracking and RTR closes these important research gaps. It is assumed that the combination of awareness and evaluation (RTR) can be used to show (at least short term) effects.
In an experiment with 32 participants we analyzed the perception and evaluation of two videos from strategic communication. The experiment was performed in December 2014 at Johannes Gutenberg University Mainz. In order to prove the combination of methods for different types of organizations we chose one video from corporate communication and one from the area of political communication. The selection of realistic stimuli material (videos from youtube) enhances the external validity and thereby the transferability of our study results. The two videos resembled each other regarding certain recurring elements, such as speaking persons. Eye-tracking was applied to all participants, half of them using RTR-dials simultaneously. Besides analysis of an “undirected” reception situation, the influence of specific tasks was tested (e.g. Birmingham et al., 2007). In our study, participants with task were requested to pay attention to arguments and information in the stimuli. This constitutes a setting with four different experimental groups (2×2 design, see tab. 1). For valid interpretations these methods were coupled with preliminary and follow-up interviews to extract effects and influencing factors (e.g. attitudes towards companies/parties), as well as with a second-by-second content analysis (Nagel et al., 2012).
|With task||Without task|
|Eye-Tracking & RTR||8||8|
Tab. 1: Experimental Design (Variation of independent variables)
In general, our first results indicate that the combination of eye-tracking and RTR Measurement works and offers new methodological opportunities to analyze perception, effect and evaluation of moving images. This result is also supported by self-disclosures afterwards, as the participants in general had no problems handling the RTR-dial “blindly” while watching the stimuli. However, participants with task allocation felt significantly more distracted from the videos by using RTR-dials. Taking a closer look at the RTR-results, participants with task (high involvement) evaluate contents significantly more often and more intensively than the group without task. Concerning gaze behavior, the task has no influence. Total fixation duration of different areas of interests (AOIs) differs across participants with and without RTR-dials. But participants using RTR-dials stay longer on the first dynamic area of interest of a scene. For instance, participants with RTR-dials fixate even longer faces of protagonists at the beginning of a scene, instead of being distracted by elements and settings that direct the gaze. We interpret this not as a result of the combination of methods but rather that the employment of RTR-dials seems to modify gaze behavior. It can be assumed that persons and faces in particular serve as information shortcuts during the formation of opinions.
Birmingham, E., Bischoff, W., & Kingstone, A. (2007). Why do we look at eyes? Journal of Eye Movement Research, 1(1):1, 1-6.
Brasel, S. A., & Gips, J. (2008). Points of view: Where do we look when we watch TV? Perception 37(12), 1890–1894.
Duchowski, A. (2007). Eye Tracking Methodology: Theory and Practice. New York: Springer.
Fahr, A., & Fahr, A. (2009). Reactivity of Real-Time Response Measurement: The influence of employing RTR techniques on processing media content. In Maier, J., Maier, M., Meyer, V., Maurer, M., & Reinemann, C. (Eds.), Real Time-Response Measurement in the Social Sciences (pp. 45-62). Frankfurt/M.: Peter Lang.
Geise, S. (2011). Vision that matters: Die Funktions- und Wirkungslogik Visueller Politischer Kommunikation am Beispiel des Wahlplakats. Wiesbaden: VS Verlag für Sozialwissenschaften.
Holsanova, J., Rahm, H., & Holmqvist, K. (2006). Entry points and reading paths on newspaper spreads: comparing a semiotic analysis with eye-tracking measurements. Visual Communication, 5(1), 65-93.
Maier, J., Maurer, M., Reinemann, C., & Faas, T. (2006). Reliability and validity of real-time response measurement. A comparison of two studies of a televised debate in Germany. International Journal of Public Opinion Research, 19, 153-173.
McQuail, D. (2010). McQuail’s Mass Communication Theory. London: Sage.
Nagel, F., Maurer, M., & Reinemann, C. (2012). Is There a Visual Dominance in Political Communication? How Verbal, Visual, and Vocal Communication Shape Viewers’ Impressions of Political Candidates. Journal of Communication, 62, 833–850.
Papenmeier, F., & Huff, M. (2010). DynAOI: A tool for matching eye-movement data with dynamic areas of interest in animations and movies. Behavior Research Methods, 42(1), 179-187.
Scherer, S., Layher, G., Kane, J., Neumann, H., & Campbell, N. (2012). An audiovisual political speech analysis incorporating eye-tracking and perception data. Paper presented at International Conference on Language Resources and Evaluation, Istanbul, Turkey (23.-25. May 2012).
Sekiguchi, T. (2011). Individual differences in face memory and eye fixation patterns during face learning. Acta Psychologica, 137, 1–9.
The combination of two real-time-measurement methods (eye-tracking, RTR) and preliminary/follow-up interviews allows valid assumptions about the interaction of awareness and effects of audiovisual information and therefore is not susceptible to certain biases of one-method-designs, such as recency-effects in follow-up interviews. In this paper we focus on the applicability of the combination of methods and its potential. Therefore, results concerning the attention distribution on the content and evoked judgment processes were not included. However, the bulk of data that is generated by combination of four different methods enables complex analysis of attention-grabbing elements, their composition and effects of different settings, persons or stylistic devices within the video material. We are convinced that the combination of these methods can be applied to various fields of communication research to gain insights into perception, processing and evaluation of moving visual stimuli and thus derive conclusions about the effects of presented information.
The application of four methods generates large amounts of data – especially when analyzing moving images – and this poses challenges for the researcher. Concerning the synchronization, analysis and interpretation of various data, a solid theoretical framework and specific research questions are essential. Synchronization of RTR-data with eye-tracking data is difficult due to delayed reaction time of participants while using the dial. Therefore, a time interval between seeing and evaluating an information has to be determined and taken into account.
Results of eye-tracking research do very much depend on the stimuli presented. In our study the videos consisted of a high number of cuts and unique elements that may have guided the gaze of participants (ET) on the one hand and on the other hand may have complicated judgment formation (RTR). Furthermore, the time of reaction while using the RTR-dial depends on the cutting speed and the complexity of the information (e.g. TV debate vs. TV ad)
Due to a relative low number of participants advanced statistical analysis and comparisons are not appropriate. However, recruitment of participants and execution of the experiment is very time-consuming and we argue that inclusion of 32 subjects allows us to gain first insights into potentials of the presented study design. Furthermore, our analysis only allows assumptions about short-term effects triggered by the stimuli. But in our opinion, results concerning attention-grabbing elements (e.g. faces) and the applicability of the methods can be generalized.