The purpose of this study is to establish the theoretical framework of cognitive visual informatics by proposing a theoretical framework that integrates the humanities and engineering approaches. For this purpose, this study used self-report measures ...
The purpose of this study is to establish the theoretical framework of cognitive visual informatics by proposing a theoretical framework that integrates the humanities and engineering approaches. For this purpose, this study used self-report measures (e.g., survey, interview) as well as EEG (electroencephalography) measures to better understand how people perceive, organize and evaluate visual information. This research used both measures because it is difficult to collect all related data through only self-report methods.
This study consists of nine chapters. Each chapter detailed important theories and methods after reviewing previous studies related to each chapter’s theme. And then case studies were reported in the end of each chapter (Chapter 3 ~ Chapter 9) to better understand theories and methods mentioned above. The case studies deals with our study results obtained through experiments and surveys.
This research described topical relevance, theoretical framework for visual Information recognition and processing, and narrative theory as key theories and then how to apply these theories to cognitive visual informatics in Chapter 1 (Introduction) and Chapter 2 (Theory). This study mentioned the research method focusing on physiological-psychological measurement methods and then reported an EEB-based relevance study as a case study in Chapter 3 (Research method).
This study described the visual information models proposed by Wang and Baddeley, respectively, along with Wilson’s information model and emotion models in Chapter 4 (Visual information models). Both visual information models explain how people perceive and recognize visual information. It also reported the case study on how their personality traits, which are related to the cognitive space of individuals, influence the information seeking behaviors of undergraduate students.
Whereas this study reported the previous studies on metadata for multimedia content (e.g., MPEG-7) and social metadata (e.g., tags) in Chapter 5 (Metadata). It also described the case study on investigating which metadata elements are suitable for the recycling of audiovisual materials.
This study reviewed the key theories and methods on video summarization including speech summaries and video storyboards in Chapter 6 (Visual information presentation and summarization). And then it described five speech summarization methods (e.g., social summarization, latent semantic analysis method, and acoustic method) and one key-frame extraction algorithm as case studies. Whereas this study reported the case study on EEG-based key-frame extraction algorithm in Chapter 7 (Video abstraction using EEG/ERP techniques for information browsing and retrieval).
The research mentioned the previous studies and trends on social information searches and described the experimental study on the efficiency of tag-based search query extensions as a case study in Chapter 8 (Social information search). The research reported the text- and content-based search methods for visual information and their performances in Chapter 9 (Evaluation). It also described two case studies on evaluating the social summarization algorithm mentioned in Chapter 6 and the EEG-based key-frame extraction algorithm described in Chapter 7.
This study can contribute in terms of academic and educational aspects and practical solutions as well. First, our research results can be used as educational materials for undergraduate (seniors) and graduate students who are majoring in library and information science, cognitive psychology, computer and information science, and visual communication. Our research results obtained from brain (EEG) studies can be used as basic data for brain science (integrated science), which in recent years has been actively studied in the natural sciences as well as humanities and social sciences.
Second, this study results can be utilized to facilitate access to and use of the visual information in digital libraries, or Web environments. That is, the research results obtained from the two EEG-based case studies reported in Chapter 3 and 7 can be applied to information retrieval systems. For instance, system interfaces for Internet search engines and digital libraries can be designed to automatically perform the following tasks (e.g., detailed record display and downloading) only by using a user’s relevance feedback obtained from EEG measurements while searching information. Furthermore, this study result might be utilized to automatically extract the key-frames from the video while a user equipped with an EEG device is watching a video.