Using Advene to accompany research in AudioVisual Digital Humanities
Olivier Aubert - www.olivieraubert.net
FOSDEM 2020 - 01/02/2020
Plan
- The Advene project
- Use cases in Digital Humanities
- REMIND method
- AdA project
- Conclusion
Advene project
Annotate Digital Video, Exchange on the NEt
to be added to something or become a part of it, though inessential (Webster, 1913)
- Accompany active reading of AV documents through annotations
- Create, user, share analyses of AV documents as hypervideos
- Started in 2002 with Yannick Prié and P.A. Champin
- Free Software (GPL) cross-platform desktop application (Python/Gtk/Gstreamer)
- www.advene.org
Interface
Principle
Scientific challenges
- Knowledge engineering
- Document engineering
- HCI
- Dataviz
- Activity traces
Features
- User defined structures and visualizations (templates, rules…)
- Preserve structured annotations as long as possible in the workflow
- A package is a set of (annotations, annotation structure, views, queries) independent from the video
- Generic data model. Use of text-based microsyntax.
- import filters
- no strict constraints
- Flexible and extensible (through templates and plugins)
- Offline work
2 use cases in Digital Humanities
- Interview analysis in museology : REMIND
- Systematic quantitative analysis and semantic annotation in media studies : AdA project
REMIND method
Goal: study museum visitors' perception through video-based autoconfrontation
- Work by Daniel Schmitt (Univ. Valenciennes)
- Self-explicitation of visitors through replaying subjective recording of the visit
- Online publication of produced visualizations and raw data
Example: Museographie
Analysis process
- Transcribe interview (note-taking view)
- Identify hexadic signs (from course-of-action methodology)
- Use relations to express courses of experience
- Generate visualizations through templates
- Write analyzes
- Publish on the web
REMIND method
Exploratory structuration / Publication of analyses as hypervideos
AdA project
www.ada.cinepoetics.fu-berlin.de
- Affektrhetoriken des Audiovisuellen - Audio-visual rhetorics of affect
- Systematic-empirical study of staging patterns in audiovisual representations of the 2007 financial crisis
- Interdisciplinary collaboration: Cinepoietics (Media Studies, Freie Universität Berlin), HPI Potsdam (semantic web, video analysis)
Challenges and needs
Challenges
- Developing a methodological framework for analysis, with quantitative support
- Annotation content semantization
- Exploration of automation possibilities
Needs
- Build a ground truth for future automation
- Optimize the manual annotation process
- Provide a bridge between users and semantic data
Application
Process
- Exploratory modelling of an ontology through Advene schemas definition
- Conversion of initial Advene schemas into bootstrap ontology
- Conversion of OWL ontology into Advene schema
- convert properties (URIs) into "local" vocabularies (folksonomies)
- preserve URI/vocabulary mapping through metadata
- Non-blocking data validation feedback
- Export of information into an extended WebAnnotation model
Principle
Produced ontology (HPI)
- 8 annotation levels: Segmentation, Camera, Montage, Image Composition, Acoustics, Language, Bodily expressivity, Motifs
- 78 annotation types
- (currently) 86000 annotations over 5 movies and many TV reports
- CC-SA 3.0
Data visualization
Advene developments for the AdA project 1/2
HCI
- Update code for adapting to new systems (esp. Python3/Gtk3/Gstreamer1)
- Performance improvements
- Constraint vocabularies support
- Data visualization
Advene developments for the AdA project 2/2
- Improvements of fusion/import features for collaboration
- Semantic web interoperability
- Ontology import
- RDF export (WebAnnotation + extensions)
- Integration of a concept-detection server from HPI
Conclusion
- Advene - a useful tool for AV Digital Humanities project
- Able to export/import from multiple other models/formats
- Flexible and extensible
- Supports the whole scholar workflow from exploration to publication
- Available support for custom developments and training - contact me
- Many ideas, looking for funding
Thanks for you attention
Any question?
Appendix: Active Reading
1
Using Advene to accompany research in AudioVisual Digital Humanities
Sorry, your browser does not support SVG.
Olivier Aubert - www.olivieraubert.net
FOSDEM 2020 - 01/02/2020