Current Active Research






Completed Research Studies


Twitter display example

There is an increasing amount of social media generated and tagged with the geographic location of its creation. A lot of research has focused at analysing and visualising this media, but very little has looked at  how to present it to users when those users are collocated in the physical location where that media was generated. How can media generated around the user’s current location better inform and engage the user with that location.

 PULSE attempts to investigate this by providing an ambient sonification, using both speech and non-speech audio, of twitter messages “tweets” recently generate around the user’s current location. Lab and in-situ studies showed that such presentation could be useful, providing advantages over visual interaction, as well as providing new insight into people and activities taking place in otherwise familiar locations.

This work extends my general research goal of applying multimodal interaction to more deeply engage users with their immediate environment, and other people within in. Questions raised by this work, regarding how users consider location when generating social media, are currently being investigated. 

PULSE is discussed in more detail in the following publications:

Virtual Excavator


Many historical sites of national and international importance are unstaffed, have few facilitates and little in the way of visual remains for visitors to see and understand.  Often these sites are located in rural locations, with limited access to communication infrastructure, and contribute little to the local economy in terms of visitors passing through.   The Virtual Excavator project is a first step in making these sites more accessible, understandable and attractive to visitors.  The overall goal is to produce a toolkit that will allow content owners, local museums or historical organisations to produce interactive collaborative experiences at these unstaffed sites and, as due to the limited resources such organisations have,  to do so at low financial cost.

The current stage of this work is developing a multimodal virtual living museum that can run on standard smartphone platforms and supports exploration of the site.  This provides auditory sound effects to contextualise the activities taking place in and around the remains of buildings (e.g. hammering, cooking, horses etc), virtual actors that represent personas of people who lived and worked at the site, as well as virtual artefacts that users can dig up using gestural interaction.  All of these are presented in a spatialised auditory environment, presented over headphones as users explore the site, which is 1:1 mapped to the real site.

So far, Virtual Excavator has been extensively evaluated, generating findings in how the visual remains influence site exploration and how collaboration develops amongst multiple users.  Current work is investigating how to build “content packs”, that will allow organisations to easily build experiences for their own sites. These can then be loaded into the app.  More details on current work in this area are available from the following publications:

  • McGookin, D., Vazquez-Alvarez, Y., Bergstrom-Lehtovirta, J., and Brewster, S., Shaking the Dead: Multimodal Location Based Experiences for Un-Stewarded Archaeological Sites. In proceedings of NordiCHI 2012 (Copenhagen, Denmark) ACM Press (2012) pp. 199-208
  • McGookin, D. Vazquez-Alvarez, Y., Brewster, S and Bergstrom-Lehtovirta, J Digitally Glueing the Past : Multimodal Interaction for Un-Stewarded Historical and Archaeological Sites In proceedings of Digital Economy 2011 (Newcastle, UK) 2011

IMG 0017IMG 0019IMG 0009Excavation viewExcavated find


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s