Being There - immersive documentaries

Is critical immersion possible? Media archeologist and art historian Erkki Huhtamo of UCLA fame thinks that the suspension of disbelief required for deep and complete immersion works well for fantasy stories such as Avatar or Alice in Wonderland, but not necessarily for creating engagement with pressing real world issues. On a visit to Hong Kong, Huhtamo says:” I’m not sure if we need technical immersion to engage with social  and political issues. If the content is compelling, seeing a frame is not a problem.’ In order to activate the viewer, we need to attract the viewer’s attention to reality, he says, but we also need a distancing act to induce reflection on that reality.  This is a viewpoint shared by many media artists and critical filmmakers of my generation (and older): alienation - not immersion - leads to awareness, reflection and, ultimately, social change.

As an example of a succesful immersive media work with political content, Huhtamo mentions ‘World Skin’  (1997) by French artists Maurice Benayoun and Jean-Baptiste Barriere, who used 3D CAVE technology (three wall video projection), to present a kind of  battle field animation based on 2D newspaper cut outs of war scenes. Standing in the middle of the CAVE, the viewer is invited to act as a photojournalist, taking pictures of the cruelties surrounding him. Huhtamo particularly likes the fact that the immersive effects produced by CAVE are counterbalanced by the flatness and ‘unreality’ of the war photographs in the animation, which he assumes will stimulate the viewer-turned-war photographer to reflect on the madness of war.

 

Unfortunately, I have not seen World Skin myself, but it seems to be a very succesful art work. The impact of the installation, however, is limited to museum visitors who have already decided to be thoughtful, attentive and probably critical. I am interested in finding out how immersion in everyday media can be compelling (young) viewers/users to think about reality in unexpected ways and put their minds in motion. 

Huhtamo points out that the artificiality - which is always part of the act of total, technical immersion - may be counterproductive for relating to real life horrors. For example, would it really be more moving to see refugees struggling on rafts to reach our shores in stereoscopic 3D with special glasses than to see them on TV in the grainy images shot on a mobile phone ? Probably not. As with the CNN footage of the Haiti disaster, it could be turning a disturbing reality into a kind of reassuring fantasy,  farther removed from us mentally although we may feel temporarily immersed in it physically.

Still, the way media are fusing with everyday life and practically all social relations, I wonder if we can rest assured on the effectiveness of the alienation-thesis. After all, the only time my students really woke up this semester is when I presented them examples of interactive documentaries, gaming and non-fiction stories in Second Life. Huhtamo’s students in LA are also constantly connected and can not live without checking with their friends on their mobiles every fifteen minutes. With documentary (and news) increasingly perceived as ‘boring’ and newspapers on the way out for most people aged under twenty five, I really doubt that there will be many opportunities for succesful and productive ‘alienation’. Getting attention, by using methods that are somehow, in some way  ’immersive’ and pull you viewers into a non-fiction story seems to me an urgent and challenging task for journalist and storytellers.  What we need though is good, independent, immersive examples.

I’m not at all convinced that immersion itself excludes raising the kind of social and political awareness, which leads to action. After all, my own most radical immersive experience this fall has been reading - or rather ‘deep reading’ as Huhtamo calls it appealingly - ‘La Carte et le Territoire’, the new novel by Michel Houellebecq. It sucked me in, I could not put it down, it put my mind into motion and it probably is influencing my acts right now. So if immersion in a novel can change my mind, why coudn’t full immersion in a non-fiction film do the same?

Thanks to Digital Magic, a state-of-the-art film and video studio in Hong Kong, I had the pleasure of watching the full uncensored trailer of ‘Sex and Zen in 3D’ in excellent conditions . The original ‘Sex and Zen’ (traditional Chinese: 玉蒲團之偷情寶鑑, Yù pútuán zhī tōuqíng bǎojiàn) is a 1991 Hong Kong erotic comedy film directed by Michael Mak and starring Amy Yip and Lawrence Ng. The film is loosely based on The Carnal Prayer Mat, an infamous Chinese erotic novel by 17th century author and playwright Li Yu.

This 3D sequel (there has been a 2D remake already) is advertised as ‘the first 3D porn movie in the world’. It’s being produced in Hong Kong for a 3 million USD budget and eagerly awaited by a certain part of the public, thanks to an extensive marketing campaign. I was wondering what the porn industry was waiting for, as they are usually among the early adopters for new technologies. ‘Sex and Zen in 3D’ is not only a feast of flesh, but also a pixel party, since it’s shot in 3D and 4K (the highest resolution currently available for movie production). Suffice to say that watching the 5 minute trailer is an immersive and breathtaking experience. This is certainly a very convincing showcase of technological advances and digital immersive storytelling potential. Nipples were jumping at me, as hands reached out to grab me and every imaginable body part seemed to come very close to my face. Luckily,  I was wearing the 3D glasses.

The trailer on youtube is censored (and 2D, of course):

But if you want to see the uncensored trailer, you can at this website, but you have to enter your age first. The first reviews are out.

It is easy to see that porn consumers will be a prime market for new 3D content, since the added value of 3D for porn by far exceeds that of sports, dance or animation in my estimation. It’s one thing to see a football coming right at you, but an open mouth or a penis (not in the trailer) is something else. Now that the sales of 3D televisions is lagging behind, because of insufficient attractive content among other reasons, this is probably the type of content that may help get the new tv sets into the living rooms (or bedrooms, rather).

This is what CNN did in terms of ‘immersive journalism’, using the Google street view camera to take viewers into ravaged Haiti. The uncut footage provides a 360 look on the human disaster, filmed from the roof of a driving car. For me, browsing these interactive videos definitely provokes a new and strong sense of ‘being there’ - but painfully so. Being confronted with the surround view, I dearly miss ‘perspective’. Who is looking through the camera? Who is looking at whom? What for? Why am I here? Gazing around from the top of a car makes me feel like a voyeur or a tourist, rather than a connected and empathic citizen. Here, the interactivity and 360 do not produce a sense of connection, but rather a confrontation with a reality that I am clearly not a part of, because I can manipulate it at will. Unlike the persons shown in the video. A lot of pixels and little storytelling.

This is just a Youtube rendering, but look on the CNN website for the real deal, with full control over your point of view. It’s a rather alienating way to relate to the world, if you ask me. Ffrom the CNN website: “Haiti: 360. Use your mouse to click and drag around the video to change the view. You can also zoom in and out. Pause and explore at any time by pressing the play/pause button under the video to stop and look around. The video was shot on Monday, January 18, at 9:52 a.m. EST in Port-au-Prince, Haiti.” I wish the Haitians in the picture were able to ‘pause and explore’ their world ‘ at any time.

My most intense encounter with immersive documentary so far involved a face-to-face meeting with cosmonaut Yuri Gagarin, the Russian hero and man of my dreams, who happened to be the first human being to travel in space and orbit the earth, five years before I was born. The meeting with the space traveller took place in Macau, one hour by ferry from Hong Kong in the impressive and somewhat eery Macau Science Center (designed by Japanese architect I.M.Pei, of the Louvre pyramid). The building is located on a huge lot of empty and probably very expensive land adjacent to the Macau-Hong Kong-China ferry terminal, facing the sea on one side and the giant casinos of Asia’s gambling metropolis Macau on the other.
In December 2009, it was opened officially in the presence of no one less than Hu Jintao, president of the People’s Republic of China. In his speech, Hu said that “the Center showcases a number of exhibits featuring current technologies, as well as exhibits and models, where young people can learn about the remarkable progress and achievements in aerospace technology development made by our home country. The Center shall become the hub of science education in Macao, and an important platform for civil education on patriotism.”

In addition to a patriotic celebration of Chinese achievements in aerospace technology and robotics, the building features a Planetarium with one of the most advanced 3D/360o movie theaters in the world, being the first one with a 8000 x 8000 pixel resolution. It has a tilted semi-dome screen of 15 meters in diameter, supported by high resolution 3D digital projectors, 135 seats equipped with interactive controls for a choice of language (Cantonese, Mandarin, English and Portugese are offered) and so-called ‘4D’, meaning that the chairs rock and tilt in line with the story on the screen to increase the immersive experience for viewers.

Although I entered the dome sceptically, I was totally won over by the sense of Yuri’s presence and the truly immersive experience of the ‘Dawn of he Space Age’ film. For the first time, I felt that I was part of a documentary story that was not just a demonstration of visual effects and entertainment, but a technologically supported invitation to re-live a very important episode of our history: the race between Russia and the US to conquer space. It still takes special glasses to see 3D, but the powerful database coupled with projection effects takes viewers on something like a journey traveling freely into space.
The fact that this race was actually run by men, rather than machines, made it compelling and emotionally engaging for me, and the 3D-storytellers had managed to keep just enough of the human aspects of the story to connect me to the space race as a major political event. Although framed here in the chauvinistic and official discourse of China’s conquests beyond the earth, this is definitely the kind of new storytelling that I am looking for. As I watched the credits roll, the film turned out to be made in The Netherlands. Now, I will continue looking for the first 3D Indie films….

Just imagine a 1000 m2 black box  filled with stereoscopic 3D machines, interactive video and 360 projections, located in the science fiction like setting of the brand new Hong Kong Science Park in Shatin. This is where Philips also recently moved its Consumer Electronics division and this is where City University of Hong Kong opened the spectacular ALiVE lab last summer. ALiVE stands for ‘Applied Laboratory for Interactive Visualization and Embodiment’ and is the baby of director and dean of the School of Creative Media Jeffrey Shaw and research director Sarah Kenderdine. Both have extensive experience with immersive installations as art works and museum displays and more recently they have ventured into 3D stereoscopic representations of cultural heritage sites such as Hampi in India. There are three installations which have possible uses as documentary storytelling machines.

The first one is the iDome, a half round screen which accommodates content shot with a 360o camera like the one used for Google street view. In this iDome, one viewer can manipulate a mouse to see a 360o film from all angles. The current content is limited and shows mostly ‘place’ (e.g. a street in Australia and a mosque in Istanbul)  and very little ’story’. It seems to me that storytelling could be introduced most convincingly by constructing a sound scape of interviews and location sounds recorded separately. Visually, I could imagine experiments in observational, ‘fly-on-the-wall’ style documentary if the scene is extremely interesting and happening in one room (e.g. a court room, an interrogation room, an IC unit at the hospital, a class room challenging the teacher). Shooting and editing are still extremely limted with the 360 camera (called ‘Ladybug‘), because rendering takes about one hour per minute and footage takes up enormous amounts of memory.

In the ‘Media Dome’, viewers can lie on their backs and watch stereoscopic photos projected onto the domed ceiling, so this limits the non-fiction storytelling potential to anything taking place in the sky. The media dome has a high ‘lounge-feel’ to it and could potentially work with a documentary story taking place in the top of a tree for example. I saw it with a slide show of photos of ornate ceilings from all over the world (and without spoken words).

Thirdly, T_visionarium is a wonderful and extremely engaging cylinder of 12 meter diameter based on six powerful 3D video projections, which can host around 24,000 video clips at once and has the most powerful storytelling potential. In terms of content, it originally is an archive of random clips from 24 hours of Australian television, which could be replaced by any other type of content that is properly clipped and tagged to fit the software. The effect on a viewer of this machine is truly impressive and ‘immersive’ and invites further thinking about the development of database-stories. For example, the Asia Art Archive in Hong Kong has an extensive collection of video interviews with artists and critics about the recent history of Cantonese art, which could potentially be transformed into a mesmerizing interactive and alternative history of contemporary Cantonese art as told by its creators. The drawbacks are the production cost and the complexity of the installation. It takes considerable man power to cut tens of thousands of clips and tag them manually, which is needed to enable viewer interaction and meaningful rearrangements of the clips in the space.  However, if this can be mastered and overcome, T-visionarium is much better than imax and could potentially host emotionally engaging and interactive documentary storytelling.