Will Pappenheimer

Associate Professor, Pace University

Tamiko Thiel

Independent Artist

After Deleuze & Guattari, A Thousand Plateaus, Introduction, page 23 (our ‘re-assembly’ in bold):

“There is no longer a tripartite division between a field of reality (the world) and a field of representation (the Augmented Reality artwork) and a field of subjectivity (the author). Rather, an assemblage establishes connections between certain multiplicities drawn from each of these orders …”

Introduction: Augmented Reality art as “software assemblage”

Inspired by the writings of Rewa Wright, “Mobile Augmented Reality Art and the Politics of Re-assembly,” this paper expands on her characterization of our augmented reality artwork as “software assemblage.” [1] Wright’s application of Deleuze and Guattari’s assemblage theories analyzes how augmented reality (AR) artworks function as assemblages of process, participant, location and meaning. [2] Though Deleuze and Guattari’s work, A Thousand Plateaus, is the foundation of Wright’s thesis, this presentation does not follow a careful application of their combined theory, but rather explores how it informs our work. [3]

Wright proposes a specific formation of Deleuze and Guattari’s machinic assemblage as “software assemblage,” operative in location-based artworks such as mobile AR. This model describes software assemblage as a conjunction or configuration of technologies, media and software that, together with their geographical and social contexts and implications, are actively instantiated by the user/participant at a corporeal and perceptual level. Because these works are participatory, ambulatory and site-based, they emerge out of a fluid experience, configured by us as artists and but only “assembled” in their final form through the agency of viewers on location.

A broad outline of this process includes the notions that mobile AR, as it is perceived through portable media devices, brings together procedural coding, network identity, social connectivity and bodily motions that incorporate the user into the technologies involved, or vice versa, the technologies are incorporated in the user. Inherent in geo-locative AR technology is the assumption that active bodily engagement by the user/participants is required to view virtual objects and media. We liken them to birdwatchers, as they must physically scan the environment with their mobiles to locate the artwork, twisting and turning, looking up and down, walking towards distant artworks to finally stand inside them. Their mobile devices function as handheld virtual magnifying glasses or telescopes, allowing glimpses into previously imperceptible strata. In the process, users of AR must negotiate the shared space with other people, vehicles, furnishings, buildings and spaces, in an encounter with not just the artwork as media but with an assemblage of multiple cultural, social, physical and territorial matrices.

Wright suggests that geo-locative AR artworks, as intervening or juxta-positional imagery, serve as transformative or critical collage and commentary on the multiple contexts that the site inhabits, revealing influences including but not limited to politics of institutional space, contemporary world events and cultural memory. In this already complex assemblage, user/participants are active agents, bringing their own perceptions and contextual references to the multiplicities of allusions composed by the AR artist.
Layers of signs and signifieds at a site are often stratified into hierarchies of importance, with the physical, the written, the official and the real, as determined and controlled by the dominant culture, having precedence over the intangible, invisible, colloquial, minority or subaltern views of history. We however concur with Deleuze and Guattari’s rhizomic, non-hierarchical structures that proclaim a level overlapping playing field: where a GPS topology has the same importance as physical geography, where mediation has an equivalent affect to appearance and where software is an integral part of physical bodies and systems. We see AR not as representation or simulation, as virtual versus physical, but rather as action upon or transformation of pre-existing schemes of representation, in order to shift or undermine them. The AR artwork is immaterial but nonetheless real in the dictionary sense of “actually existing or happening, not imaginary,” being “fixed, permanent, immovable” with an “objective independent existence” as it can be called up by anyone with the proper hardware and software at any time at that site, as long as the artist chooses to it leave it there. [4]

In the context of Wright’s concept of software assemblage, we additionally foreground the artistic strategy of décollage that triggers a transformative, perceptual re-assemblage in the viewer: the tearing away or re-configuration of layers of situational assemblage to reveal meanings more profound than the superficial physical or material layers alone. It is in this sense that our work becomes an interventionist space for critical thought. Virtual augmentation therefore is not utilized to enhance or commodify objects or space, but rather to reveal problematics of public or institutional site and memory. The virtual artwork, integrated into the actual Cartesian environment that claims a specific functional or ideological territory, reveals what is otherwise hidden, functioning not merely as a technological apparition but also as an index of suppressed social objects or strata of allusion.

Shades of Absence, Tamiko Thiel (2011–2013+)

Décollage is utilized as a primary strategy in Tamiko Thiel’s series Shades of Absence. [5] Premiering at Manifest.AR’s guerilla intervention into the 2011 Venice Biennale, the artwork is a virtual transnational pavilion for censored artists, an interventionist décollage in response to Biennale curator Bice Curiger’s interrogation of the inherent nationalism of the Biennale in her question: “If art was a nation what would be written in its constitution?”

In these pavilions of absence, photographs of contemporary artists whose works have been censored are reduced to gold silhouettes and enclosed by terms of censorship. The erased silhouettes, some full bodies, some dismembered heads, allude to the negation of the artists’ works and symbolize the countless lesser-known artists facing censorship with no public support. The gold color of the silhouettes alludes to artists’ role as transnational producers of high value commodities, and to the ‘value’ of commercial artworks. Users touch the augment on the screen to view a website listing censorship cases related to the theme of the respective piece, including but not limited to the depicted artists.

In the artwork geo-located in Piazza San Marco, Shades of Absence: Public Voids (2011), the silhouettes reference artists whose public works have been censored (including at the Venice Biennale itself) to perform a critical décollage on the illusory assumption that there is artistic freedom in so-called ‘public’ space. Warning of the fragility of this freedom, it has been re-sited at numerous public art exhibitions such as DUMBO/Brooklyn Bridge, LA.Replay/Los Angeles and the Virtuale Switzerland.

The silhouettes of dismembered heads in Shades of Absence: Outside Inside (2011) allude to artists threatened with arrest or physical violence. In 2011 the art world, assuming that international fame protected their insiders, stood under shock at the disappearance of Ai Wei Wei. Geo-located with the Biennale’s main national pavilions in the Venice Giardini, an ostensibly protected and safe curatorial space, the artwork performs a décollage to strip away illusions of artists as protected “supernational beings” (sic) with license to speak truth to power. The work was re-sited in 2011 at the Kasa Gallery in Not There, an official parallel event to the Istanbul Biennale. In the claustrophobic gallery, a subterranean bank vault in a country infamous for government threats to undesired voices, the (virtual) entrapment of viewers as embodied presence amongst terms of censorship implicated them as participants in an assemblage of power and control.

Shades of Absence: Governing Bodies (2013) was created for the Corcoran Gallery of Art in Washington D.C., with additional sites at the U.S. Capitol Building and the National Endowment for the Arts. Silhouettes of artists censored due to pressure from powerful U.S. government officials reveal conflicts between splendid facades and problematic underlying realities, referencing the Corcoran itself (Robert Mapplethorpe, Paul Cadmus), and the assault on the N.E.A. launched from the Capitol building during the 1990s “Culture Wars,” (Andres Serrano, NEA Four). [6] As with the previous works in Shades of Absence, user/participants both instantiate the artwork and incorporate themselves into the software assemblage, scanning spaces of representational architecture to reveal the contentious history beneath the calm, seemingly munificent veneer of a powerful institution.

Skywrite: Will Pappenheimer/Zachary Brady (2011–2015+)

Skywrite, originally created in 2011 by Will Pappenheimer, is a series of large-scale public AR artworks that allow individuals to create virtual sky-written drawings and messages, utilizing the open sky as a site for public commentary. [7] Situated at architectural sites or landmarks, the work functions as site-specific public space address or redress systems, challenging the use and access to the sky above in relationship to meaning and politics of the topology below.

The Skywrite system includes a touch pad web app, created in collaboration with Zachary Brady, that enables participants to draw or write on their mobiles, then to post these as virtual vapor lines in public spaces, hundreds of feet above the viewer. Participants are invited to move around the city-wide works, to look up as a typical bodily gesture and to become part of the project by posting their sentiments. Each of these practices assembles a located work out of overlapping corporeal and incorporeal fields, creating a critical or transformational embodiment within a multivalent space.

The first series, We Need Something, was created for Occupy Wall Street in New York City and consisted of hand-drawn phrases from the movement. This piece was re-situated along the Los Angeles area coast as part of the LA RePlay show during the 2012 College Art Association conference. The artwork democratized and occupied the privileged access to the sky as real estate, and disseminated an urgent citizen-based address sited along picturesque and affluent beach areas.

Signs Over Semiconductors, created for the Zero1 2012 Biennial in San Jose, CA, addressed Silicon Valley isolationism. Participants were invited to write or draw messages and post them over 24 local companies, breaching the topology of inaccessible, dispersed and secretive corporate technology mini-cities. Climate March Skywrite, suggested by organizers for the Peoples’ Climate March in New York in 2014, featured texts written by event organizers and marchers. Virtual sky-writing acted as route markers while suggesting an alternative, citizen-based protest messaging.

Sky Petition City, sited in Washington D.C., was inspired by Zhao Liang’s 2009 Chinese film Petition, documenting city-sized governmental grievance bureaucracies. Launched at the Corcoran Gallery of Art in 2013, this version highlights similarities to U.S. governmental paralysis. Participants can draw virtual skywriting above 14 iconic governmental buildings, as ‘David’ against the ‘Goliath’ of seemingly unassailable governmental apparatus architecture. Replaced sky-written images are stacked onto the Washington Monument, as an ‘office paper stack’ for endlessly delayed consideration. As this iteration of Skywrite addresses the U.S. capital, so each variant of Skywrite, as machinic assemblage, ‘re-assembles’ with configurations and influences specific to its sites of performance. The AR artworks high above function as décollage to reveal problematics beneath the surface of the physical, institutional locations.

Biomer Skelters: Thiel & Pappenheimer (2013–2016+)

The bio-sensor driven ‘eco-game’ Biomer Skelters incorporates participants’ proprioceptive bodies into a self-organizing machinic assemblage, transforming them into a ‘Johnny Appleseed’-like generative force. [8] As users walk the streets, a simple, wearable bio sensing system polls their heart rates. When the system measures an optimal mind-body state, it plants fantastic augmented reality vegetation, ‘greening’ the city in their wake.

Participants must choose sides: as defensive Indigenators protecting homeland biomes, or as conquering Exoticators forging alien landscapes of invasive plant biomes propelled by climate change. This terminology is conventional for plants, but applied to humans it becomes racially and politically charged, implicating users as participants in a deeper discussion on disruptions of national borders and forced migrations of peoples.

In Liverpool, a city grown wealthy in the 1700’s through the Atlantic slave trade, native plants were confronted with exotic specimens from abolitionist William Roscoe’s botanical collection – many from South Africa. For Virtuale Switzerland (2014–2016), in a country with no history of slavery but very restrictive policies for refugees and citizenship, the discourse between beloved natives and strange invasives – again primarily from Africa and Asia – highlights contemporary ‘nativist’ fears of being overwhelmed by foreign cultures.

In Dubai, native desert plants were confronted with iconic English foxgloves and oaks, touching on its colonial history as a British Trucial State. Dubai, however, a desert where life is only possible in cultivated oases, has a very different relationship to transformational landscapes. Biomer Skelters AR propagation of British native plants, set against Dubai’s hyperbolic oil and real-estate boom skyscrapers and man-made islands, were seen as delightfully quixotic exotics rather than as threats to the established order, even while questioning its sustainability and consequences. Additionally, an interesting cultural problematic was that Biomer Skelters sensors, worn intimately on the body, touched taboo subjects when there were gender differences between artist and participant.

Biomer Skelters thus creates an AR software assemblage consisting of participant choice and proprioceptive agency, the artists’ insertion of material from the historical and cultural biome of the geolocative site at which it is performed, and the Biomer Skelters system of mobile smartphone device, app code, heartrate monitor, augmented reality server and data. Participants, as the assembling agents of the geo-located work, instantiate a unique re-assemblage in each different path at each different site, in relationship to local botany, topography, cultural histories and issues, in addition to more generalized dialogues with the fields of physiological computing, psycho-geography, global climate change and intercontinental relations.

Summary

Augmented reality, instantiated by participants using mobile devices, can be considered – in Deleuze and Guattari’s terms – as an ‘AR machine.’ Rewa Wright’s thesis further posits that AR works, when reinstalled in different locations, are ‘re-assembled’ by each viewer as a matrix of shifting contexts of site, institutional space, contemporaneous culture and cultural memory. Our examples illustrate how AR artworks function as assemblages of process, material, participant, location and meaning. They are fluid manifestations that are the result of motion and process rather than product. Although the virtual layers of the work are configured by the artist, only the active participation of the user assembles the artwork to its final form, as embodied experience instantiated at a specific site.

We suggest that AR artworks function as critical dècollage, as embodied user/participant engagement with and intervention into pre-existing institutional or ideological schemes of representation. Whereas the ability to define ‘reality’ has always been a privilege of authority and power, the advent of AR art enables artists to reveal the multivalent realities of often suppressed or un-enunciated strata of experience and place, forming an interventionist – and hopefully transformative – assemblage. Décollage in this scenic encounter represents an artistic intervention that reveals resonant or discordant, aesthetic and critical relationships of strata thinly veiled by prevailing or conventional orders of the real.

References:

1. Rewa Wright, “Mobile Augmented Reality Art and the Politics of Re-assembly,” ISEA2015 Proceedings, (2015).

2. ‘AR’ = ‘augmented reality.’

3. G. Deleuze and F. Guattari, trans. Brian Massumi, A Thousand Plateaus: Capitalism and Schizophrenia (Minnesota: University of Minnesota Press, 1987).

4. Merriam-Webster Website, “real”, http://www.merriam-webster.com/dictionary/real (accessed April 1, 2016).

5. “Tamiko Thiel @ Venice Biennial 2011: Shades of Absence” Website, June, 2011, https://manifestarblog.wordpress.com/thiel_venice-2011/. Bice Curiger’s questions: http://www.labiennale.org/en/art/archive/54th-exhibition/54iae/. Shades of Absence censored artists website, http://tamikothiel.com/AR/sa/shades-censoredArtists.html (all accessed April 24, 2016).

6. Shades of Absence Website, http://tamikothiel.com/AR/shades-of-absence.html (accessed April 1, 2016).

8. “Skywrite” Websites: “Skywrite AR” 2012, “Signs Over Semiconductors” 2012, “Sky Petition City” 2013, “Climate March” 2014, Will Pappenheimer and Zachary Brady, http://www.willpap-projects.com (accessed April 1, 2016).

9. “Biomer Skelters” Website, Thiel and Pappenheimer, http://biomerskelters.com/ (accessed April 1, 2016)

Bios

Tamiko Thiel is an independent visual artist exploring the interplay of place, space, the body and cultural memory. She is a recognized pioneer in developing the dramatic and poetic capabilities of virtual reality for exploring social and cultural issues. As a founding member of AR artist group Manifest.AR, she participated in their path-breaking augmented reality intervention at MoMA in 2010, and was main curator and organizer of their intervention at the Venice Biennial in 2011. Her work is featured in Whitney curator Christiane Paul’s “Digital Art” and Stanford professor Matthew Smith’s “The Total Work of Art: From Bayreuth to Cyberspace.” Awards include MacDowell, Rockefeller Cultural Innovation Fund, MIT Center for Advanced Visual Studies, Japan Foundation, WIRED, IBM Art and Technology Award, ZeroOne Biennial, FACT Liverpool, Seattle Art Museum. Guest professorships include Carnegie-Mellon, UC San Diego, Bauhaus-University Weimar, University of the Arts Berlin, and Nanyang Technological University Singapore. Email: tamiko@alum.mit.edu, www.tamikothiel.com.

Will Pappenheimer is a Brooklyn based artist working in new media, performance, video and installation with an interest in shifting virtual and physical spatial and object relations, often as a form of institutional or spatial intervention. He is a founding member of the Manifest.AR collective. His work has been shown at Whitney Museum of American Art, LACMA, Los Angeles; San Francisco MOMA; Stedelijk Museum in Amsterdam; FACT, Liverpool, UK; Contemporary Istanbul Art Fair, Istanbul; Fringe Exhibitions in Los Angeles; the ICA, Boston; the Museum of Fine Arts, Boston; New Museum in New York; the Corcoran Gallery of Art in Washington; FILE, Sao Paulo, BR; and Turbulance.org. The artist’s works have been reviewed in Christiane Paul’s historical editions of “Digital Art,” Art in America, New York Times, Hyperallergic.org, WIRED, EL PAIS, Madrid, and Liberation, Paris. He teaches new media at Pace University, New York. Email: willpap@gmail.com, www.willpap-projects.com.