Installation works are presented and proposed which utilize a synergy of kinetic sculpture and projected digital graphic objects in the exploration of the underlying mechanisms of perception. This research seeks to exploit artifacts of low-level visual perception, and expand on the long tradition of Op Art by integrating large-scale robotically controlled sculptural elements in close synchrony with purely digital objects.
Object Permanence, 2012, Marco Pinter, robotically controlled sculpture, ©Marco Pinter. (Used with permission.)
Arguably the most significant early artwork exploiting low-level visual constructs were perspective-correct paintings. Early exploration dates back at least to the Byzantine period, with substantial investigation by Giotto in the early 14th century, as seen in the Scrovegni Chapel in Padua, and further refinement during the Renaissance. Perspective techniques in painting directly map to known cues for low depth perception in human vision. These include linear perspective, occlusion, shadows, texture gradient and aerial perspective (or “distance haze”). Thus painters of these eras exploited the mechanism of depth perception in human vision in order to create the illusion of depth in two dimensions. While these initial successful forays may have amazed and stunned viewers of centuries ago, painting with perspective has now clearly been digested into public awareness and expectations.
In recent history, the period most closely associated with exploration of low-level visual constructs was the Op Art movement of the 1960’s, and to some degree the parallel New Tendencies movement in Europe. Much of this work was “interactive,” not in the sense that we use today, but in that viewing and appreciating the work required physiological processes in the viewer’s eye in combination with his physical exploration of the piece. For example, Bridget Riley’s Current creates the sensations of both color and motion in the viewer from a static black and white painting. Victor Vasarely’s Keple-Gestalt creates a periodically changing sense of dimension in the mind of the viewer. A false sense of the motion in the eye of the viewer resulted in some Op Art paintings being called “kinetic art.” Composition, to the Op Artist, involved both the seen and unseen elements, or rather both the painted and the unpainted, since the unpainted elements were still “seen” when viewed. Bridget Riley stated that “perception is the medium” for this work.  Vasarely’s work, for instance, “relied on the perception of the viewer who is considered the sole creator.” 
Op Art works exploited a variety of low-level phenomena of visual perception. Cyril Barrett described these as occurring at the physiological level, as opposed to techniques such as those in Impressionism that operate at a more cognitive level.  This binary classification has proven to be imprecise in terms of what we now know about the visual system. Still, this classification is generally useful in distinguishing art with the primary purpose to disturb and expose visual perception mechanisms, like Op Art. This type of work is concerned with inducing an effect where the higher and lower levels of consciousness are at odds with one another. For example, the viewer knows there is no movement and color in a certain black-and-white Riley painting, yet can’t help seeing it. Brian Scholl describes these effects as guided by “those principles that are wired into the visual system… [that] are typically fallible heuristic principles that get things right most of the time, but fail spectacularly in some special circumstances.” 
The Op Art of the 1960’s explored a variety of basic visual processes, including figure-ground reversal, texture gradient, linear perspective, induced color, and several others. The works arguably succeeded in drawing attention to the fallibility of visual perception and in creating a form of “interactivity” with the viewer through their own low-level cognitive processes. Yet the technology of that period was highly limited, so the optical effects studied were those that could be painted (and perhaps given some basic motion, as evidenced in prior work by Duchamp, with his Rotoreliefs). Exploration of these optical effects was important in achieving the goals stated above, but after a decade, constrained by such limitations, there was arguably a limited amount of content remaining to be explored.
As the movement faded, entire classes of visual phenomena were left unexplored due to the limitations of the technology and medium. There exist many evolutionarily programmed subconscious visual processes which not only impact our basic parsing of shapes and colors and motion, but may have significant impact on our higher-level decision making processes in ways which we are not consciously aware. The existence of these mental processes can be brought to public discourse through artistic exploration. An effective exploration of many of these, one that truly elicits the perceptual effect in the viewer, requires complex kinetic control and graphics programming. This is now possible due to current advances in high-resolution graphics systems and affordable closed-loop motor control systems, as well as custom technologies and methodologies for synchronization between physical and virtual environments.
The work described here exploits the perceptual phenomenon known as “object permanence.” This was defined by Piaget as occurring in infants between 9 and 11 months of age, after which the child perceives an object as existing in their mind even when behind an occlusion, whereas previously that object would have ceased to exist for them when hidden.  For example, when an adult holds a red ball and moves it through the air and then behind a chair, a young infant will act as if that ball no longer exists, and if it appears again on the other side of the chair it is as if the ball came into existence again. To the one-year-old, however, they perceive the ball behind the chair, moving at the same speed and trajectory it was moving when it became hidden, and they fully anticipate its re-emergence at the appropriate time and position on the other side of the chair. The same effect persists through adulthood, of course. Our assumed knowledge of the position and existence of a hidden object is based on convenient heuristics that we have developed in order to function in the physical environment around us. These are, however, only heuristics and assumptions. In fact, though highly improbable, the ball may have imploded or been otherwise transported out of its trajectory while hidden. Yet at the lower levels of our cognitive processes, we know absolutely that the ball is there even though it cannot be seen and conceivably may not even be there at all.
The phenomenon has begun to be explored in an artistic context, bridging the physical and virtual worlds in a hybrid Fragmented Reality. The resultant work exploits the perceptual phenomenon of object permanence, creating situations of conflict between the viewer’s higher-level consciousness and lower level perception. When successful, the viewer is caused to perceive objects over time that do not in fact exist, through use of graphics, computers and robotically-controlled sculpture. The “virtual” objects in the works exhibit behavior in, and impact, physical space.
More specifically, a Fragmented Reality work creates an environment in which projected digital objects transition on and off multiple screens, spatially synchronized such that they are perceived to exist in physical spaces between and around the screens. Furthermore, the projected (virtual) objects impact physical objects in those spaces. For example, the virtual object can be a circle, moving horizontally across a screen that is mounted at a distance of 24” out from the wall. The circle exits the side of that screen and enters another screen elsewhere in the room, a few seconds later, timed and positioned precisely to give the impression that the circle was traveling through the air between the screens. Further, in certain areas of the physical environment, the imagined, invisible circle will impact a piece of sculpture that is also precisely controlled to behave exactly as it would if a circle of that size had impacted it. Thus the screens and sculptures work together to create moments when an “object” exists neither in the real world or the digital world, but only in the viewer’s own consciousness. The experience of what happens in the spaces, or voids, is more central to the intention than what is visible.
Two Fragmented Reality pieces, entitled Object Permanence I and Object Permanence II, were displayed as part of a solo exhibition at the AC Institute gallery in New York in 2012. New pieces are being developed for the Museum of Contemporary Art of Santa Barbara, CA for Fall 2013, and for other venues in 2014. The work is funded in part by grants from the Interdisciplinary Humanities Center of UC Santa Barbara, the Santa Barbara Arts Collaborative, and the University of California Institute for Research in the Arts.
Object-ness has been studied in philosophy from Heraclitus to Plato, Locke, Leibniz and beyond. It has been researched in cognitive psychology through experimentation over the last several decades. Works of Fragmented Reality explore object-ness in an artistic context, attempting to reveal our low-level cognitive constructions. The work demonstrates that our distinction of objects is artificial, programmed to create divisions and borders simply to facilitate our basic interactions with the world and with each other. The intention is to bring the discourse of object perception, and its impacts on cognition, to the general public through works of art.
Fragmented Reality work is facilitated by technology that was not available during the Op Art period or for many years after. The level of perceptual impact of such a piece, which cognitive psychologists term the “saliency of the effect,” is directly related to the precision of timing and motion. While a basic exploration could exist utilizing simple video effects and standard “open loop” kinetic sculptures, resultant pieces would be imprecise and would not achieve the saliency needed to create the perceptual effect on the viewer. Enabling this requires coordinated control of three or more monitors and projectors, projection mapping technology, and “closed-loop” cybernetic control of two or more motor/encoder systems.
Perception and Evolution
Low-level object perception may significantly impact how we interact with the world. From an evolutionary perspective, our bodies and our survival strategies have shaped the way we think. For example, we have ten fingers; hence all numbering systems are base-10. As another example, the proto-Indo-European root for the word “to have” is “to grab” or “to seize.” Thus the early equating of owning with the physical act of taking or grabbing may shape our instinctual senses of ownership. Further, the root for “to decide” in many languages is “to cut,” as in cutting the branch from a tree. Thus the embodiment of deciding as a physical and irreversible cutting action may create an artificial perception of finality in our decision-making.
In a similar sense, mental constructions around object permanence and persistence may have significant impact on our higher-level decision-making. Our artificial distinction and separation of objects may shape our senses of materialism, personal space, and perception of boundaries and borders on a regional and global scale. Drawing attention to these subconscious biases through the context of art can create a public dialog around these issues.
Within this practice is a commitment to create work that is accessible to a broad audience. The graphical element of the work resonates easily in today’s culture. At the same time, the physicality is fundamental. Dance theorists, particularly John Martin, have discussed the bond between dancers and viewers, using terms such as “kinesthetic sympathy” and discussing “internalizing the dance into his (the spectator’s) own neuromuscular system.”  This author posits that a similar mechanism exists when viewing large-scale sculpture, especially when in motion. The large-scale, physical, kinetic element of the Fragmented Reality work may stimulate the viewer to make connections between the art and their own perceptual constructs and experiences, beyond what would occur with presentation in a two-dimensional medium.
Focal and Subsidiary Awareness
Michael Polanyi discusses the distinction between “focal awareness” of a Gestalt whole versus “subsidiary awareness” of the component elements.  For example, subsidiary awareness of a painting of a bowl of fruit, facilitated by viewing it from a close distance, reveals it is simply comprised of “blobs” of paint. In an analogous fashion, a Fragmented Reality work may create a dialog around perception of objects. Given that the objects we perceive are in fact only mental constructions, a “subsidiary awareness” of this perception of the physical world can facilitate a dialog regarding non-dualism and the impact of perceptual evolution on human behavior.
The works created and discussed here have utilized robotically-controlled mechanisms for creating motion in the sculptural elements. This results in well-defined patterns of motion for the “objects” as they transition between the physical and virtual worlds. One direction for future practice is the creation of work designed to be installed in an outdoor environment, in which the kinetic power will derive from nature, typically wind, as opposed to electric motors. Thus the sculptural elements will move naturally in the wind. Simultaneously, the precise motion of the sculptural elements will be tracked through encoder sensors, and the outdoor LCDs will display virtual, imagined objects which are pushed and thrown by nature itself. In a sense this turns the paradigm on its head, whereby the physical world impacts the graphical world rather than the other way around. Additionally, this adds an element of chance and indeterminacy to the system, guided by natural weather patterns.
Additional work is planned in the area of live performance. In this context, dancers replace kinetic sculptures and engage in a dialog with on-screen forms, actively manipulating them and guiding them in their off-screen trajectories. A variety of narrative themes and abstract movement can be explored within this framework.
The sculptural/digital methodology will also be adapted to allow integration of physical kinetic objects into visualization of information. Further, the framework will be adapted to enable creation of a dynamic physical component to sound visualization projects.
1. Bridget Riley, “Perception is the medium,” Art News, October (1965), pp. 33.
2. Susan Ryan, quoted in ARTATLAS, “Opto Kineticism and The New Tendency. Part I: The Schoffer Effect,” Digital Art History (blog), September 25, 2012, http://arth4482.wordpress.com/2012/09/25/opto-kineticism-and-the-new-tendency-part-i-the-schoffer-effect/ (accessed November 4, 2012).
3. Cyril Barrett, Op Art (New York: The Viking Press, 1970), pp. 38-68.
4. Brian Scholl, “Object Persistence in Philosophy and Psychology,” Mind & Language 22 (2007), pp. 563-591.
5. Jean Piaget, Howard Gruber and J. Jacques Voneche, The Essential Piaget (New York: Basic Books, 1977).
6. John Martin, Introduction to the Dance (New York: Dance Horizons, 1939).
7. Michael Polanyi, “What is a Painting?” British Journal of Aesthetics 10 (1970), pp. 225-236.
Marco Pinter creates artwork and performances that fuse physical kinetic form with live visualizations. He has degrees from Cornell University and University of California, and has served on the faculty of Bucknell University. His work was most recently shown at a solo exhibition at the AC Institute gallery in New York. He is a 2011 recipient of the Visual, Performing, Media Arts Award of the Interdisciplinary Humanities Center. His interactive installation work can currently be seen at the California Nanotechnology Institute. His most recent performance work, Gravitational Forces, was commissioned by the NEA-funded Contemporary Arts Forum of Santa Barbara. Pinter is a contributing author to The McGraw Hill Multimedia Handbook and The Ultimate Multimedia Handbook. He has 7 issued patents and 23 pending patents in the areas of live video technology, robotics, interactivity and telepresence.