Media Arts in Support of Science Education

Daria Tsoupikova

University of Illinois at Chicago, Illinois

Helen-Nicole Kostis

NASA Goddard Space Flight Center, Maryland

Tina Shah

The Field Museum of Natural History, Chicago, IL

Brenda Lopez Silva

University of Illinois at Chicago, IL


This paper presents three interactive computer applications and the role of media artists in the development of these learning environments for science, technology, engineering, and mathematics (STEM). There is an ongoing movement to integrate art and design to STEM content, to examine convergence points, and how the fields can empower each other. This effort is currently gaining momentum through a wide variety of programs, teams and organizations.  The projects described in this article are the products of collaborative interdisciplinary teams of artists, technologists and scientists, and emphasize user engagement, interactive design and aesthetic principles. The educational content was designed to create informative and motivating learning experiences. We describe the role of media arts in the development of these projects and examine how artists can contribute to the development of innovative educational STEM learning applications. These projects have been deployed in three different educational settings, one for middle school students, one as an iPad application and one as an installation in a zoo.


Interactive 3D applications and visual storytelling are of increasing relevance in our dynamic contemporary culture. Results of art/science/technology collaborations directly can affect educational and pedagogical practice and society at large. In the last few years, there have been a number of meetings that studied art and science convergences supported by efforts of the National Science Foundation (NSF), the National Endowment for the Arts (NEA), the Association for Computing Machinery (ACM), the Conference on Human Factors in Computing Systems (HCI), as well as other institutions (Comer). Another significant effort in this field is the STEAM movement (STEM+Art=STEAM) which is led by the Rhode Island School of Design (RISD). The purpose of this movement is to transform the research policy and to place art and design at the center of STEM education and encourage integration of art and design in K–20 education (STEAM 2011). On February 6, 2013, Congresswoman Suzanne Bonamici raised the importance of STEAM education before a hearing of the U.S. House Committee on Science, Space and Technology. She highlighted the role of creativity and innovation in maintaining America’s leadership position in global economy and discussed how the integration of Art and Design education into traditional Science education can yield the creative innovation workforce. (The US House Committee hearing on Science, Space and Technology). The National Education Association, a founding member of the Partnership for 21st Century Skills, a national advocacy organization that encourages fusion of art and technology into education and provides tools and resources to facilitate that effort (Partnership for 21st Century Skills by NEA).

Roger Malina, executive editor of Leonardo Publications at M.I.T Press and distinguished professor of arts and technology, whose current work focuses on connections between science and art, chairs the Network for Sciences, Engineering, Arts and Design (SEAD) Working Group on White Papers. This group works on a report to address new opportunities and roadblocks to improve collaboration between science and engineering and arts and design and analyze existing data on this subject.

Despite these efforts, the integration of arts in science education has yet to become subject for research and implementation in mainstream cultural institutions and educational programs. To date, little research has been conducted to investigate the role of art, aesthetics, and creative storytelling in the design of advanced technologies for education. In this paper, professional artists involved in contemporary practice examine how media art applications influence informal science education. Each of the authors managed the artistic development of the interdisciplinary interactive learning applications that educate the public on fundamental science disciplines. We describe our case studies and the role of the artists in the design of these STEM educational projects, along with critical perspectives on the benefits, and weaknesses of each project.

Case studies

The Cryptoclub: Cryptography and Mathematics Afterschool and Online is an NSF-funded interdisciplinary research project introducing cryptography and mathematics to middle school students across the country through the encryption and decryption of codes. Project objectives include increased awareness of cryptography as a STEM topic with connections to mathematics, as well a greater understanding of effective strategies for integrating and supporting online and offline activities within informal learning settings (Beissinger). The project’s team is from the University of Illinois at Chicago and includes two mathematicians with extensive experience in math education; one artist, whose research is in visualization, interactive animation and educational multimedia; a cognitive psychologist with extensive experience in research, development and evaluation of multimedia mathematics and science materials for middle schools; and expert teachers who pilot the project materials. Project partners include the Young Peoples Project (YPP), a national afterschool program; Eduweb, an award-winning educational software design and development firm; and American Institutes For Research (AIR), an evaluation firm with experience in afterschool evaluation. Project materials include afterschool and online activities that enable students to learn and apply cryptography and mathematics skills. This includes a Cryptoclub website ( with challenges, a treasure hunt clue generator, cryptography adventure games (2D single-user games and 3D multi-user games), graphic novels based on cryptography, and offline games and activities that involve active participation, a leader’s manual, and training workshops for afterschool leaders. The project materials were developed and tested in afterschool programs in Chicago and are being field-tested in approximately fifty sites around the country.

The artist and mathematicians collaborate with software developers and work together to address requirements framed by NSF and teachers, as we well to overcome technical limitations. The project was originally initiated due to a genuine interest in an intriguing interdisciplinary project and the group was eventually successful in securing NSF support for continuation. Another very important aspect is that the artist and scientists get their rewards in fields that are relevant to them, for example the budget includes funding for artistic conferences as well as for scientific and educational conferences and publications. However, collaborators commonly experience difficulties when publishing in discipline specific journals, which are not recognized by official policies of collaborating departments. For example, for an artist, a publication in the mathematics journal may be considered irrelevant and vice versa. Fortunately, all collaborators do share relevance to education as educational publications are recognized by all hosting organizations of specialists involved in this project.

The artist’s role focuses on the development of the art and design portion of the project. The artist collaborates in the design of the website, the online games and other digital activities, and develops concept designs for 3D environments of the computer games. The artist leads the development of graphic novels, a series of stories in comic format that are based on historical events related to cryptography (Figures 1 and 2). As part of this effort, the artist is responsible for researching historical information about the environment, architecture, clothing, and other details in order to achieve convincing and historically accurate visualizations.

During the development it was imperative that all specialists would discuss and agree on the project plans, goals and objectives. As a result of these meetings mathematicians learned more about production, aesthetics and design including terminology of web design and development. The artist in turn learned a lot about mathematics, cryptography and history of cryptography.

A theme in our development is to use art as a motivation for students to learn complex subjects in informal settings. The website is still under development, but preliminary results show favorable response. While students are attracted to the games and activities on the website, teachers may not be fully aware of how they can use the website in their teaching. A future plan is to develop additional teacher support material to help them incorporate the website into their afterschool program.

Figure 1. Trying to crack one of Beale’s ciphers. (From Beale Papers Mystery graphic novel, currently available at

Figure 1. Trying to crack one of Beale’s ciphers. (From Beale Papers Mystery graphic novel, currently available at

Figure 2. Mary hearing her sentence.  From the Mary Queen of Scots graphic novel, to be available soon on the CryptoClub website.

Figure 2. Mary hearing her sentence. From the Mary Queen of Scots graphic novel, to be available soon on the CryptoClub website.

NASA Visualization Explorer (NASAViz) is an intuitive and interactive free iPad application available in the Apple iTunes store that delivers bi-weekly science stories about NASA’s exploration of the Earth, the Moon, the Sun, the planets and the Universe (“NASA Visualization Explorer for the iPad – Home”.) The app was released to the public on July 26, 2011 and as of June 2013 has achieved approximately one million unique downloads by users worldwide. The stories emerge from produced and visualized satellite data (animations and images) from the Scientific Storytelling team at NASA Goddards Space Flight Center, with the purpose of educating the public about NASA’s science research in an informal and visually rich environment. Sample stories include hurricanes, ocean currents, daily snow cover, solar eruptions, discoveries of new planets, galaxy formations and the orbits of satellites of NASA missions.

Figure 3. Screenshot from NASAViz science story Zone Coverage demonstrates how weather satellites collect data.

Figure 3. Screenshot from NASAViz science story Zone Coverage demonstrates how weather satellites collect data.

The app was developed for the general public with a strong interest in science, NASA and visualization. The app is an in-house production of the NASA Goddard Space Flight Center and the broader team is divided into three groups: the Editorial Board, the App Development Team and the Content Development Team. The Editorial Board and the App Development Team make up the core multidisciplinary team of the NASAViz project where members include science writers, data visualizers, software developers, producers and interactive designers (Figure 4). The roles and skills required in each of these groups are described in the following diagram:

 Figure 4. Skill sets available in the teams involved in the production of NASAViz stories. Skill sets in red hues signify a higher proportion of media arts. For example, darker red hues imply higher amount of media arts expertise.

Figure 4. Skill sets available in the teams involved in the production of NASAViz stories. Skill sets in red hues signify a higher proportion of media arts. For example, darker red hues imply higher amount of media arts expertise.

Even though the app is developed for the general public it is also used by teachers in the classroom as an informal education medium for curriculum support. For this reason the NASAViz team in collaboration with the NASA Office of Education and Einstein Fellows launched the NASAViz Teacher Pilot project in September 2012. The purpose of this effort is to: 1) learn more about how teachers use the app in the classroom 2) receive input on new features which may assist the teachers and 3) receive feedback from teachers about the content, especially about stories on hard-to-explain scientific and natural phenomena that are part of the curriculum. During this pilot project stories were introduced to the students by the teachers to either support the academic curriculum or to engage students more in the classroom material. The team is currently evaluating the data of the educational project and as a next phase will revise the educational goals for this project. One of the weaknesses is that since stories are designed for the general public and especially for the science aware audience, elementary and possible middle school students have difficulty understanding and following key subjects described in the stories.

Figure 5. Students explore NASAViz stories in the classroom during the course of biology.

Figure 5. Students explore NASAViz stories in the classroom during the course of biology.

The continuous interdisciplinary effort of developing, releasing stories and new updates of the app augmented the capabilities of the team in storytelling, mobile app development and engaged new audiences that were not reached before. The app has been selected and featured by iTunes as a leading Earth Science Education app continuously since its release. A challenge encountered by the project has been the establishment of a common language between the teams involved and its members. Overcoming this challenge has been an ongoing effort and the team members have shown improvement that has manifested in the reduction of turnaround time for the development of the stories.
As described in the diagram of Figure 4 the project teams employ the involvement of media arts professionals from various fields in the development of the app and the content. The contribution of media arts skills and experts benefit the overall production of the content (visualizations and videos), the visual flow of the stories, ensure a consistent look and feel on the storytelling effort and in the user interface experience.

 Figure 6. Users experiencing A Mile in My Paws immersive environment.

Figure 6. Users experiencing A Mile in My Paws immersive environment.

A Mile In My Paws is an interactive 3D application designed to raise awareness of the effects of climate change on the polar bear population. It introduces zoo visitors to concepts of ice melting in the North Pole by giving them the opportunity to traverse and explore a visualized area of the Beaufort Sea – a terrain based on historical data, and on projections of ice coverage in the future. The main goal is to teach that the longer the polar bears swim, the more energy they consume for hunting.

Visitors are put in control of a polar bear avatar that must traverse the arctic environment searching for food in the past, present and the future. As sea ice decreases, from the past, to the present, to the projected future, polar bears must work harder to travel the same distance, as they burn 2.6 times the calories swimming versus walking. Consequently Paws’ users work harder, by swimming more, as they change time periods. This embodied experience is intended to convey directly – via associative learning – the difficulty that climate change causes polar bears, and to set the stage for learners to be receptive to understanding the rate and magnitude of climate change.

The navigation and interaction requires users’ physical effort and embodiment to support learning. For swimming, users wear a pair of polar bear paws with an embedded iPod touch that sends the acceleration data to the system as the move forward in the virtual environment. For walking, users step on a step-pad with pressure sensors connected to the system. The virtual environment is complemented with an iPad application that displays real-time information related to users’ location on the virtual environment, their performance and climate change facts such as images and graphs.

While the exhibit was (unsurprisingly) found to allow only a fraction of passing visitors to directly interact with it, educational benefits were found in visitors who merely observed others using the exhibit, or who “played along” with it themselves. These benefits were facilitated by the presence of interpreters, who were able to make a limited, performative experience more inclusive and participatory.

Our research looks at factors related to user experience, embodiment and learning outcomes about climate change. Our group has done formative studies about system calibration and user experience/interaction (Lyons) that includes: 1) judgments about different levels of efforts; 2) attention focus and degree of immersion; 3) degree of personalization of the experience; 4) individual approach to using the embodied controls and 5) interpretation methods to expand the knowledge to broader audiences and not exclusively to the player.

The design process included a series of discussions with experts in climate science, education, computing and designers. The purpose of the group’s research has been to accurately inform the general public about current research on climate science and to make it engaging through accurate graphics, immersion, and embodiment. The first revisions in the design were content driven –guided by science experts. The latest phases were influenced by the results of the formative studies, and resulted in the redesign of the system’s graphical and physical interfaces: replacement of wiiMote with iPods, and later on with optical tracking; weight addition to the globes; and changes to visual elements. They also led to the creation of an additional iPad interface to display users’ progress, performance graphs, and to showcase facts triggered by the user’s location in the virtual environment. The development team fueled the creative process, and it has driven content changes, visuals, and the aesthetics of the virtual environment. It has been an informed design that perhaps without the team’s feedback, could have introduced misconceptions.


Multidisciplinary teams developed and designed innovative interactive applications that support STEM education, for museums, the iPad and the Internet. Artists shared critical perspectives on the benefits, weaknesses of each project and their outcomes. Overall artists found collaborative work rewarding and consider their collaboration with scientists as an integral part for successful development of these projects. It has become clear throughout the research and development process that the skills and interests brought by the various artists and scientists of each team greatly enhanced the potential of each project. These creative synergies serve as examples for future interdisciplinary arts and science collaboration for STEM education. The process of identifying elements of a successful art/science development that focuses on creative engagement and incorporates practices and theories from media arts and STEM education is just beginning. There are increasingly growing opportunities for artists to collaborate with scientists in the design of innovative interactive learning applications. The differences and similarities of art and science research methods, creative discoveries, evaluation methods, and the challenges in the development of STEM applications will be covered in future research of this group.


1. Janet Beissinger et al., The Cryptoclub: Cryptography and Mathematics Afterschool and Online (DRL, NSF Web. 2009.)
2. Chris Comer, “SymBIOtic ART & Science: An Investigation at the Intersection of Life Sciences and the Arts”, Final Report for the NSF Art and Science Workshop, NSF and the NEA (Arlington, VA, pub 2012.)
3. Lyons, L. et al, “Don’t forget about the sweat: effortful embodied interaction in support of learning.” in Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction, TEI ’12 (ACM, New York, NY, USA, 2012), pp. 77-84.
4. NASA Visualization Explorer for the iPad – Home. NASA Viz, n. d., a (accessed September 6, 2013).
5. NASA Visualization Explorer for the iPad – Credits. NASA SVS, n. d., (accessed May 27, 2012).
6. SEAD, (accessed September 1, 2013).
7. The STEM to STEAM Briefing, Rhode Island School of Design, (accessed September 2011).
8. The US House Committee hearing on Science, Space and Technology, CSPAN coverage on, (accessed February 6, 2013).
9. Partnership for 21st Century Skills by NEA, (accessed September 1, 2013).


Thanks to the Electronic Visualization Laboratory (EVL), the NASA Visualization Explorer Development, Educational, Editorial and Story Development teams (“NASA Visualization Explorer for the iPad – Credits”), Cryptoclub team, Janet Beissinger, Andrea Barbarin and Deepika Yuvaraj from ASB 2012 at University of Michigan, Tom Moher, Leilah Lyons, Brian Slattery, Priscilla Jimenez, Learning Technologies Group, UIC.
The discussion of the CryptoClub is based upon work supported by the National Science Foundation under Grant Number 0840313. Any opinions, findings, and conclusions or recommendations expressed here are those of the authors and do not necessarily reflect the views of the National Science Foundation.


The Media Art Group has been working together for ten years and focuses on integration of art in the development of interactive learning environments for science, technology, engineering, and mathematics (STEM.) Our experience is derived from our participation in art-science collaborations at the Electronic Visualization Laboratory in the fields of virtual and augmented reality, scientific and space visualization, computer games, learning technologies, and interactive art installations for museums and galleries.

Currently, members of our group lead the artistic development of innovative learning applications in the following organizations:
Daria Tsoupikova and Brenda Lopez Silva (University of Illinois at Chicago)
Helen-Nicole Kostis (NASA Goddard Space Flight Center)
Tina Shah (Frank Lloyd Wright Preservation Trust in Chicago)