Meryl Alper, Assistant Professor, Communication Studies
Christoffer Holmgard, Assistant Professor, Art + Design
Deirdre Loughridge, Assistant Professor, Music
Ronald Smith, Associate Professor, Music
Design for Human Experience: Automation and the Senses
Humans have long been fascinated with technologies that automate processes through various mechanisms, including mechanical, electrical, and computational means. Because automation off-loads to machines manual and mental activities that would otherwise be performed by people, it raises profound questions about the nature of the human experience. This includes our physiological capacity to provide data for perception, otherwise known as sensing. While the human sensory system is enormously complex, sophisticated algorithms draw ever closer to being able to emulate how the human brain interacts with the world using multiple sensory modalities (e.g., “computer vision” that automates tasks completed by the human visual system). In turn, automation has generated serious ethical concerns considering how easy, quick, and cheap it is becoming to simulate and approximate how humans feel, see, and hear. This includes facial recognition software that potentially contributes to racial discrimination, as well as “deep fake” audio and video clips that superimpose sources to create a politically persuasive look- or sound-alike.
Against this backdrop, the social sciences, humanities, and arts have a central role to play in providing a richer understanding of how humans relate sensorially to their increasingly mediated environments, uncovering the histories and associated lessons of past experiments in automation, and pushing the imaginative boundaries of where robotic sensors and the human senses might productively meet. This team of Dean’s Research Fellows will interrogate “Automation and the Senses” from the disciplinary perspectives of art and design, communication studies, game design, and music history and composition. Alper (Communication Studies) will explore how disability, core to the diversity of human experience, illuminates both the promises and perils of algorithmic automation for sensory processing. Loughridge (Music) will examine the history of automation in music, attending to the dynamic between replacing existing practices and enabling new ones in order develop a theory of design for human experience. Holmgård (Art + Design, Game Design) will examine the constant presence of personal automation and surveillance from smartphones through a pervasive game foregrounding and reinterpreting the technology’s ability to continuously track, analyze, and respond to our movement through the world. Smith (Music) will examine the relationship between the mechanical and the musical through an interactive computer composition that is informed by recent work in machine listening and music cognition.