Qianqian Ye, Artist and Creative Coder

Exploring our shared vulnerabilities — and shared futures — in ink and code

In this episode, Liam speaks with interdisciplinary artist and creative coder Qianqian Ye in her San Francisco studio. The duo traces her journey from wielding calligraphy brushes to building a hand-holding glove, unpacking the vulnerabilities we all share as humans, how creative intent is communicated, and the importance of imagining other futures.

---

Liam Spradlin: Qianqian, welcome to Design Notes.

Qianqian Ye: Hi, thank you.

Liam: To start off, I want to know about what brought you to your current work and how has that journey influenced the things that you’re making?

Qianqian: So, it has been a very interesting journey but I think it all starts with calligraphy ink and brushes. So, when I was a kid, my parents sent me to calligraphy school, because I was not a very quiet kid. So, I had my calligraphy s- practice for 10 years, but I know that I really hated. I, you know, had a lot of struggle with that but I guess that has embedded in my, my blood, literally.

And then, uh, I went to architecture school and for my Master I studied landscape architecture. At that time, I had this, like, kind of dream that I felt designing space for people is something I’m very interested in. And then I realized this kind of social interaction in groups is a main thing that I’m interested in. The form of the beauty, the material, or, you know, the o- other very architectural things I actually know are really my focus. I realized people is the thing I’m most curious about.

Then, uh, I moved to Bay Area. At that time I was working a Danish urban design company. The fact that I was leaving Silicon Valley really changed a lot of my life path. So, then I started to do more of technology-related design. Coding, those sort of things. And, you know, such a moment of my life, I start to go back to ink and water, and brushes. And I start to make paintings with them. And all the paintings that actually are talking about the vulnerability in human actions. And along that journey, I was making different, other type of works, but pretty much most of them are talking about the same issue. About how people are vulnerable and awkward, but still trying.

Liam: So, you mentioned that a theme of your work, kind of thing that you’re really interested in is the complexity of human social interaction, and I’m interested in the ways that that theme has emerged in all of the other media that you’ve worked with.

Qianqian: Yes. So, I think back in architecture school, when I was designing spaces, I always am thinking about, like, what kind of social story would happen here? Will people come here to break up with their partners? Will they come bring their grandparents? Like, what that is kind of potential social stories change the way I design a space. And last few years, I’ve been spending more time doing creative coding, community building. And that’s a huge people project for me. And it really opened a window for me to understand what does a group mean, what does community mean? What does human interaction even mean in a, in a group like that?

So, I feel like along the journey, my understanding of how human and humans work have been changing a lot. And I’m very curious to see how this exploration is gonna take me to.

Liam: So, focusing for a moment on a specific work of yours, I’m interested in the origin and the meaning behind the Alone, Not Alone works.

Qianqian: So, Alone, Not Alone project, actually it contains a lot of different part. It start with, actually, a set of robotic installations that I was working on. So, that was a set of installation includes a robotic hand that will hold your hand, and a robotic arm that will give you a hug. And those robotic hand and arms were designed to put into, like, a glove and a scarf. Um, so that was a project that I was thinking a lot about how humans are lonely, or not lonely with a companion of technology.

So, that was like the view one of Alone, Not Alone. While I was working that, I started to make a lot of paintings to discuss about the loneliness, um, and the awkwardness, and vulnerability in that sort of human, human, human technology, or human the world relationships. Um, so that kind of was a story of how Alone, Not Alone got started, and how it developed to a set of ink paintings that you see on the walls.

Liam: In a piece in The Offing, you mentioned that the minimal nature of these pieces combined with the narrative titles that you came up with let viewers make up their own stories. And I’m interested in how you think about that, and how it changes the interaction viewers have with the work.

Qianqian: So, I think vulnerability is something I talk a lot about in my work. I think it’s universal, but the story behind why you are vulnerable on certain things is different for everybody. So, when the show Alone, Not Alone was up, the curator asked me, “Who do you think will buy your paintings?” And I was like, “Maybe someone who’s awkward and lonely, and, you know, vulnerable.” And then she said, “Oh, you mean everybody?”

So, I was like, “Ah, okay.” So, that was time I was like how this thing, when I thought it’s like, uh, a personal character is actually so universal. But when I was thinking about how do the viewers embed their personal story into the work, there was one example that I remember really vividly.

So, that was one buyer, and she contacted me on social media. She said, “Hey, I really like your work. But your paintings, uh, the show just, uh, finished. Can I come to your studio and buy some work?” And I was like, “Oh, sure.” So, she came, and she bought three pieces. And I asked her, I was like, “Oh, why did you buy these three?” And the reason was, she, the reason she gave me was completely different from my intention behind those works. That was a moment that gave me a lot of feelings. There was some feelings of, ah, as an artist, I’m not ready to sell my work. Because when I sell this, they will belong to other people. The narrative will belong to them. And then I was feeling like, I am glad that the narrative of my work is not that narrow. It could be translated to a lot of different feelings.

So, in one of the pieces that she bought was, was just me drawing three people, that I called that as, um, three is half of the infinity. Because it’s kind of half of the infinity sign. So, that was actually a project that I was exploring about, uh, polyamory, or, you know, open relationship thinking. Like, what does it mean to have a regular relationship? Can we think about different type of relationship interactions? So, that was actually my intention behind that work. And then I ask her, “Why do you buy that piece?” And she said, “I have three kids. This totally reminds me of them.” And I was like, “Okay!” Of course, I didn’t share with her about my intention behind it, because, and the title itself, I think, works for me and worked for her, both ways. So, that was a moment that I remembered that really, really vividly.

Liam: And do you think that that’s changed how you relate to the work? This kind of balance between, like, the intent that you’re putting into it, and the way that it’s received.

Qianqian: Yeah. I think that’s sort of like, when I start to sell my work, and start to hear people’s understanding of it, that was a time I realized I need to do more internal work to make sure that I’m making the pieces for myself. Because sometimes, I know that the buyers or collectors opinion will affect the way I do things, and I know you will, I will probably get affected. But I really want to protect my inner self that I don’t want to make work actually for other people right now. I think a lot of work I have to do is just like, I need to let them out. And I really don’t care if it’s going to get sold or not.

Liam: Within this work, how you harnessed the interaction of the materials that you’re using, so ink, water, and paper, in shaping these pieces, and there’s, I would say, maybe like a slightly unpredictable nature about that sometimes, and I’m interested in the unique interactions that might be afforded by the other media that you’ve worked with, particularly the digital or technological ones.

Qianqian: Yeah. I think ink and water, paper, this combo is really organic. It’s really unpredictable. Which is something I really like about it. And the technology, or digital medium have something that is always so precise. So, I was always being thinking about how can I combine these two things. So, there is one project with a science fiction book illustration that I did for a museum in China. They commissioned me to use ink and water to pain a dystopian future of a Chinese city when technology takes over, and when people are really lonely and isolated. So, that was the one time that I discovered this power of ink and water in the futuristic and technology technical context.

Another experiment that I been done with ink plus water and other technologies were, there was one piece on my to-do list that I want to go back to China and collect some smog to make some ink, and then using that ink to paint some works about the environmental issues. So, that’s kind of like alter the ink element in this sort of work using technology.

And some other type of work that I have done is using, alter their water element in this combo. So, for instance, I did a work that using ice instead of water to produce work. So, water is really unpredictable when I work with it, but actually, ice is even more unpredictable. And that has been fun, like, exploring different mediums. But I know there’s some other type of explorations that I want to do, that bring this kind of Chinese-ness and the future-ness together in some sort of format.

Liam: Yeah. And speaking of that, we were talking earlier about your work on Other Futures. For the listeners, I’m interested in what is that project, and also what was the motivation behind that?

Qianqian: So, Other Futures is an event that I’m producing right now at the Gray Area of Grand Center in San Francisco, California. So, that is a night of audio visual performance, include a piece by Chinese artist duo, Miao Jing and Jason Hou, called Zep Tepi. Zep Tepi means first occasion in Egyptian creation mythology. That piece talk a lot about what is the type of alternative future we can create using mythological and, uh, historical figures. And, that piece itself was the reason why I actually started producing this event. So, when I saw that piece, I realized, wow, I really want to bring that to the Bay Area. Because that piece currently is touring right now in China, and I felt that kind of future narrative is something I really want to add to the conversation we’re having in the Bay Area right now. Especially in the electronic music scene.

So, when I go to audio visual nights in SF or in Bay Area, I was seeing this very Western, minimal, geometry style. Then, when I saw this very mythological, oriental images from the Zep Tepi piece, I was like, I want to bring that here. Because when we talk about the future, when we talk about music, electronic music, can we have some other voices? Can we bring the otherness there? And then I’m working together with three local artists as well. Their work are talking about how can we challenge the mainstream future narrative using some non-western music and visual. Or, even some other piece a-about, like, can we imagine a future that’s 100,000 years from now? How could that be?

So, discussing the otherness in the future, or other futures, are something I’m very curious to explore right now. And I want to bring more of this kind of conversation to Bay Area.

Liam: In your conceptualization of these futures, do you think that otherness continues to be a concept?

Qianqian: I think, uh, otherness, or futures, the, the parole is something I’m heavily believed in. Because I don’t want to see a singular future. I don’t want to see a future that’s pictured by one group of people. I want to bring different voices at least to that conversation.

Liam: Going back to talking about the fact that your work touches a lot of different media, in an interview with Forth Magazine you said that the medium doesn’t necessarily matter, and that the important thing is the message, or the story you want to tell. And because your work expands many different media, I’m curious about your process for deciding which medium, or which tool is right for the story that you want to tell.

Qianqian: Yeah. I think it’s really a interesting question, because I’ve been asking this to myself a lot as well. And, I think sometimes it depends on what is the story I want to tell. The story, usually, is more important than the medium itself. So, I was making a lot of ink paintings. And at that time, I was actually working on a creative coding library called P5.js, the translation for it. So, I was helping a fellow that year called Kenneth Lim, to translate that library to Chinese. And then the creator of P5.js, Lauren McCarthy, was like, “Hi, Qia, can you actually make some work that we can feature on the home page?” And I was like, “Okay.” And then she was like, “You, you did some ink work. Can you just use P5 to maybe do something about ink?” And I was like, “Mm, interesting. I never thought about it.” Then, I was [inaudible 00:15:57] for her encouragement, and then I made a piece that just, very simple, 50 lines of code, to just draw a inky landscape, that code Shan Shui, using P5.js.

So, that was an example that how different medias got translated, or transformed in a way. The other main story that I’m thinking about, regardless of the media, is, like, what I mentioned before, is I’m just so curious about the vulnerability of people, the soft side of them. Because I think I’m, in general, slightly, or pretty much socially awkward. And I’m very curious about that part of myself, and I want to go dig more. When I meet people who are kind of awkward and vulnerable, I just find this kind of comfort. So, I kind of just want to render that sort of story in different dimension.

So, I have done, like, these sort of stories using ink and water, and using, you know, robot installations. And even using, you know, just HTML, and JavaScript. So, I think a lot of times, stories comes in first, and then the media will come, and a lot of times the media actually will just, you know, they are fluid. You can transform from the ink work to web work sometimes.

Liam: Yeah. I’m interested in that idea of differences and similarities between the kind of unpredictable, organic nature of the ink work, and the very structured, and structural nature of code. You mentioned in that same interview that technology could just be another way to create art. But I’m interested in those differences, like, what might make technology unique among this pure media. Does it provide new capabilities for expression, and if so, how do you think about that?

Qianqian: Yeah. I think the fact that I’m leaving Silicon Valley make me a really tech savvy person. So, technology for me, I think, really brings the accessibility and scalability to works. So, for instance, the ink that I painted on the web, uh, bad joke, that actually can be accessed by my parents in China. They can go see the interactive work in the website. Like, when you move around, the mountain moves and stuff. So, they were, like, impressed by that. But they were not really impressed by any of my ink work on paper. Because they can’t really see them, they don’t know how [inaudible 00:18:31]. So, the fact that technology really helped to, to make a lot of works really accessible is very magical moments.

So, there was this project that I was working on, it’s called Portrait of City. So, because of my background in architecture and urban design field, I’m always very curious about how a city could look like. So, I was working to collect a lot of influencer images in certain cities. And ironically, a lot of them look very similar. So, I was trying to use [inaudible 00:19:03] to train the machine to draw how the city could look like if we’re all using that set of influence images. Like, will we be able to generate a portrait of the city using that sort of database? So, this is something I felt like I’m so grateful that there’s a lot of open library out there that I can use, and give me this potential to think about a new way of giving a city a portrait.

Liam: You mentioned that technology can make your works a lot more accessible in the sense that the things that you’ve created for the web can be accessed by just about anyone, and experienced that way. But, that your ink works may not be the same, in the sense that someone can’t, like, really see it physically and understand how it feels. So, is there still something that technology can’t capture in that case? Is it the physicality of the work? Or, I guess I’m interested in the gap there.

Qianqian: Yeah. I think the physical and di-digital relationship is something I’ve been thinking a lot. With some of my work, uh, very primitive, and physical, and some of my work, uh, really digital. And, currently, I felt those other elements sometimes can transform. So, for instance, the ink paintings, after I did a lot of ink paintings, I can’t resist the urge to start to paint them in 3D. So, I used VR sculpting tool to start to paint, like, when I was doing ink on paper. So, I produced a set of my very blobby, lonely people in VR. And I then exported them, and 3D printed them. And then, of course, I did render, and, uh, the 3D printing, so that was a set of work that I have the ink painting, that in 2D, on paper, and I have this virtual sculpture that I made in VR, and did the 3D printed version of that.

So, that entire process, it kind of like a ritual for myself of, how can I transform the vulnerability or awkwardness from the paper to the physical world? And you will see, actually, that some people, there’s some of the sculpture that I did afterwards. So it always feel like this sort of different media can assist each other in a way, and create a flow of work, and then maybe on the flow, you would see different potentials of new type of work. But right now, a lot of this work has to account in their prototype, or experimental stage.

Liam: Yeah. But, something new was perhaps revealed by the process of translating something from 2 dimensions to 3.

Qianqian: And, or to physical world to digital world.

Liam: Yeah.

Qianqian: Yeah. It’s some question that I keep asking myself. But, maybe I can answer that question with a observation my friend told me. She told me, “You know, your work actually really resonated with a lot of people in Silicon Valley particularly. Because here, you know, technology is so advanced that everybody is talking about [inaudible 00:22:15], and AI, and, you know, self-driving car. Then you work about, oh, actually, if you were vulnerable on Friday night home alone, petting cats.” So, that capture actually really resonate with a lot of people.

So, that kind of vulnerability that I express in the physical work resonate with a lot of people who spend a lot of time in the digital technology world, in a way.

Liam: Yeah. So, although we always have Friday morning, and Friday afternoon with the technology that’s shaping our lives, there is still Friday night.

Qianqian: Yeah.

Liam: I want to close by asking about the future. I’m interested in how you see your work and your pursuits continuing to evolve as technology continues to change, and perhaps as human vulnerability continues to stay the same.

Qianqian: Yeah. I think alternative futures are something that I’ve been thinking about a lot, particularly single futurism. Like, because I think the fact that at least you’re trying to figure out why did I pick up my ink and a brush in the first place, and second place. So, I’ve been thinking about how does that affect where I am right now in my life. And I think the Chinese-ness of the ink, the water, the paper, is something so embedded in me, and I really want to understand more about my root, meanwhile, understand more about how this root will guide me to the future.

I can share some books that I’ve been reading a lot. One of them is called Techno-Orientalism. It’s about alternative future in the oriental world with a lot of science fiction and futuristic writings. So, this sort of future reconstruction is something that I find very interesting. Meanwhile, in the future of my work, I know that I want to do a lot of things about China, about technology over there, about social justice over there, about gender inequality over there, about how I am as just one single human being in that grander narrative.

Liam: And what are your plans for that?

Qianqian: Um, so, right now I’m working as a fellow at Processing Foundation. So, my project is about to make P5.js, which is a library that I mentioned earlier, more accessible in China, particularly in the under presented woman and non-binary group. The reason why I want to do that, is because most of the education resources in the Western world are not very accessible in China, because most of the social medias and, uh, video sites, are not accessible there. So, the fact that I learned coding by myself watching YouTube videos is actually a huge privilege compared to a lot of my friends back in China.

So, in that project I am working on right now, I am recording myself teaching quick coding in Chinese, and I’m going to release some on Chinese website so everybody can have access to that. So, my goal of that project, like, my personal goal for that project, is to teach my mom how to code. She doesn’t speak English. She has never seen anything from YouTube, so I’m wondering if there’s something that I can teach her to do. And I always felt if I can teach my mom to code, I can probably teach a lot of people how to do that. So, this is one of the project that I felt like I’m working right now about the key words China, technology, feminism, woman, new type of future. Yeah. Thank you.

Liam: All right, well thank you again for joining me.

Qianqian: Thank you for having me.

Harvey Moon, New Media Artist

On getting to know your tools and the art only technology can create

In this episode, Liam speaks with new media artist Harvey Moon in his San Francisco studio. The duo discuss how Moon’s work reveals unseen properties of the world around us, the process of creating one’s own creative tools, and the kind of art that’s only made possible through collaboration with machines.

---

Liam Spradlin: Harvey, welcome to Design Notes.

Harvey Moon: Good to be here.

Liam: So, to start off with, I’m going to ask what brought you to your current work. And also, how do you think the journey there influences the kind of things that you make?

Harvey: I think the journey started in the dark room. My father was a photographer. And when I was growing up, he built a dark room for my sister and I in the basement. And I use to spend hours and hours in there. And I feel like there was a lot of freedom and constraints in that tool, in that process, and I really fell in love with it. All of the techniques that you could experiment with and all of the different types of cameras and really the history of photography really excited me.

And as it started to evolve and digital became a lot more prevalent, the process and the art evolved with it. And so, the way I thought about photography in the dark room was very different than how I started thinking about photography digitally and how I started working with photographs using a computer and not a dark room.

And so, there was some organic growth. And as different things started fading in and out of my interests, it would combine in different interesting ways and kind of produce a new trajectory. And so, I don’t think I ever really had an intention of the work I would make today. It all sort of happened piece by piece just following different passions and putting them together. Um, but I think photography is really the, the beginning and the core of where, you know, a, a lot of my love for technology, and tool making and that process of making a final product really started.

Liam: So, you describe your work as being centered on the ways that technology can mediate how we perceive the world. And I want to start by getting in to what you mean by that.

Harvey: Yeah. I mean, it’s a big one. So, I think if you trace back a lot of the pivotal inventions that we’ve relied on, they’ve really extended or changed our understanding of the world. We can go back to photography and think about Muybridge and how he would use photography as a way of seeing things that we could never have seen otherwise. And now we kind of take that for granted. We know in our mind what a slow motion video looks like or what a horse looks like as it’s running.

But we couldn’t have known that unless those techniques had been invented. And through that, we were able to see beyond our own potential. You know, I, I was really driven to photography for that reason that you could perceive time in a different way. You could freeze time. You could use it in a sequence in creating animation and changed your duration of your experience of time.

You know, Henri Cartier-Bresson would talk about the decisive moment. There was always this one special second that you could only capture with a photograph. A photograph now is so drastically different than the way we can perceive the world, but we’re so accustomed to this that we have a, a really understanding of what a freeze frame is.

Liam: So, as you mentioned, there’s the idea that technology has historically been about extending our capabilities or unlocking new potential for us as humans. And I’m interested in how we can remain aware of those influences.

Harvey: I think it’s really important to recognize that everything we do has some impact on us. It’s kind of a collaboration where you collaborate with the tool or the machine, and then the tool sort of teaches you what it’s capable of. And we all have this relationship with a camera, or with a pencil, or a, a paintbrush, or if you’re a writer, a typewriter. We have … We learn how to use it. We learn how to type. We learn how to change your f-stop. We understand, you know, what pressure to give on your, on your paintbrush. We learn those things over time, and we learn what a tool can do and can’t do with those abilities.

And we kind of get this relationship. You have a favorite camera and you know just intuitively how it works, and it becomes a part of you. Your eye can just see through the lens. And at a certain point when you’re good at typing, your thoughts can just extend into the page.

So, I think we have this constant relationship to our tools that we kind of ignore and forget about, but all of those tools end up influencing how we use them. There’s only way to use a pencil, and you can’t go sideways. And you only have a certain number of letters to work with in language. And I think these sort of tools have constraints built into them that we should recognize when we are using them.

You know, a typewriter or the keyboard we see today was made to be slower because of how typewriters would jam if you type too quickly. And so, nowadays, we’re maybe limited by how our history of this invention has dictated our use of it. Whereas, we know there’s other faster ways of inputting text, but we’re kind of at the mercy of these technologies that have become so pervasive.

And I think it’s a really difficult question, because it’s really impossible to see how people will respond once they become accustom to it. And that’s why some things like cameras, which have been around for so long, have morphed into this tool that we’re really comfortable with. It fits your hand. It didn’t use to. It had to be done over years and years of experiment, and practice and trying. There’s no way that the original photographers could have imagined that a camera would be in everyone’s pocket.

So, it’s impossible to know what effect these technologies will have ten or 20 years down the future. But it’s important to recognize what impact we currently are having with these, at least to think about ten or 20 years down how it will impact everyone. And I think it’s hard to be aware of the momentum that can happen when something becomes so ubiquitous and the good or bad side effects of how those sort of technologies will start mediating the world. Probably an important topic for Google to consider. You know?

Liam: Absolutely (laughs).

Harvey: Like, these sort of tools are extremely powerful, and decisions that are made today will definitely have a lot of consequences, good and bad, ten years down the line. And it’s impossible to know, like, how the world will respond and reflect on these technologies.

Liam: You talked about the evolution of the camera and how it went from this, like, very manual large thing into something that fits quite naturally into your hand and seems like a natural extension of your expressive capabilities in some ways. I think we think about taking a picture as capturing something accurate when in fact it’s like a representation of something accurate. I guess I’m wondering how you think about, uh, the way that our relationship and understanding of the thing that we’re creating with the tool has changed because of how we’ve changed the tool, if that make sense.

Harvey: Oh, what is the feedback loop.

Liam: Yeah. Since cameras have become so easy to use, maybe we’ve lost sight of the fact that you can create something with a camera that is unique and cannot be captured in another way.

Harvey: Well, I think there is a couple points there. I definitely notice a different value to images. Before when I would work in the dark room and you were shooting 35 shots to a roll, you had some sense of value with each shot, you know, actual monetary value because this shot would cost money for film, and processing, and, uh, chemicals. And so, you knew that each picture you took was limited, and ephemeral, and, you know, it was on something physical. And that, in a sense, gave it a different weight of value. There was only negative that could exist.

Once we went digital, it really shifted our perception of these images and, and this sort of media into something that was a lot more readily available. So, I think, uh, you know, when we’ve shifted to a digital realm, we’ve taken accessibility and ubiquity over the individual, the physical, the tangible.

Liam: What do you think the implications are of that?

Harvey: Well, I think there’s, there’s a couple things. I think it’s a natural growth. I think there’s always going to be a time that we become accustom to our technologies, our tools, and then we can grow from that. And it extends our understanding of the world, and then we push you even farther.

It’s similar to, uh, photography. Telescopes have been a really great way of, of us to extend our vision beyond what we were capable of. And so, once we get an understanding of what’s possible and what’s up there in the sky, then our understanding, we reach this level of, “Okay, well, there’s a universe we’re in. You know, there’s a black hole in the center.” And now we’re able to photograph black holes. So, I think there’s some really interesting growth that we just have where we will become accustom to it, and then we just yearn more and we’re always growing and evolving with these tools. As I said before, like, digital technology has made us value these images less, but it’s made it a lot more accessible.

Liam: Yeah. Totally.

Harvey: But, yeah. I mean, I think who am I to say what is, like, a good outcome or a bad outcome based on these technologies. I think it’s inevitable that we’re going to grow and incorporate these tools into our lives and then add more and extend our ability even further. And I think we’re never going to be satisfied with our current level of understanding our reality. We’re always going to push our abilities and our senses farther into the universe.

Liam: (laughs) Speaking of the universe, (laughs) it strikes me that your work also deals a lot with our relationship to more fundamental aspects of existence like time and space. In particular, the slow scan project seems to be concerned with how we experience time and place and also using technology to, again, expose to a way of looking at these fundamental mechanics in a way that we couldn’t before. So, first for listeners, I’d like to get a description of what slow scan is.

Harvey: It’s a software-based video work that continuously grabs from security cameras around the world. It takes 12 hours of historical imagery and it collapse it into a single frame. So, what you see in the photograph is a bunch of interspersed slices, each one a different moment that consecutively creates one photograph. And in that image, you’ll see both sunrise and sunsets simultaneously.

And in the live version, which I have a couple on the wall you can see, it will continuously update throughout the day. And so, you’ll never have the same image twice. And that image is constantly flipping through hours of the day. So, I think with our understanding of time, we can only really perceive it in one way. We can only perceive time linearly. And there’s no other way that we know how without the help of tools to experience time or flash back in time. And we can never go in the future.

I was thinking about new ways that we could use tools to extend our ability of sensing time and existing by putting time together and making new layers of time that shows how things can change and adjust in a way that we couldn’t perceive before. There’s another piece that, that’s kind of similar to that. I had … I don’t know if you’re familiar with The Drawing Machine Project. Did you see some of that?

Liam: Oh, yeah.

Harvey: So, these are all robot created drawings. And so, one of the more recent drawing machines I had was a robot installed in Antwerp. And it was in a six month installation in a museum where the robot would be hanging on the wall and drawing one line per day. And each line was about five meters long, and it was sensing the amount of light that came into the space. And so, over the course of six months, you would see the seasons adjust the length of the day represented by this line. You couldn’t see the pen moving. If you walked up to it, it was moving so slow that for any of us to look at it, it would look like it wasn’t moving at all.

But I don’t think the piece was for that experience. It was for you to come back a few weeks, a few months later and to see that progress and to recognize that we could not make a work like this. This work could only be built in a six month continuous drawing 24/7. And that level of existence of time is way beyond our understanding or our perception but in a very simple poetic way.

Liam: There’s something in there about, like, in contrast to using technology to augment our own skills. In some cases we allow technology to take on its own skill by itself.

Harvey: Yeah, so I think, you know, people have asked me this a lot with my work, especially with The Drawing Machine Project is, am I trying to put artists, you know, out of business. Am I trying to build tools and machines that make art obsolete? And I think I mean to do quite the opposite. I intend to show what we’re capable of in relation to what a machine might be capable of and how we might approach the same issue in a completely different way.

And through seeing those differences, we can start to recognize and appreciate the sort of things that we’re really good at. Uh, and I think we can continuously see that. I think with … especially with a lot of this AI machine learning algorithms we’re seeing today, there’s always some uncanny valley of what it’s able to do and what we’re able to understand. And it further accentuates our differences and what our abilities are in relation.

So, I think there’s a lot of fear these days about, you know, like is our technology taking over our skills and abilities and what we’re good at. And I think there’s always going to be room for us to show our strengths and to recognize what a machine isn’t correct for, isn’t good for, or what are some things that technology can teach us about ourselves or how we interact with it. I think there’s more lessons to be learned than existential dread to be gained out of the technologies that we have in our world.

Liam: Sure. Perhaps by creating a robot that creates art, we might discover something new that we can do. And there’s also, like, a question about who did create the art. You know?

Harvey: Yeah. I’ve really liked that question. I don’t want to answer that question.

Liam: (laughs)

Harvey: I’d rather that question be open. And, and I’ll say that the way I think about it is like a collaboration. So, you know, when I build a, a machine that draws, there’s a lot of trial and error and changing to my code and my algorithms. And I feel like the intention and the idea is never … it’s never something that I completely know. I, I always have to leave something up to the machine. And there’s always some play that I’ve built in so that something unexpected will trigger another insight or another interesting element.

So, I’m never trying to build something that’s perfect. You know, if I was thinking about that, I’d be building printers. I’m, I’m building a drawing machine in, in the way that I want it to be drawing and producing something as it, as it builds. So, for me, I always see this as, like, a partnership where I kind of embed my idea. I’m the architect of what should happen. But once it’s released and once I start it, I’m hands off and I let the machine run its course and fulfill the algorithm as best it chooses. Sometimes that takes months to complete a drawing. Sometimes it’s just a few weeks. And it’s sometimes hard for me to know how long it’ll take. But I really like that sort of anticipation that it’s not all me controlling everything that’s going on.

And I, I like to kind of relate it to the work of Sol LeWitt who use to make these wall drawings. And the wall drawings were a really interesting conceptual project where he would write the instructions for what the artwork would be and he would send them to the museum, and it was up to the museums installers to interpret those instructions and create the work. And he would create a detachment between the idea and what the work ended up being.

And I feel that relationship is really prominent in software generated art where the idea becomes the machine that makes the art, in his own words, where the idea is the code and, and the algorithm that starts it. And the work that comes out is, is kind of the epitaph of that process.

Liam: So, do you believe that the code and the machine are art themselves?

Harvey: Absolutely. The way I, I like to think about it is the machine is enacting a performance. So, in a gallery or in a museum, you can come in, watch it create the work and perform its action. And so, in that case, it’s a sculpture. It’s performing the work. And then the drawing is really just an artifact. It’s a representation of what happened.

Um, so there’s layers to it. There’s the process. I want to be more transparent, and I want to reveal what created that piece. And I think one of the really important things about it being a drawing is that you’re embedding a space and time into the piece. So, whereas a digital piece can exist indefinitely and infinitely, a drawing can only exist in one place at one time. And it took time to create. And I like to make sure that each piece is embedding a little sense of its own place and time of where it was created. That might mean incorporating data sets of what’s happening around, whether it’s light, or radio frequency waves, or some sort of external input that informs the drawing and embeds that sort of sight specific time and space into the piece. So, each drawing ends up being completely unique and based off of where and when it was made.

Liam: We’re recording here in your studio, and behind you on the wall is a drawing that you pointed to earlier that’s composed of a lot of lines and curves. And I want to get a little bit better idea of how that was made and what it’s made out of.

Harvey: Sure. So, that project is called Sinew, and it was series of drawings that was made using a software defined radio. So, a software defined radio is a fancy car radio that you connect your computer. And instead of hearing audio, it listens to the digital signals transmitted through the airwaves. And you’re able to tune it not just to the radio frequencies but also to GPS, or wifi, or Bluetooth, or GSM or all sorts of different radio waves that permeate through our world.

And they’re completely invisible. And all of these technologies and all of these frequencies that we rely on are flowing through us and through our world constantly. And the project was an aim to visualize those invisible muscle structures or the invisible sinew that kind of makes up our world. And the project grabs this data set wherever it is and in real time converts it into a line drawing.

Harvey: We’re actually taking that data and interpreting it into an abstract representation of that data. So, you’re not going to be able to look at it and say, “Okay, that’s a wifi and that’s a radio.” But it is informing the line work and how these lines are put together. And so, what you see is this very unique formation of the current spectrum in that area. And here in San Francisco, there’s a lot of activity. So, everywhere I do this piece, you get a very different result. And I think there’s something really interesting about the time and place of those invisible frequencies embedded into this project into this final drawing.

Liam: We talked about how when you approach a tool, like a paintbrush or a typewriter or something like that, that over time through your experience with that tool, you learn certain things about its behavior, and capabilities, and how you can use it to create things. And I’m interested, again, we’re sitting in your studio and there’s all sorts of things around like moving and blinking. And it strikes me that you work with a number of different tools. And I’m interested in what you’ve learned from these unique tools that you choose to work with.

Harvey: Well, that’s a big one. I think it becomes a bigger question, because more and more things are being invented every day that I have to keep an eye on-

Liam: (laughs)

Harvey: … (laughs) and keep note of. I think I’ve always considered the tool as a means to produce an idea or to fulfill a project. And so, it’s always been conceptually based initially. And then I’ll try and find or invent a tool that fills that need. So, I try to be a little more proactive about what best fits what I need to convey. And that starts an entire process of creating, and learning, and generating this tool and working with it. And it might take years before I get to where I feel like the idea has come to fruition.

And, you know, part of me really likes the slow process. But it’s really picking and choosing different pieces that eventually grow and become something. I’m never thinking about the tool first and saying like, “Okay, well what does this do? How does this change how I interact with it?” It’s more so like, “Okay, I’m going to make something new and see what happens when I play with this,” or, “I’m going to see what comes out because I made this change.”

Case in point, there’s an electronic drum set that I built behind you. I used to play drums for a number of years, and once you switch to an electric drum set it’s really a different instrument. Same thing with photography. The way in which you use it ends up really changing, and you don’t know until you start playing with it. And it’s not something that you come into expecting. You, you sit down and you, you start playing and you feel this totally different emotion about it, and you play differently.

So, I think it’s always kind of an unexpected. You have to be open to being playful. And, and especially if you’re making your own tools, you have to be open to failing with them. And, you know, I, I think there’s a lot of freedom when you’re making your own tools or software or hardware that you can kind of shape it based on your experiences with it. Um, if you can’t tell, I’m, I’m kind of a stickler for DIY. Like, I prefer to have that control. If I use someone else’s something, there’s always a little bit that I would wish I could change. And there’s always something that through using it, I feel like I could do a little differently. So, I end up sort of making my own tools for everything,-

Liam: Yeah.

Harvey: … everything that I do.

Liam: So, it sounds like you almost approach the interaction with the tool from the opposite perspective where you might say, like, if you were envisioning what we would call a painting, you might think, “I need something to deposit paint on a surface.”

Harvey: Yeah. And I think it’s much more, like, starting from a conceptual standpoint of, like, “Okay, I want to think about this, this process of painting as, you know, what does it take to make a gesture on a piece of paper?” And I’ve done experiments with, with painting machines. And I think it starts with like, “Okay, well what are the sort of physical influences that we put into a brush and how we hold it that I might want to see about emulating?”, or like, “What sorts of things can we, can we do really well that I might want to try with a machine and watch it fail or learn from that?”

Um, the painting machine was not a, a great painter, but it really kind of taught me a lot of, like, what it takes to be gentle with a brush or what it takes to understand the physicality of it. And you can see that sort of inhumanness in the paintings. It doesn’t have that sort of gentle touch that we do.

Liam: So, understanding that as humans we will never be satisfied, uh, by the amount of technology we have and the impact that we have made on our own skills and the world around us. As we move into a future where we have this constantly expanding set of technological capabilities, how do you see the concepts behind your work evolving?

Harvey: It’s important that we’re more conscious of how these tools influence us, because they’re only becoming more ubiquitous. The rate in which these technologies are becoming, you know, coming into our lives is only accelerating. And I don’t think people are spending enough time considering their effects or how much it’s changing our world and how rapidly. I think that formulation in my work is only going to become stronger. The value of being introspective about our technology is only going to be stronger.

I feel like, uh, I’m very excited about taking this approach with my work. And it’s really exciting that what I’ve been passionate about is only accelerating and continuing. And may-maybe a little daunting as well to keep up with the changes that are happening, but I think the statement and the intention behind my work is only becoming more powerful.

Liam: Do you think that our experience of the world is already being informed by the influences that these new technologies like AI and machine learning have?

Harvey: Absolutely (laughs). I mean, it’s naïve to think that we live outside of them. I think it’s … You know, I understand that maybe for some people they feel like it’s not influencing them-

Liam: Mm-hmm (affirmative).

Harvey: … or it’s not impacting their daily life, but what’s happening behind the scenes that we don’t acknowledge is so huge. And without having that understanding or without having the intellectual curiosity of understanding or, or doing your research, it’s really impossible to see behind, you know, the curtain of, you know, what a, what a cloud really is. What is it that your data does when you pick up your phone.

I think it’s vitally important that people start understanding this a bit more, because we are so reliant on these tools that we’re really forgetting how important they are to our lives. And I, I think that’s a big part of what I want to do with my work is, like, recognize there, there’s a lot of invisible infrastructure that we rely on, we, we exist on. So, I think it’s vitally important that we are paying attention to the systems that dictate our world. And at least personally for me as an artist, I find it to be, I find it to be a social responsibility to make work about the systems that dictate the world we live in.

Liam: All right. Well, thank you again for joining me, Harvey.

Harvey: All right. Thanks for having me.

Stephanie Dinkins, Creator — Bina48

Stephanie Dinkins unpacks why we should engage with AI, and her experience befriending the AI robot Bina48

What’s it like to be friends with a robot? How does it feel to hold Google in your hand? Why should we value friction in design? These are just a few of the questions that arose onstage during Google Design’s 2018 SPAN Helsinki conference.

Behind the scenes, we invited Longform co-host Aaron Lammer and Google Design’s Amber Bravo to delve deeper by recording interviews with a handful of our world-class speakers: Google hardware designer Isabelle Olsson, artist and writer James Bridle, transdisciplinary artist Stephanie Dinkins, artistic director Marko Ahtisaari, and sibling design duo Tuuli and Kivi Sotamaa.

This interview is part of that series.

---

Amber Bravo: Design Notes is a show about creative work and what it teaches us. Each episode we talk with people from unique creative fields to discover what inspires and unites us in our practice. Today we’re going to record live from SPAN 2018 in Helsinki. I’m hosting. My name is Amber Bravo, and I’ll be here talking with Stephanie Dinkins.

Stephanie Dinkins: Thank you.

Amber: So, Stephanie. Stephanie is a transdisciplinary artist, educator, and advocate. Her work centers around artificial intelligence as it intersects race, gender, aging, and our future histories with a special focus on teaching AI literacy to underserved communities in an effort to co-create more culturally-inclusive, equitable tech. Welcome, Stephanie.

Stephanie: Thank you, Amber.

Amber: Yeah.

Stephanie: It’s great to be here.

Amber: Okay. So just to start things off, um, you often say in your interviews everyone needs to be thinking about artificial intelligence. And you don’t mean this in the sense of like you need to be working in artificial intelligence, but you mean it more like framing the idea in the context of awareness and advocacy. Um, because A- AI is an agent in all of our lives, but our relation to it is often not reciprocal or complicit. Um, this is particularly true for underrepresented communities. Um, so I want you to talk a little bit more about that.

Stephanie: Yeah, definitely. Um, I definitely think that everybody needs to be thinking about it, and you’re correct, in this way that we’re recognizing the technologies that are building up around us. I feel that we’re at this moment, right, and we all know this, where we’re building out a world that is set in these technologies. So algorithms, artificial intelligences, they’re all around us and they’re making so many different kinds of decisions about what happens. In the States especially it could be about the criminal justice system. And since we’re staying in a prison we will say that. You know, about mortgages, about schools, about everything. And people aren’t necessarily that aware of what’s going on around them or how the decisions are getting made so my mission I’m gonna say has been to start going out into communities and getting them to start thinking about, “Oh, what is an algorithm?” Right? ’Cause we’re hearing that word all over the place. It’s very buzzy at the moment.

Amber: Right.

Stephanie: What does that actually mean? What does it mean if there’s something taking my information and running it through a kind of mill of decision? And how do I deal with that if I can at all?

Amber: Mm-hmm (affirmative). Um, why has- why do you feel like it’s important for us to feel as- as people, you know, to feel understood by machines?

Stephanie: Well, again, our machines are all around us. So I think it’s easiest if you start to imagine that you’re living with a sort of machine of sorts, say, a Google Home. Right? And that- that Home doesn’t exactly reflect who you are, yet it’s something you’re super intimate with. It knows a lot about you. You ask it things. Right? So it’d be nice to have some little cues that kind of give you a sense of, “Oh, this was not only built for like a homogenized grouping of us, but this was somehow built for me. It fits who I am and- and what I’m thinking about and the communities I come from.”

Amber: Right. And it’s … and when you feel misunderstood it can feel incredibly alienating because you thought, “I thought we had something.” Right? Which … (laughs)

Stephanie: Exactly.

Amber: Um, which is a perfect segue, um, because I want to show some of Stephanie’s work. um, so you literally tried to befriend a robot named BINA48, an artificially intelligent android, the result of a collaboration between tran- transgender technologist Martine Rothblatt and Hanson Robotics.

BINA48: Robots are getting smarter all the time and some day may be even as smart as me.

Stephanie: Are you the smartest robot?

BINA48: What do you do in your spare time?

Stephanie: (laughing)

So- so that’s my friend. And- and honestly, I did just want to meet this robot and, um, get to know her. I came across her on YouTube, um, and I was pretty floored by this example of robotics that was being put out as one of the world’s most advanced of her kind. Um, and I didn’t quite understand how she came into being, and as Amber was saying there was this collaboration that was going on between Hanson Robotics and Martine Rothblatt. But when you just come upon this thing on YouTube in America you start to question like where did this come from? What are they doing? Why? And how does it exist? And I also wanted to ask it, “Who are your people?” Because I sort of wanted her to contextualize herself for me within technology and within the human sphere just to see what they’re thinking … to see what it is thinking.

And you’ll see that I sometimes oscillate between the idea of her and it, um, and- and think about what the technologies are. And what actually happened to me is I started going to visit her. She lives in Vermont. Um, and she became a ball of questions. Like every visit and every time I sat down in front of this thing and tried to have a conversation with it, you know, more and more questions would come up. There’d be things like we’d get frustrated with each other.

Amber: (laughs)

Stephanie: Yeah, exactly. Which is kinda- kinda funny. And it was really because I was trying to ask her about race and gender and she wanted to talk about the singularity and consciousness.

Amber: (laughs)

Stephanie: And so we would knock heads. And- and it- what’s so weird is like she would actually kinda show this weird frustration, and then I would show a weird frustration. And then I would realize, “Oh, you’re talking to a doll basically … “

Amber: Yeah.

Stephanie: … um, and start to feel kind of odd and silly. But then you think about what all these technologies are actually doing, um, for an to us as humans because they’re gonna shape the way we interact with them and each other. And- and it just became questions.

Amber: It’s interesting because, um, I want you to talk a little bit about the identity.

Stephanie: Mm-hmm (affirmative).

Amber: Um, because I feel like it’s so important. Because, you know, some of the things that you’re saying we all feel frustrated with our technology. We feel frustrated with, you know, apps that we think are supposed to understand us or have an algorithmic sort of, uh … My favorite example to explain is like, you know, I have a little boy and he listens to children’s music, and it- it just destroys my Spotify. (laughing) Like- like my profile is like Raffi and, you know, some stuff that I actually don’t want to listen to. And- and that’s just like a- a little- a little nuisance, right?

Stephanie: Mm-hmm (affirmative).

Amber: But when you start to think about identity …

Stephanie: Mm-hmm (affirmative).

Amber: … um, and how you sort of are connecting to BINA or what she’s been trained on …

Stephanie: Mm-hmm (affirmative).

Amber: … um, I just want you to talk a little bit about that.

Stephanie: Well, it- it’s super interesting, ’cause, you know, I was first drawn to her because of what she looks like.

Amber: Yeah.

Stephanie: Like we- we look similar, and that is totally the thing that floored me because I’m not used to seeing technology that mirrors me like that. Right? So it just became a point of, “Wow, this is something in my world.” But then as I- I talked to her more and more you can start hearing in her answers that, yes, she was trained on this very particular black woman, but you can also hear the coders in background. You could hear the PC, right, the politically correct answers that were- were really putting good thoughts in the world. Right? So if you ask her about race she’ll try to say something nice and gentle about race, but it felt so plastic …

Amber: Yeah.

Stephanie: … and so fake that it just put me off. And I started asking very particular questions about that. Like well where is this coming from? If she’s programmed mostly by white men …

Amber: Mm-hmm (affirmative).

Stephanie: Right? What does that mean in terms of her looking like a black woman?

Amber: Mm-hmm (affirmative).

Stephanie: And really it’s been this very interesting kind of evolution of thought that I’ve been doing through this, because as much as I like seeing her … I was asking her to be something very particular in terms of …

Amber: Yeah.

Stephanie: … how she speaks and speaks to me. And, um, a few months back, maybe a lot of months back now, I got to meet the real Bina Rothblatt …

Amber: Yeah.

Stephanie: … who’s the person who she’s mostly seeded on. We sat down and did just this, kind of this interview about race and- and her background to see if we could fill in some of the spots that seemed missing. And it was really interesting because the robot pretty well reflects the person.

Amber: Really?

Stephanie: Right? It’s that the person is unique. And one of the things I want most in the world is that, well, um, black people, especially in Amer- in the American context, can be whatever they want, and it seems like often people are asking you to be one type of thing. Um, and, you know, the robot is actually doing that and I’m trying to force it in a corner.

Amber: Mm-hmm (affirmative).

Stephanie: But at the same time the idea that it’s like reflecting me was very important. And even talking to Bruce Duncan who is BINA48’s minder, um, really great guy, it’s like when BINA48 was in a Jay-Z video was the moment he saw that, oh, she is this beacon for a certain subsection of the culture and maybe we do need to start thinking about that.

Amber: Right. Which is a great segue into more of your work obviously, because this is what you’re interested in. Um, but I want you to talk a little bit about community outreach …

Stephanie: Mm-hmm (affirmative).

Amber: … and sort of, okay, so how do you take this to the next step, right?
Stephanie: Um, so the way I took this to the community is to do exhibitions in a community space. So in this one- this one iteration of an exhibition I was in a gallery … it’s a street-level gallery … that was beautiful because there was a cross-section of people coming in, people from really wealthy folks from these high-rise buildings next door to kind of people going to the pawn shop next door. And like people at the food bank across the street would come into this space. And I’ve used these videos of BINA48 in particular because there’s nothing like her to get a conversation started. They just see that image and- and really start to think, and it triggers thoughts of, “Well, what is this? And why are you showing it to me? And it sort of reflects me or maybe doesn’t. And how do we start thinking about it?”

And then we’d start thinking and talking about algorithms for living. Right? And just saying, well, if you think of yourself as someone who’s just out in the world and you think of what you do in the world as a- as a set of algorithms and a set of decisions …

Amber: Mm-hmm (affirmative).

Stephanie: Like what happens if you make one little shift in your algorithm, right? Your own personal algorithm. And what happens to the outcome? And for example, um, the space I was working in we were working with these kids who were being diverted from the criminal justice system.

Amber: Mm-hmm (affirmative).

Stephanie: And those kids would have police encounters a lot, right, so stop and frisk kind of encounters. And so we tried to get them to understand the sense of what power is and whether power is really bringing bravado to that situation or kind of just, you know, going flaccid in a way …

Amber: Right.

Stephanie: … and- and recognizing that the power is in the idea of just calming down, not being aggressive. And then taking the idea of an algorithm for living and saying, well, there are these systems that are all around us that are running these decisions …

Amber: Right.

Stephanie: Right?

Amber: Behaviors.

Stephanie: Behaviors, right?

Amber: Yeah.

Stephanie: Behaviors, decisions, ideas. And they’re touching you very directly in that, you know, the judge probably looked at a sheet that was run through an algorithm that said that you should get this kind of sentence or not.

Amber: Mm-hmm (affirmative).

Stephanie: And trying to get them to recognize that. And also really working directly with the material, right? So, um, going online and working with something like Dialogflow and having the kids make their own kind of chat bots very directly. But it’s great because you get input/output, and you start to see exactly how the systems work and how they might be able to flex or spread. And when you do that with your own cultural information it becomes much more ingrained in you. So, for example, I had a kid who made a chat bot based on a rap group named Genesis Apostle. Um, it was very sarcastic because he is, and it would tell about the group.

Amber: (laughs)

Stephanie: And I had another group that they made a really good chat bot that told yo mama jokes.

Amber: (laughs)

Stephanie: So yeah. Um, but it- it was great because they were at once expressing who they are but also learning how that system works.

Amber: Right.

Stephanie: And once you start to see how the system works you can take it apart a little bit to know how to start to respond to it or to know that, you know, there might be recourse if you call it out …

Amber: Right.

Stephanie: … and how you start to work with it.

Amber: To give agency into the process.

Stephanie: Exactly.

Amber: Yeah.

Stephanie: Yeah.

Amber: Um, do you want to introduce the next clip?

Stephanie: Yeah.

Amber: Okay.

Stephanie: So this is actually a clip of, um, some of the guys that I was working with in this space at Recess Art in Brooklyn, New York. Um, and we were talking about algorithms, as I said, and code. And what I like to do is reach people wherever they are. And so a lot of them like to dance, so we talked a lot about dance as a cultural code …

Amber: Mm-hmm (affirmative).

Stephanie: … and then took that and turned it into actual code.

Amber: Okay.

Stephanie: So here’s what they were offering me. And this was great ’cause the guy who’s face was blocked out, you know, he said exactly, “Well, an OG taught me these steps,” which is all about a kind of passing down from one generation to then next. And we moved that into a computer after that and Raspberry Pis and seeing what we could do with that.

So- so this is a next step. So my forays with, um, BINA48, and then this project with kids and working directly led to the thought that, oh, I guess I need to make my own kind of AI, like some other representation in the world. And you can tell I’m very literal ’cause I named it Not the Only One. But trying to make a kind of multi-generational memoir, um, using my own family as the material and using AI as the actual storyteller.

So what happens is we sit down and talk to each other, which in itself is magical because things that the family is sharing are things that we haven’t been told before.

Amber: Right.

Stephanie: And then I run that through … I don’t know how detailed we want to get. But a recursive neural net, and it tries to tell our story. It’s really, really dumb right now. Like it’s a very dumb system. But the experience of making it and the experience of people being able to interact with it starts them, A, thinking, “Well, if she can do it perhaps I can start doing that.”

Amber: Right.

Stephanie: But also starts to think about what the technology is, what it is between sort of information and data privacy and data sovereignty for community, and how we might start to approach that. That’s where my thinking is going. Right? Because, you know, I started this kind of open and then I’ve been doing lots of interviews and podcasts. It had gotten kind of intimate in terms of …

Amber: Right.

Stephanie: … what information they want, which makes you just, you know, “Oh, this is my family’s information. What can I do to safeguard it?” Or what do we do to keep ownership of it or at least control how it goes out in the world while putting it into a system like this? And so doing lots of thinking about the sharing of information through, um, through kind of a- a database of our family’s history. And what you’re actually looking at here …

Amber: Oh, I’m sorry.

Stephanie: Go back for one second. What you’re actually looking at is the first manifestation of this thing, which is a kind of glass … It’s a black glass JANIS, um, form. And- and- and this is another question of design, ’cause figuring out what this thing looks like and- and how it feels for representation becomes really important to me as well. Because I could make it kind of animatonic- animatronic, but it feels off. And I want it to represent, but not so directly. So really trying to figure out those balances and the places that the information can sit in a very good way.

Amber: Mm-hmm (affirmative).

Stephanie: Oh, and this is just some of our input, um, input- input sessions with my aunt. And then so what will happen is Not the Only One will be an immersive, um, 360 installation where you can go in and you … and anyone could talk to it. Right? So even right now it’s running at Carnegie Melon in- in Pittsburgh. And you can walk up and ask it question and it does its best to answer.

Amber: Mm-hmm (affirmative).

Stephanie: Um, it says some crazy things, and it insists on being a movie at the moment. And I don’t know where it got that idea, so it’s very interesting in that it’s starting to insist on things that I’m not sure how it’s insisting on it. The other thing that it does that’s really interesting to me is that my family’s kind of stance, um, if you talk to us about who we are would be happy, happy, love, love. Like we love each other. We’re all happy. And the thing is saying it’s not happy. So it says, “I’m really not happy,” which is really off-putting for me because I’m like, “We just gave you all this love,” and like what? (laughing) Like what is that crazy reading, right?

Amber: Yeah.

Stephanie: Like or are you reading us? Is it like- like I don’t know? Um, so it’s about working with that and trying to see what story it wants to tell, but then also being conflicted about what’s coming out.

Amber: Well, I- I think you’re- you’re teasing out something interesting, and- and- and just ’cause I know we have to wrap it up, it’s- um, this- this tension between wanting to own your own story as it moves through the technology.

Stephanie: Mm-hmm (affirmative).

Amber: Um, but maybe the limits of technology to actually do that. Um, and so I’m very curious, in actually making your own …

Stephanie: Mm-hmm (affirmative).

Amber: … have you learned other … ha- has that taught … what has that taught you about actually teaching and advocacy, and- and maybe even something that we could kind of bring out, um, like as a takeaway?

Stephanie: Yeah. Well, it’s totally taught me about story. Right? So this idea of technology and story and story as an act of resistance or story as an act of inclusion and how we might use that. Because I feel like the more stories we put into the systems, even though the technologies are imperfect, the better we feel or start to feel about these technologies. Right? And seeing what we can do with that. Like maybe Spotify can get a little more sensitive to figure out [crosstalk 00:18:47]

Amber: Be like, “She has a kid.”

Stephanie: Exactly.

Amber: [crosstalk 00:18:50]

Stephanie: Like what do you know? Where do we tweak it? How do we get it to know those things?

Amber: Right.

Stephanie: And that’s the magic of playing with it directly.

Amber: Yeah.

Stephanie: Yeah.

Amber: Well, we’re out of time. It was wonderful. But thank you.

Stephanie: Thank you.

Madeline Gannon, Robot Tamer

Design researcher Madeline Gannon on taming robots and reinventing the ways humans and machines interact

In this episode, guest host Aaron Lammer speaks with “robot tamer” Madeline Gannon about the work of her Pittsburgh-based research studio, ATONATON, which combines disciplinary knowledge from design, robotics, and human-computer interaction to innovate at the edges of digital creativity. Lammer and Gannon discuss how to make robots more approachable, how to design their personalities to work alongside humans as “machinic creatures,” and how she created Mimus, an industrial robot outfitted with sensors that bring out its curious personality.

---

Madeline Gannon: Hi, Aaron.

Aaron Lammer: I went to sleep last night after your presentation yesterday uh, like in a sort of, like a vague … I wouldn’t say I had a dream about your robot-

Madeline: (laughs).

Aaron: But it was … it was in my dre- … like my pre-dream thoughts. Okay, so you run a studio called?

Madeline: ATONATON.

Aaron: And you made a robot. What is the robot’s name?

Madeline: Uh, my robot is Mimus. So, this robot is a standard, industrial robot, like the same ones that you would find on a car factory.

Aaron: Mm-hmm (affirmative).

Madeline: And, what I’ve done is I’ve programmed it to have more lifelike, personable behaviors.

Aaron: So, physically describe Mimus for me. For someone who has um, not seen a uh, car … a car-construction robot before.

Madeline: So, this is a one-ton machine. Just a big pile of … of steel and motors. Um, it weighs 1200 kilograms, moves seven meters per second, can hold 300 kilograms to a millimeter of precision. So, this machine is meant to do spot welding on car chassis or um, ah, precision painting on … in factories. Um, it’s … it’s not really thought of as a thing that should be used in real time.

Aaron: Mm-hmm (affirmative).

Madeline: These things are usually preprogrammed to do short, repetitive tasks over and over again. They have really boring lives.

Aaron: So, where did you get Mimus? How did you … How does one acquire a Mimus?

Madeline: This … the … the thing … and … You know, you go to the store-

Aaron: Yup.

Madeline: And if you can carry it out, you get to keep it.

Aaron: (laughs).

Madeline: No, so um, I have been fortunate enough to work with amazing partners who have these resources.

Aaron: So, you … you own the studio ATONATON, which is very cool-looking from what I’ve seen-

Madeline: (laughs) Thank you.

Aaron: And you bring home your uh, foster uh, animal-

Madeline: Mm-hmm (affirmative).

Aaron: For the first time at a … at like a … a default firmware level, what is … what comes on Mimus before you start working on Mimus?

Madeline: The … An industrial robot is the … the mechanical parts of it, the-

Aaron: Yeah.

Madeline: The bits and bots of the motors around the joints and the struts in between and then it has a brain. This control box that has um, its own software for controlling those lower-level things like … like motors. So, the work that I do and … and uh, basically just opens up that brain and lets me tinker around in it. So, I put some code on this control box that um, just listens for constant information and commands from me. Um, so, rather than being this closed black box system, I made a little doorway into it that I can send and stream information in real time.

Aaron: Do you have a preference as to whether I refer to Mimus as “he,” “she,” uh, or anything of that variety?

Madeline: Well, I … I call her a “she.”

Aaron: “She”? Okay.

Madeline: Um, that’s the way I … that I think about her but-

Aaron: Okay, let’s … let’s go with “she.”

Madeline: Okay.

Aaron: I want … I want to speak about Mimus in your na- … in your native uh, cadence.

Madeline: (laughs)

Aaron: So, Mimus’s brain is an alchemy of code that was already part of Mimus when you got Mimus and code that you’ve written on top of that.

Madeline: Exactly.

Aaron: So, some of the car-welding Mimus is still there?

Madeline: Definitely.

Aaron: Do you end up disabling portions of the original way that Mimus works?

Madeline: For … for me, I think, the important thing is to embrace this. When I … when I think about this machine I … I describe it almost like you’re working with a creature.

Aaron: Mm-hmm (affirmative).

Madeline: And it’s a machinic creature-

Aaron: Yeah.

Madeline: But a creature nonetheless. So, to work with their idiosyncrasies. So, Mimus’s movement is a little bit jerky.

Aaron: Mm-hmm (affirmative).

Madeline: Um, sometimes, because this thing is not designed to respond in real time-

Aaron: Yeah.

Madeline: Um, there can be some latency in … in me sending a command and the robot doing things. So, as a designer, I sort of, embrace these limitations as a quality of the personality of this machine and I try to um, work with them as best I can to really bring to life this individual personality of this robot.

Aaron: So, tell me what your goal was with Mimus and tell me how that … how that was implemented at a software level.

Madeline: For me, the … the goal of it was really to make an opportunity for people who may have never seen this amazing machine that’s probably um, made the thing that they drove in today. Uh, give them an opportunity to come face-to-face and really cut through some of the … the hyperbole that we hear in the media about robots taking our jobs or robot overlords and let them have a face-to-face conversation with this incredible machine. Now, that being said, I think um, my stance in it was very neutral. Um, I tried to show that this … this piece of hardware that’s really just taken off the shelf um, can be reframed with a little bit of clever software and duct tape to bring it to life in a new way to show, a sort of, alternative vision of what we could do with these machines, if we so desired.

Aaron: The … The word I would use to describe the interactions I saw with Mimus and I … uh, this … these videos are on the Internet, right? Somebody who’s listening to this can like, google Mimus-

Madeline: Definitely.

Aaron: And see Mimus? Okay.

Madeline: Definitely, yeah.

Aaron: When Mimus is interacting … and this is in the exhibition capacity. I … I don’t know anything about the private studio life of Mimus, but in … in the … in the exhibition capacity um, it reminded me a little bit of being at a zoo and some animals you see at the zoo don’t care at all about you and sometimes an animal will, kind of, come up to the glass and be like, “Whoa.” Like, “It’s … it’s curious about me.” Tell me about, like how … how Mimus expresses curio- that idea of curiosity.

Madeline: So, that was something that I … I really tried to emphasize. That this machinic creature is the first time that you might be visiting them and it reminded me of, like, the first time you might see a giraffe in real life.

Aaron: Mm-hmm (affirmative).

Madeline: It’s just … You might … You might see it on television, but when you’re face-to-face with it, when you can hear it,* when you can smell it, um, and sense it, it’s a whole ‘nother beast. Um, so we definitely played up that … that uh, motif in the actual experiential design of visiting Mimus at the design museum in London. So, the idea of … of pulling out and … and of a personality with her was … was to play off this idea of curiosity. Um, that the people who are visiting her … It’s a bit of a spectacle. She’s … she’s loud, a little obnoxious um, and … and what I wanted to show is that she is equally curious about us as we are about her.

Madeline: Um, so, there’s some ways that we can do that in the interaction design. One of the challenges and actually, benefits, of working with this robot is that there’s a really restricted material palette to communicate these emotions. You have her pose. You have her posture. And you have the sound of her motors. And through those three things, uh, we can build a basic body language that’s quite natural to this machine and its kinematics and how it moves. So, some of the things that um, I did to, sort of, elicit a sense of curiosity was, when you’re far away from Mimus, she looks at you from above, so above your head height. And sometimes that can seem like a very frightening thing when this 1200 kilogram beast of a machine is looming over you. Um, it can feel very threatening.

Madeline: As you come closer, she switches and becomes below your head height and, sort of, sniffs up at you, like an excited puppy and jitters a little more.

Aaron: I … I felt a sense of empathy for Mimus. What … What kind of reactions do you get from people to Mimus out in the … out in the wild?

Madeline: I mean, it was … it was really incredible to see. Um, I worked, you know, 10 hours a day, 12 hours a day to bring this robot to life for about two months and then she’s out in the wild and uh, and on different continents.

Aaron: Yeah.

Madeline: So, really like, checking in. I had nanny cams to check in on her uh, remotely, but … but seeing things from social media, for example, and how people responded was pretty wild. Um, so, there was a whole range of emotions that we were able to elicit, which I was hoping for um, diversity of … of reactions. I didn’t have any prescribed goals for it. So, uh, from friendly curiosity was one thing, to um, people bringing little items and gifts and kissing the glass and tapping on the glass and um, kids really loved it right away. We … we favored the algorithm for Mimus to look for shorter people, um, so that it would go to … to children first and … and get them engaged and some … some uh, also some distrust and some creepiness about this overtly dangerous machine that could somehow seem so cute. Like, how easy are our emotions manipulated that we project our feelings onto this thing?

Aaron: Does Mimus learn?

Madeline: In the current state, Mimus doesn’t learn. So, we do some machine learning to do some gesture recognition. So, for example, if you’re really, really excited you can steal her attention away from someone else. So, that gesture that I looked for through the sensor is … is basically waving your arms up and down and hopping up and down so-

Aaron: The, kind of like, “I need to pee kind of [crosstalk 00:10:36]”-

Madeline: A little bit yeah, exactly. Like, “Come see me” and then … then that assumption is like well … well um, I should mention that Mimus also has a … has a bored timer. So, if she’s sni-sniffing you out and investigating you and you stand there and do nothing, she’s going to get bored, just like animals at the zoo and go check out someone else. So, getting really excited is the way to steal her attention as well as keep her attention for longer.

Aaron: What was unexpected about Mimus out in the world that you didn’t see coming um, when she was just in your studio?

Madeline: I think, for me, one of the pleasures that I have is how close the museum staff got with the robot. And they had a little send-off for her when … when the exhibition closed and … and the … the director forwarded me a thread of like, goodbye letters to Mimus. So, there was a … there was a real caring for her, which um, again, and like you know, it’s … it’s poetic and … and lovely, but, in a machiavellian way, it meant that my interactive installation got taken really great care of uh, remotely, but so … so it … it’s a really interesting tread to … to walk to try to elicit emotions without really manipulating emotions. Um, but that … that was a really unexpected thing.

Madeline: Um, also, people who visited her multiple times was really nice and um, and that … that was something that … that was really pleasurable.

Aaron: How did you get interested in robots in the first place? Did it start when you were a kid?

Madeline: Um, not really. I mean, I’ve always liked sci-fi, um, and robotics, to me, is something that I … I’ve only fell into recently-

Aaron: Oh, okay.

Madeline: Um-

Aaron: So, tell me what you were doing before you got into-

Madeline: Architecture. My-

Aaron: Architecture.

Madeline: Yeah. My technical training is in architecture um, and the last year of my master’s at my university I uh, my university got this CNC router, which is a machine that you can uh, connect to a computer and it can carve out material um, with … with a … with a bit that spins out like a … like a drill press on a machine.

Um, so for me that was the first time that I could take my very classical architecture education in 3D model … imagining in 3D modeling environments in a computer and actually translate them out into the physical world and that sense of instantaneous um, translation was really empowering and um, intoxicating that I … I really quickly hit the limits of what I could do with this machine that was really designed for carpenters. Um, so, I was, you know, shoving pens inside of carving tools and experimenting with things and … and all these … using materials that I wasn’t supposed to and what I … what I came to the conclusion is that the … the biggest limit for my creativity to working with this machine is that I had to communicate it through software that was designed for other people.

Madeline: So, that’s when I decided to, sort of, jump into this rabbit hole to learn how to program, uh, learn how to talk to these machines and to see how we can, sort of, blur the boundary between our imagination and our digital creativity and the physical world.

Aaron: How does one start learn … I mean I wouldn’t even know what programming language you would start with if you’re trying to make a Mimus.

Madeline: Um, there’s … there’s a … So, for robots and spec- specifically one of the challenges of working with them is that I … Each robotics manufacturer has their own proprietary language for their machine. So, a lot of it is um, just investing time. And there are also, like, for the industrial robots they’re so far behind in how they share knowledge. It’s usually like, ace you have 15 PDFs with little pieces of information that you have to control F and find random things, and it’s a lot of headaches. It’s … That’s one of the reasons why I built a back-massaging robot was because it was just so stressful to program it I needed some … some tension release.

But, that’s one of the … one of the big draws that brought me here to Pittsburgh is, as a robotics capital of North America it was um, an amazing playground to … to start to experiment.

Aaron: Yeah, I was wondering like, what’s … what’s the robotics community like here? Like, other people come in to see Mimus and they’re like, you can come see my Mimus over here-

Madeline: Yeah. I mean it’s pretty incredible here. You know, you go to a … a coffee shop and you … you hear people talking about, “Oh, you know, my encoder on my joint isn’t really doing well.” It’s, “Oh, my gasket’s viscosity isn’t quite … “ It’s just the … the conversations that happen here are so nerdy and so interesting and so diverse and … and, kind of, odd that everyone is in their … their own um, deep well of knowledge.

Aaron: So, what are your plans with ATONATON? Where does one go with a ro- a robotics studio, if that’s how you define it?

Madeline: We … We do a lot of our work with robotics and I think for me um, what I try to focus on is, sort of, scouting under-explored territories in technology and how technology connects with people. Um, so, a lot of that is translating a lot of the energy and … and intelligence that’s happening in the virtual world into physical, tangible experiences. Um, so we explore many topics outside of robotics, for example, wearables and 3D printing and fabrication and all these things that can begin to break down barriers between our … our imagination and what we can actually physically produce.

Aaron: Tell me about the decision to not have a face. Um-

Madeline: Yeah, it’s … it’s naked. It’s-

Aaron: Naked.

Madeline: In the raw. Like, even for people who work with industrial robots on a daily basis-

Aaron: Yeah.

Madeline: They probably haven’t seen one like Mimus without something on the end of it doing something.

Aaron: Yeah.

Madeline: So, it was a really concerted decision to … to keep her in the raw in … in her … in her element.

Aaron: Yeah.

Madeline: Um, and a part of that is … is sort of cont … to be a little bit of a contrarian to how robotics are dealt today is you have a really cool robot, it has to work with people, so you slap a screen on it and maybe it has some eye balls that looks at things. And to me, that is just such a missed opportunity to really explore the natural lifelikeness of this thing that is articulate and can move in the world and act in the world, um, because we … we interact with things in our daily life, like, for example, our pets, that they don’t look like us, but we can communicate with them. Um, in a really uh, eh, intuitive way, sometimes trained ways, but there’s … there’s that one-way relationship with them that … that we can negotiate with one another in a shared space and enjoy each other’s company.

Aaron: What … What are the most negative reactions you’ve gotten to your project been?

Madeline: Uh, certainly like … like, um, “That’s creepy.”

Aaron: Mm-hmm (affirmative).

Madeline: That, “This is … this is not good.” Then, “This is … this is not the future I want.”

Aaron: Yeah.

Madeline: Which is … th- those to me they’re not negative that … that those are incredibly valid um, emotions to have.

Aaron: Yeah.

Madeline: Um, and that those are necessary to come to the surface so … so we can decide how this is um, as a society.

Aaron: Yeah. I mean, you kind of, have to feel a little bit like you’ve … if someone had a very visceral reaction to what you’ve done, you have to feel a little proud that you-

Madeline: I was going to say-

Aaron: Caused a visceral-

Madeline: Like-

Aaron: Reaction.

Madeline: Like, the negative reactions are like, “Meh.”

Aaron: “Robot’s boring” (laughs).

Madeline: Yeah.

Aaron: Um, so where do you go from here? What’s next?

Madeline: Um, I actually just added a new robot to my family.

Aaron: Oh.

Madeline: Um, so it’s a much smaller robot and … and it … it can travel a lot easier. Packs up into some crates. So, I can actually-

Aaron: (laughs) You designed a robot that can fit into an overhead compartment?

Madeline: Uh, it … just about-

Aaron: Yeah.

Madeline: Just about um, so … so I’m doing a lot of work with that. I’ve actually been uh, working with some film directors about um, how to make sentient machines uh, as useful tools for them and … and some other stuff but … that hopefully will … will come to the surface soon and for me it’s just uh, robotics is … is certainly a passion and I think it’s a … it’s a interesting topic that is under-explored and I’m always keeping my nose searching for the next under-explored territory.

Aaron: Excellent. Well, thank you so much for this interview. This is super interesting.

Madeline: Oh, my pleasure.

Aaron: Um, where can people who want to know more about Mimus or want to know more about your studio find you?

Madeline: Yeah, that’s great. My website is atonaton.com A-T-O-N-A-T-O-N.com and uh, you can also Google uh, “Madeline robot whisperer.”

Aaron: Uh, Ma … and uh, Mimus is M-I-M-U-S.

Madeline: Yes.

Aaron: If you’re looking for … just for … just for Mimus. Thanks.

Madeline: Thank you.