The Robotic Gaze

I began the morning by reading a fantastic summation of the male gaze. Not the happiest way to start out, but a subject at the back of my mind. I recently watched two movies at opposite ends of a cultural spectrum: The Godfather, and Cannonball Run II. The latter was so unredeeming I left the room and was bitterly chastised for over-sensitivity. The former had obvious merit and I stayed the course. But it was so male dominated that I didn't feel offended, I felt bored. There was so little I could relate to. My interest was sustained by the beauty of the shots and Marlon Brando's acting, and by my husband's delivered promise of a good parting shot. The only woman who really speaks is Diane Keaton. She's wonderful, but in her two big scenes she's a walking incarnation of this footnote from The System of Objects:

12. 'Loud' colours are meant to strike the eye. If you wear a red suit you are more than naked — you become a pure object with no inward reality. The fact that women's tailored suits tend to be in bright colours is a reflection of the social status of women as objects. - Jean Baudrillard, 1968

It was fortuitous to read this text and see this movie for the first time within weeks of each other. I couldn't stop thinking about it. I love this section of the book; here is what surrounds the footnote:

The world of colours is opposed to the world of values, and the 'chic' invariably implies the elimination of appearances in favour of being:¹² black, white, grey — whatever registers zero on the colour scale — is correspondingly paradigmatic of dignity, repression, and moral standing.

'Natural Colour'
Colours would not celebrate their release from this anathema until very late. It would be generations before cars and typewriters came in anything but black, and even longer before refrigerators and washbasins broke with their universal whiteness. It was painting that liberated colour, but it still took a very long time for the effects to register in everyday life. - p. 30

The good news here is that robots are ungendered. They're unburdened by history, with the potential for an aesthetic unshaped by its influence. Maybe they'll show us that the female human form is objectively beautiful and we were right all along. Maybe they'll show us something different. I really love the photos my robot Nila takes while she's painting, and am trying to figure out how to expand this with Neko.

The viewer never saw the image feed from Nila's camera, because people like looking at their own image, and I wanted them to look at Nila. I think I'll change this with Neko, and am experimenting with different modes of interaction.

To cleanse the palate, here is my favorite piece of art on the subject. The camera is given its rightful place as neutral observer. Below is an interview with the artist by MoMA.

Picture for Women, Jeff Wall, 1979

Learning About Learning

I started Andrew Ng's coursera class on Machine Learning this week. It's fun so far, and I've learned some new terminology to help frame my goals. There are two domains in which I aim to use ML: 1) learning to associate colors with words through an expansive database, and 2) learning to recommend a color given a text prompt. Here's a tidy definition of ML offered by Ng, and how I think it can be applied to both of my domains:

A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P improves with experience E. - Tom Mitchell, 1998

1) In the case of Neko learning from datasets:

   E is the collocation of colors and words in a database.
   T is the clustering and re-clustering of colors with words.
   P is the score of the clusters (how well-sorted they are).

An example of k-means clustering

2) In the case of Neko learning from people:

   E is testing colors on different individuals.
   T is returning a color, given some text.
   P is the number of well-liked colors.

An example of a support vector machine

The categorical names for each are that Case 1 is unsupervised clustering, and Case 2 is supervised classification. K-means is a likely algorithm for the former, and a support vector machine for the latter. Because order is meaningful (Orange is closer to Red than Yellow), color is a regression problem with continuously valued output. But there is a sense in which colors are discrete as well, so that's what I'm mulling over now.

How Color Works

Color is a signal. It's an interaction between radiating bodies, reflective surfaces, and vision systems. Radiating bodies, like a lamp or the Sun, undergo a chemical process that emits energy in tiny packets called photons. These photons travel along wavelengths varying across the range of the electromagnetic spectrum. At the low-energy end of the visible sub-spectrum is red, with a very long wavelength. At the high-energy end of the range is violet, with a very short wavelength. After traveling the path of least resistance the photons hit the retina of the eye, stimulate rod and cone cells tethered through the optic nerve to the brain, and become the phenomenon of light, color, and image.

Twilight in the Wilderness, Frederick Edwin Church,  1860

Consider this Hudson River School painting of a sunset. We call sunlight white but it's technically golden, and this is exaggerated at sunset, when atmospheric haze makes it easier to see the star. The light is generally white because the photons emitted by the Sun are full-spectrum. This does not mean all photons travel at an average value; some photons travel on the red wavelength, some on the green, etc. Each color occurs with a roughly equal proportion, so they blend as beams into white. Church used Lead White (now deprecated) under secretive blends of oil, wax and resin to give his painted Sun internal reflection.

As matter interacts with photons on the journey to Earth, the wavelength dictates each color's reaction. The sky is blue because the paths of the short blue waves are more prone to collision. The blue light scatters easily in atmospheric dust and moisture. Church used a pigment called Cerulean, which was developed for skies, despite being a little too green. Interestingly, if our vision was more accurate, the sky would be violet. Our brain makes it blue. Our art history makes it turquoise.

Clouds are wonderful paint and color subjects. They're generally white, because the water vapor that composes them diffracts sunlight and makes them glow. When clouds are dense with vapor, on the brink of rain, less light escapes and they turn gray. At twilight only the photons on longer waves reach the clouds, lighting them up red like fire. Of course that's a bad analogy because we tend to see fire at relatively low temperatures— reds and yellows are called 'warm colors' while blues and greens are called 'cool colors,' despite the fact that blue light is hotter than red.

The remaining colors in the painting are desaturated but mostly green: the predominant color of plant-life. Terra Verte is a likely pigment. Plants live through a process called photosynthesis: photons from sunlight provide the energy to chemically bond carbon dioxide to water, forming carbohydrates. The subroutine of translating light energy to electric energy occurs in the photosynthetic reaction center: a mesh of proteins and pigments. Chlorophyll is a typical pigment— chlorophyll absorbs the blue and red light, and reflects the green. If a plant's leaf is the color of Terra Verte, that is its resonant frequency. The leaf system stores vibrational energy at that frequency, so the Terra Verte photons are amplified upon reflection. A well-informed painter might also know Terra Verte has been used as an undertone for flesh since the 11th Century, and can subtly anthropomorphize a landscape.

Supplying a Robot with Paint

Paint delivery one of the more challenging aspects of art robotics. Paint is messy, wet, complex, and ruins a brush that's allowed to dry. I use oil paint because it's extremely beautiful, but also because it's a particularly difficult medium that art roboticists tend to avoid.

Spray paint is the most common solution to the problem. (Ink works as well; I'll address that in a future post on drawing.) The first artistic robots, and the ones I consider most successful, are spray painters. They work in factories worldwide, like this trio in Germany:

This is an ideal application for robots. It's a toxic setting that requires an absolutely steady, smooth and consistent hand. Spray paint requires a constant distance from the surface to create an even coat, which is much easier for a robot to calculate than a person. Though the paint won't give them respiratory problems, it can still gum up the works, which is why they're fitted with protective socks.

My 6 month old palette

It takes a lot of dexterity to get oil paint onto a palette. Selecting, grabbing, moving and squeezing the tube, along with several iterations of cleaning the brush when finished. I've tried every mechanical delivery system I could think of (paint rolled out of tubes by motorized rods, linseed oil delivered through a solenoid-opened tube to dry pigment, hanging bottles of diluted paint with mechanized lids). Until my robots have a hand built for more than a paintbrush it's something I'll do for them, so we can keep using my beloved oil paint.

Neko the Modernist

Modernism is a movement in philosophy and art, concerned with self-examination. It begins by discarding all a priori knowledge, and building up a system of evidence-based understanding. The modernist uses their medium to examine the medium itself.

“[Modernist art] happens to convert theoretical possibilities into empirical ones, in doing which it tests many theories about art for their relevance to the actual practice and actual experience of art.” Clement Greenberg, Modernist Painting, 1960

Neko's database will allow him to correlate words to colors to pigments. It's my hypothesis that he'll reveal a wealth of information about color can be used in painting to represent different concepts. I consider him a color field painter.

“Through its fetishization of the base, the sculpture reaches downward to absorb the pedestal into itself and away from actual place; and through the representation of its own materials or the process of its construction, the sculpture depicts its own autonomy.” Rosalind Krauss, Sculpture in the Expanded Field, 1979

Autonomy is the goal as I reconstruct Neko. This will come through 1) a well-balanced feedback system with me, viewers, and the work; 2) a unified, articulate, transportable body and pedestal ; 3) a well-organized archive.

Why teach robots to oil paint?

Some form of this question is the one I'm most frequently asked. I have many answers but the one that's most honest is probably the least satisfying: I do it because I like it. I like bringing arms to life from unassuming piles of desk clutter. I like thinking about all the surrounding ideas, reading the research, and discussing it with everyone I can. It seems like the right thing to be doing.

More often, my answers to this question are: 1. as a gesture of friendship, 2. to learn more about oil painting, 3. to uncover an objective standard of taste. These are the answers I'll be addressing in this blog using examples from art history, philosophy, color theory, and computer science.