Building a Color Space

There are several major systems for defining color, each well-suited to different needs. RGB is the default for computer screens; each pixel is basically made up of a red, green, and blue LED. CMYK (cyan, magenta, yellow, black) are the ink colors which produce the best gamut for printers. CIE XYZ accounts for the green-dominance in human sight.

For me and Neko, hue is central. Hue is the relationship between primaries— a pale cherry red and a dark cherry red share a hue. I found this little algorithm for converting RGB is HSV (hue, saturation/white, value/black), and wanted to make a note of it for the future.

// r,g,b values are from 0 to 1
// h = [0,360], s = [0,1], v = [0,1]
//  if s == 0, then h = -1 (undefined)

void RGBtoHSV( float r, float g, float b, float *h, float *s, float *v )
{
 float min, max, delta;

 min = MIN( r, g, b );
 max = MAX( r, g, b );
 *v = max;    // v

 delta = max - min;

 if( max != 0 )
  *s = delta / max;  // s
 else {
  // r = g = b = 0  // s = 0, v is undefined
  *s = 0;
  *h = -1;
  return;
 }

 if( r == max )
  *h = ( g - b ) / delta;  // between yellow & magenta
 else if( g == max )
  *h = 2 + ( b - r ) / delta; // between cyan & yellow
 else
  *h = 4 + ( r - g ) / delta; // between magenta & cyan

 *h *= 60;    // degrees
 if( *h < 0 )
  *h += 360;

}

Once I have completed Neko's new set of pigment swatches, I will assign each pigment to a specific hue. When Neko has a target digital hue, he can refer to that database to find the closest pigments. Our color system is also growing in text references to each pigment, so word associations can be made as well.

Edit 4/23: Found this comprehensive color system website today.

Neko 2.2

Hardware

For my needs, the accelerometer and gyroscope gave both too much and too little information. I decided to scrap the locational sensors and use mild trigonometry. Using the ultrasonic sensor as a guide for keeping the brush on the canvas did work quite well, so I might reinstall that. I'm testing out a color sensor, cameras and lights now that the arm is working smoothly.

Software

I'm expanding my text-based approach after reading the wonderful novel Galatea 2.2, by Richard Powers. Instead of just looking for color words in texts about art, I want Neko to do freer association on a larger corpus. I'm still going to be selective about the texts I incorporate, but I liked the relationship Powers' narrator had with his machine.

Firmware

This is the firmware that's running in the video. This was just a piece of test code, but I really like the motion it makes. I don't want it to get too rigid and boring, treating the painting as a 2D grid. I want to find the kind of strokes Neko is good at. I'm starting to think of it like a dance: programmed sequences of graceful motion, recombined.

Click to expand

Art Loops

A creative system involves artist, art and viewer in an information-generative feedback loop. A creative system’s success can be measured by the amount of information generated. Some art conveys a deep message to a targeted audience, some a shallow message to a general audience. The art that fails to reach anyone returns to dust. The art that achieves both depth and breadth disseminates via mechanical reproduction, and can be considered extraordinarily successful. Hans Haacke is an artist more aware of the feedback loop than perhaps any other. His pieces are site-specific reflections upon the politics and environments of the art world. Some work is more reserved: Condensation Cube is a plexiglass box which maintains an equilibrium with the museum climate, showing the water cycle in isolation. In a more aggressive gesture, Haacke produced a series of framed photographs and text: Shapolsky et al. Manhattan Real Estate Holdings, A Real Time Social System, as of May 1, 1971. The series outlines the shady dealings of a landlord named Harry Shapolsky. Haacke created the piece for a show at the Guggenheim Museum, which the trustees promptly cancelled when learning of the plan. This act of censorship is of course the best affirmation Haacke could hope for.

Condensation Cube

Hans Haacke

1963 - 1965

Water in plexiglass

Influence of the three components of a creative system— artist, art and viewer— is not evenly distributed. Some artists lend creative control to the viewer directly. Some encode complex ideas fully in the work itself, some rely on wall text, some offer only abstractions. At the center of all art is metaphor: colors become sentiments, figures become archetypes, symbols abound. It is through metaphor that an artist expresses ideas. Good art is both surprising and relatable. A good artist blends deliberation with instinct— they intend to represent something, but remain open to the influence of their medium. By being beautiful, or funny, or eye-catching, the art invites viewers to spend time interpreting it. The artist relies on large swaths of overlapping taste, or else finds the right audience. 

Creative art presents ideas not otherwise available to the senses. Images of mythical creatures, extinct creatures, and creatures of an artist’s invention are examples of creative art. Representing these three kinds of creatures well requires a biological understanding of how animal bodies work and look, a practical understanding of an artistic medium, awareness of historical context, and an experienced aesthetic to fill in the gaps. Often the most challenging part of creation is deletion. Much creativity happens in the edit. Robert Rauschenberg is infamous for his Erased de Kooning Drawing. To paraphrase his interview on the piece by SFMOMA, it was a continuation of his series of white painted canvases. He began by erasing his own drawings but was unsatisfied, and decided it must be art first. So he went to de Kooning, whom he considered the most indisputably artistic, and asked for a drawing. After careful consideration, de Kooning selected a piece that was both dear to him and inclusive of several different media, making it as difficult to erase as possible. Rauschenberg says the piece took three months, and uncountable erasers.

Erased de Kooning Drawing

Robert Rauschenberg

1953

Drawing media on paper

Creative systems occur at micro and macro levels simultaneously. At the center of this scale is the local ecology of an artist creating and viewing their own work.  To the left of the scale are subroutines of neurons and nerves. To the right are collaboratives, institutions, and arenas. The richer the subsystems, the more successful the work. This idea reveals the first opportunity for a synthetic creative system: unlike a typical human artist, a robot feels no different painting in a studio than in a gallery. Viewers are often left unaware of what goes in to a piece, and artists are often secretive about their process. Viewers can watch a robot work with unabashed curiosity. If designed to work quickly, a robot can even take willing viewers on as subjects.

Hue Sort

My project is too big for a regular spreadsheet but not quite ready for a relational database. After testing the waters of the Python Imaging Library, I've settled on a file system to organize my colors. Six top level folders named Red, Orange, Yellow, Green, Blue, and Purple each have sixty subfolders. This allots one folder for each of the 360 hues in the color wheel. I'm going to name each of these 360 hues myself, and ask my viewers to name as many of the 16,777,216 unique hex colors as possible.

But I just hit a stumbling block. The 360 major computer generated hues do not fall neatly into 6 categories of 60. I made a little program to draw swatches for each hue so I could name them all; I was hoping to end up with a column for each ROYGBP. I shifted the spectrum so the hues were offset by 30 which helped a bit, but not much. There's too much Green, and not enough Yellow.

The visible spectrum image from Wikipedia looks much more balanced. I wonder if it's rendered in something more attuned to this than Processing, or if the person who generated it did some lerping. Either way, I don't think I'm going to use the Processing hues as the second tier of my color hierarchy. I'll hand-pick them. I'll deal with Magenta (which occurs in the color wheel but not the rainbow) by splitting it across Red and Purple.

Below the Wikipedia image are the spectrums from Processing and the ColorPy library I will be using in the near future. ColorPy is better, but still not great. Computer screens are a matrix of Red, Green & Blue LEDs— displaying secondary and composite colors by blending and gamma correction— which might also account for the Yellow deficit. I look forward to building my own spectrum, and finding the paint and light equivalents.

Pair Programming

Tom and I did some pair programming last week: he drove (wrote the code) and I navigated (dictated instructions). What we ended up with (link to GitHub) was something that crawls through text looking for color words. It divides the text into lines, and the lines into words. Any time the words red, orange, yellow, green, blue, or purple come up, the other words in that line are filed into a matrix. Once the algorithm is finished, it returns the set of words that have more than one correlation. For example, given the text, The Loves of Krishna in Indian Painting and Poetry by W. G. Archer, the program returned this:

The only really interesting result here is that Krishna is indeed blue (the literal translation of the name is "black" or "dark" but most depictions give him blue skin). It's singular, but still very exciting. One text is too small a dataset, so I'm building up some compilations of my favorite poets, transcendentalists, aesthetic philosophers, etc. I'll also tokenize the text by sentence rather than line (except in poems), and weigh the associated words by how close they are to the color word (so that in the line, red shoes by the newsstand, shoes gets more points than newsstand). I'll also add the words color/s and colour/s.

The results improved dramatically when I added words like black, dark, white, and light. These words are used much more often, particularly in metaphor. It occurred to me to start collecting those for Nila, my black and white painting robot, and I'm thrilled with the idea.

Neko's Brain Trust

On Monday, Neko and I were successfully funded on Kickstarter! It's tremendously exciting and I can't thank my supporters enough. I just put in a big motor order and look forward to a minor rebuild. As for software, there are two things I'm now teaching Neko: how to select a digital color given a set of words, and how to select pigments to represent that digital color. My new vocabulary for the day is distributional semantics. This is a practice based on the idea that words with similar distributions have similar definitions; "You shall know a word by the company it keeps." I'm building a text crawler that will look for my six core words— Red, Orange, Yellow, Green, Blue, Purple— and find words that are highly collocated with each. I'm picking out old public domain books to walk through, and would love suggestions on books that have high frequency of these words. While I'm waiting for the new motors to arrive, I'll put all the most-associated words into my database. Some of the Kickstarter funds will be allocated to a Mechanical Turk bid, though I'm not sure quite what to ask yet.

The second task is matching the pigments to the colors on-screen. For this I've ordered a color sensor which Neko can use to compare what's on the palette to the target color. I've picked a set of mixing pigments— Cadmium Red Medium, Cadmium Yellow Lemon, and Phthalo Blue— to be Neko's go-to adjusters.

Random Walk to Remember

The random walk is my Hello World— the foundation code I always write to get things up and running. If I'm using a new visualizer, I try to make it look like TV static. I'll use random colors or numbers or motions to get a broad sense of what my new system can do.

I'm pleased with Neko's code so far, it's very well-organized. The random walk is my best yet so I'll explain it for the sake of anyone who doesn't know how to read code, but would like a little puzzle to try it out.

This a screenshot of one of the functions in my program: randomWalk(). I tell the function to repeat infinitely with two parameters: the maximum distance the brush should jump, and the time to pause after each iteration. My current parameters are 50 steps and 500 milliseconds, so when the computer reads the instruction randomWalk(50, 500) it will run this piece of code with the variables jump and del set to 50 steps and 500ms, respectively.

This function uses randomness in two ways: it randomly picks one of the six motors, then it randomly picks the number of steps for that motor to move.

Neko also has a motorized shoulder which is not moved in the random walk, but instead responds to the ultrasonic sensor. This keeps the brush on the canvas. The accelerometer is read in a balanceBrush() function, but I don't use it much yet. There's also a readGyro() function, adapted from this tutorial, which prints the gyroscope readings but does nothing more; another noch-nicht. A zip file of Neko's Arduino program can be found here— a link I'll update with each new version.

Reading live data as scrolling lines of serial output can be quite meditative. I just disabled the motors and moved the brush around the canvas by hand, allowing my own neural network to get a sense of how the sensor data changes while drawing different shapes and lines. As a stepping-stone to a Kalman filter, I'll now try writing a complementary fiter to combine accelerometer and gyroscope data and determine how the arm is moving.