When a Developer Tries Art

I have always been interested in the intersection of art and computer science. Over the last couple years I began taking this passion a step further presenting some of my work in gallery exhibitions. This has been a rewarding experience and I want to present a few takeaways I've learned along the away.

Unique (and not so unique) Challenges

When writing software that is intended to be art you run into some surprising challenges; though, some may be familiar to those of us writing software intended for the server.

1. Up time, Up time, Up time
   So you've made an amazing piece of interactive art! You've got it looking just how you want, the frame rate is rocking, the response time is smooth, then you leave it running for a few hours...

Next thing you know the memory has sprung a leak, your event stack is piled higher than my dirty dishes (good old single threaded JavaScript), and the CPU can fry an egg. Then there's those untested edge cases that might crash your application in front of a live audience.

2. Plan for slow internet and test against connection failures
   In a recent exhibition for rules and play 2017 I noticed frames suddenly dropping, despite extensive testing. It turned out to be this button and bad WiFi.

When pressed, Every Picture Tells a Story, captures a screen shot and tweets the image. The images are first saved to disk by the process handling the display. A background process picks up new images and tweets them. This worked great in testing, but in the gallery I'm pretty sure the WiFi was dialing back in time to AOL! Images were taking a long time to upload and frequently failing. At a certain point the app would check for new images while still in the process of waiting for callbacks from images already being uploaded. I should have had guards in place to prevent this and should have made that loop less tight. It was thrashing both the disk and CPU affecting the performance of the entire machine including the display process.

All this brings me to my next point.

3. Testing
   Test early often and on a setup as close to production as you can get. Make sure your tests runs are long at least 24-48 hours. Anything less misses something. I collaborated on a piece called call and response. We thought we had done plenty of testing but discovered a machine which worked fine at home overheated when placed in the enclosure made for it at the gallery.

Set this stuff up in your living room ahead of time and abuse the crap out of it for as long as possible. Besides, what's more fun than a living room full of toys!

4. Kids Like to Break Things... I Mean Explore
   Your creation is public facing and must survive on its own in the wild. People will try to break it. Much like hardening a web service you need to think in terms of reducing the attack surface.

In this case your inputs are this biggest attack surface. Reduce physical access to input as much as possible. If you only need one button only present one button. You can build a great single button input out of a teensy lcs.

For obvious aesthetic reasons you'll want to hide the device running your piece, but the more you can restrict access to it the better.

Finally make sure your process is hard to kill. I generally have a background daemon spawn my process and monitor it. Anytime it dies due to free QA from a kid seeing how fast she can push the button, the background process re-spawns it. No harm now fowl.

5. Choose the right host environment and hardware

Consider what kind of device and operating system your artwork will live on. Its as important as choosing the right canvas for a painting.

Think about hardware costs and make sure your vision can run smoothly on the system you'll be using.

Keep in mind the system may sit unsupervised in a public space. You may lose a bit of sleep with your personal laptop full of family photos and finance info sitting in a gallery (thanks runjumpdev for helping with cost and equipment and putting on another great rules and play).

An obvious solution seems like the Raspberry PI. It's extremely cheap, and its tiny size makes it easy to secure. Sadly it has yet to work for me. My projects have all proven too ambitious or unstable for the poor little PI. In the case of call and response it ran fine on the PI at first, but I was unable to resolve an issue where electron apps stopped receiving data from the microphone after a few minutes. I could only duplicate this on the pi and only under electron specifically. Eventually I ran out of stack exchange articles on the issue and settled for an old laptop.

Another important thing to consider is the operating system. At one point we tried running call and response on a Surface Pro. Unfortunately it turns out that when windows isn't downloading updates, it's locking the screen, rebooting, or throwing up any number of screen savers! I found no less than four places that claimed to let you prevent the system from locking the screen. I event tried internet workarounds that included registry edits and replacement of the executable file that controls the lock screen, all to no avail.

Some people manage to wrestle windows into compliance, but I'm no Holk Hogan.

I've personally settled on using Ubuntu Mate. It's simple and the desktop environment is light weight; so, it doesn't steal much resources from your app. It runs well on old hardware (and new). With just a few tweaks you can create a system that boots straight into your app and just works!