Music and Machines: Highlights from the MIT Media Lab

I recently attended “Music | Machines: 50 Years of Music and Technology @ MIT,” part of MIT’s ongoing Festival of Art + Science + Technology (FAST).

One of the most interesting demonstrations was the iPhone Guitar by Rob Morris of the Media Lab. Basically, he makes use of the iPhone’s accelerometer as an input for special effects.

iPhone Guitar by Rob Morris

The iPhone is attached to the guitar, so that certain gestural movements of the guitar in space–especially those that happen during an emotional performance–are detected and used to modulate the sound. The touch screen of the iPhone also comes in handy as an accessible on-guitar interface for selecting effects and inputting other variables.

The Muse

The Muse

Digital music is no longer a new phenomena; in fact, it’s downright ancient when you consider that one of the first digital music contraptions was made in 1972. The Triadex Muse is an algorithmic music generator using digital logic, and was designed by Edward Fredkin and Marvin Minsky at MIT Lincoln Laboratory.

Music, Mind and Meaning

Marvin Minsky at MIT Media Lab, Feb 5, 2011

Speaking of Minsky, he discussed “Music, Mind and Meaning” with Teresa Marrin, Mary Farbood and Mike Hawley. Amongst the anecdotes Minsky mentioned an old concept of goals.

One of the ways human minds might achieve goals is to reduce the difference between what it has and what it wants. Music may utilize some of the same mental components–most music chops time in equal intervals and with equal substructures. These chopped experience windows can be compared, possibly in the same way that you can compare what you have with what you want.

Excerpts from the Concert

Computer based production is normal nowadays. So how would a computer and electronics oriented concert be special? Well, Todd Machover of the Media Lab was able to do that by assembling musicians that make some very unusual sounds and abnormal compositions. They all involve computers and/or electronics, but in innovative ways…and through live performances.

The concert began with a 1976 composition by Barry Vercoe called Synapse for Viola and Computer, an early work from MIT’s Experimental Music Studio. As a restaging of the 1970s performance, the digital accompaniment is inflexible, so it was up to the human soloist, Marcus Thompson, to maintain sync and “express himself within the confines.”

Vercoe – Synapse

Synapse was followed by Synaptogenesis, in which Richard Boulanger performs by triggering sound clips and transformations using a Nintendo WiiMote and a Novation Launchpad.

Boulanger – Synaptogenesis

Programmable drums machines have been around since 1972, but what is rare is to see the machine actuate physical percussion hardware. One such robotic instrument is the Heliphon, originally made by Leila Hasan and Giles Hall, and later redesigned by Bill Tremblay and Andy Cavatorta.

Todd Reynolds, Heliphon the robot, and Evan Zipioryn performing at the MIT Media Lab

The sound from this double helix metallophone is produced via solenoids hammering the metal keys. It also has lights hooked in to give a visual indication of which keys are active.

Heliphon and humans Todd Reynolds (violin) and Evan Ziporyn (clarinet) performed Ziporyn’s Belle Labs – Parts 1 & 3.

Ziporyn, Reynolds, Heliphon – Belle Labs Parts 1 and 3

Heliphon is one of various robotic instruments commissioned by Ensemble Robot, a nonprofit corporation based in Boston, MA. Ensemble Robot also made WhirlyBot, which looks like a turnstile but sounds like a chorus of human-like voices, and Bot(i)Cello, which appears to be a cross between a construction tool and a stringed instrument.

The Future of the Underground

If you’re interested in hearing more electronic music, there is always new stuff (or remixes of old stuff) being made, far below the radar of the mainstream.  You can hear some of it on the web, but being at a live performance or DJ set is a different experience, especially when the DJ modifies the music on the fly.  There are some new tools to enable this, for example, here is DJ/producer Encati demonstrating a Kinect wobble controller for dubstep mutations:

What I would like to see more of are environmental actuations triggered by music, beyond just flashing lights. We have autogenerated visualizers, and we can use MIDI to control lights (and fire cannons), but what about having a room really transform automatically based on the music? I’m taking about dynamic 2D and 3D displays everywhere, autonomous mobile furniture, materials changing shape and color, and so on.


Image credits:

4. MIT
Others by the author.


Cross-posted with H+ Magazine.

A Boy and His 286: Into the Coding Wilderness

Every boy should have a journey into the coding wilderness. To prepare yourself for this chronicle, imagine an Australian walkabout, except in the Grid of TRON.

ASCII'd.

This story begins with my first PC. It was a hand-me-down 286 Turbo, which was 1980s technology (but I got it in the 1990s).

286 Turbo

Turbo (reenactment).

I had used various computers before that (PCs, Apple IIes, Macs, Nintendos, etc.), but it wasn’t until I completely owned one that the real exploring and learning began. When you have your own computer in your room it is truly personal—you have free reign to hack it, break it, fix it, and explore new worlds with wild abandon.

Wild abandon.

The first development environment I used was the then ancient Microsoft GW-BASIC (which first came out in 1983 and was bundled with MS-DOS). Microsoft got their start with BASIC interpreters (Altair BASIC in 1975) and continued making excellent (for BASIC, that is) IDEs all the way through to the VisualBasic years.

GW-BASIC may look pretty lame compared to pretty much everything that came after it, but it does get points for having a function called TRON (you can see it listed in the function keys at the bottom in the code screenshot above).

The way it happened was I was exploring the programs on the computer and ran GW-BASIC, having no clue what it was. I was presented with a mostly black screen with a prompt, not entirely unlike a DOS prompt. I was not at all impressed and started typing guesses at what the quit command would be. Nothing worked, so I went to find the printed manual and located the command that would quit: SYSTEM, and it worked. But it was too late—I was already reading all the other commands in the manual and immediately set out to make some graphics appear on the screen, and of course try to make video games.

Screenshot from my first program.

Screenshot from my first program.

Screenshot from my first program.

Let the blazingly advanced graphics melt your eyeballs.

From the get-go, I was coding from scratch. But I found that analyzing and modifying existing source code published in books was a way to accelerate the first phase of learning. The books were usually several years old by the time I got to them—in the 1970s and 1980s they would print BASIC programs in magazines and books and the readers would dutifully type them in and try to run them. Some of those programs were for other dialects, such as on Trash-80s (RadioShack TRS-80) and other 1980s relics, so I had to translate them.

Puberty and Programming

The early to mid 90s were exciting times for me: I was finishing puberty, I was teaching myself to program computers, and the World Wide Web appeared.

At age 14 I got my first legal paid job (of course I had done jobs before that but not official taxed jobs) during the summer with a youth job training council in NH held at the local high school (which I had never seen before, still being in middle school), which then placed me in an accounting office (David L. Connors & Co. in Concord, NH).  I was like an intern, except I didn’t learn that much about accounting because I preferred to spend most of my time making spreadsheet macros as that was the closest to programming I could get there.

They were running Windows 3.11 at the time and once while exploring that I discovered a link to the DOS program QBasic, which was Microsoft’s free subset of QuickBASIC, their much better BASIC system spawned in the 1980s. When you’ve been using GW-BASIC, the QuickBasic family is like a revolutionary jump forward.

Screenshot of QuickBasic development environment

Wallow in the deep blue sea of QuickBasic 4.5.

For one thing, the IDE is probably the most user friendly DOS program I’ve ever used. I particularly liked the feature showing just one subroutine/function at a time in the buffer (and accessing the list of subs with one key press). In fact, I’ve been wanting that same feature in IDEs ever since but they never have exactly that, and only recently with code folding can I do something that results in equal efficiency. QB also had a built in help system that was useful.

SUBS (subroutines and functions) selection dialog.

The second thing is the language itself—not only had it grown, but it added intrinsic support for structured programming. Previously, BASIC programs would quickly turn into spaghetti code. You could still write spaghetti of course, but now you could control the insanity better with true subroutines, functions, do loops, data structures, etc. And those damn line numbers were no longer required.

The Golden Years of Basic on the Web

I believe the first BASIC Internet resource I encountered was the ABC (All Basic Code) packets curated by William Yu, which was started in 1995. Soon after, when the web became popular and more people had access, that hobby exploded into hundreds of websites and meta-sites.

A lot of the code was for QuickBasic 4.5 (and the “professional” superset QBX aka PDS 7.1) but there was also code for competitors like PowerBASIC. Some people even started trying to make new BASIC interpreters like DarkBASIC which intrinsicly support the elements needed for making more modern games. (More recently somebody made a cross platform backwards-compatible compiler called QB64). And of course people were sharing the interpreters and compilers.

dontcopyfloppy

Oh shut up.

Whenever an individual shared some tricks or assembly code that extended QuickBASIC, for instance accessing extended memory or outputting WAVs to sound hardware, it would spread through the community quickly. People were making libraries for loading and saving modern image and audio formats, and making games with those libraries that were pretty good considering these were hobbyists using obsolete software.

Games

I made many games myself but never finished any of them. That was ok for a learning phase, but of course eventually I had to learn that completing a project and having closure was also important.

As is usually the case, I also made and/or modified many utilities for making games, for instance graphics editors, bitmapped font editors (and yes I actually made my own bitmapped fonts), a slide-show based game editor with a GUI system I made from scratch, etc.

screenshot of font editor

Editing my Celtic bitmap font in my font editor.

screenshot of face generator

Experiment to auto-generate face textures.

screenshot of ASCIIFX

ASCII Grafix, my text mode drawing program.

This playful sandbox of BASIC enthusiasm continued for many years. I was aware of other languages, and had been informed that C was the best general purpose language in engineering industries. However, BASIC was so much fun that I ignored C for a long time. And as one would expect, an interpreter facilitates exploration and learning rapidly as compared to a compiler.

Paradigm Shift

I realized there was something called C++ and assumed that it must be better than old fashioned C. In high school one day I was at a flea market and happened upon a wicked good book deal—three C++ books for $1 each. One of them, Simple C++ by Jeffrey M. Cogswell, is possibly the best programming introduction book ever—it teaches you object oriented programming (OOP) with examples from a fake project to make a robot dog (and it has a sense of humor too). It was an intro to C++, but it did it the right way, via OOP.

I have observed that many others learn C++ the wrong way, as is common in colleges, with almost no emphasis on OOP. When I was in college I took two C++ courses but I did not buy the books we were supposed to. I chose to read more useful books like Industrial Strength C++ by Mats Henricson and Erik Nyquist (for which I raked up some hefty late fees at the library) and Effective C++ by Scott Meyers.

My point, gentle programmers, is that our forefathers of computing started out in the primordial mud of undocumented disorganization, so now us advanced apes live in the Aftermaths…of CHAOS!
—Samuel H. Kenyon, circa 1998

My programming paradigm progression in the 1990s:

⇨ Mostly-chaotic BASIC spaghetti code.
Structured QuickBasic.
⇨ Object-oriented C++.

So, my personal development recapitulated a path of programming evolution of the late 1960s to the 1980s.

To be continued…


Image credits:

  1. Turbo button: Tamas Kemenczy

A Dreadful Fine Title Page: Or, A Vast Amount of Absolutely Necessary Information

This is the title page of an 1879 dictionary I have:

THE AMERICAN POPULAR DICTIONARY: CONTAINING EVERY USEFUL WORD TO BE FOUND IN THE ENGLISH LANGUAGE WITH ITS TRUE MEANING, DERIVATION, SPELLING, AND PRONUNCIATION. ALSO, A VAST AMOUNT OF ABSOLUTELY NECESSARY INFORMATION UPON SCIENCE, MYTHOLOGY, BIOGRAPHY, AMERICAN HISTORY, INDIANS, LAND TITLES, CONSTITUTIONS, LAWS, CITIES, COLLEGES, ARMY AND NAVY, DEBTS, RATE OF MORTALITY, GROWTH OF CITIES, RATES OF INTEREST, INSOLVENT AND ASSIGNMENT LAWS, ETC. BEING A PERFECT LIBRARY OF REFERENCE, IN ONE HANDY VOLUME.

The full title is 506 characters. That’s 3.61 tweets!

And it’s quite the self-marketing title–in a single 512-page book they have amassed every useful word and a vast amount of absolutely necessary information. It reminds me of The Hitchhiker’s Guide to the Galaxy a bit.

SLANG AND VULGAR PHRASES

The title of this post is taken from the absolutely necessary section of the book called “SLANG AND VULGAR PHRASES” which informs us that “dreadful fine” is a contradiction. Here are few interesting quotes from that section:

At loggerheads is uncouth.
Bad box.—”He is in a bad box” has a vulgar air. Say bad predicament, or unpleasant situation.
Comeatable, for approachable; as, “European monarchs are not easily comeatable.”
Fizzle should be applied only to inglorious failures.
Full chisel.—”He went full chisel” is an absurd expression. Say “as fast as he could,” or “he ran his best.”
“How’s yourself, this morning,” savors of the familiarity that breeds contempt.
Otherguess, for otherguise, very different from, or superior to
A precious mess, a pretty kettle of fish, mean nothing.
Crank, for pert, saucy; as, “He was so crank that he was little respected.” New England.