SynapticNulship

I just published version 2 of my Enactive Interface Perception essay over on Science 2.0.

It’s now called “Enactive Interface Perception and Affordances”.

Today I was at the 2011 Embedded Systems Conference / DesignCon exposition. I typically attend technology expos in Boston, keeping an eye out for devices and software that I might be able to use in my job. But of course, I’m also interested in what embedded systems technology will enable in the near future.
There wasn’t anything mind-blowingly cool, but I will mention a few things that may be of interest to my readers.

First, IBM had an instantiation of Watson there, which was housed in a large black monolith that would be menacing if not for the colorful touch screen. Yes, Watson can run on a computer that IBM actually sells, which is the IBM Power 750 server.

IBM Watson

I started playing Jeopardy against this Watson, but lost interest when I found that there wasn’t any voice recognition (to get a question right after winning the buzz, the software would tell you the answer, at which time you would honorably press a button to confirm or not).

I also experienced NLT’s new (samples became available in June 2011) 3D display. This is an LCD module which does not require glasses to see the 3D, and although I only stared at it for less than a minute, it did work and I did not have to be in a very specific location relative to the screen. I’d like to try an actual application that made use of mixed 3D/2D. That is part of what’s supposedly unique about this 3D LCD, is that it can mix 2D and 3D and it’s all at the same resolution. This is due to their HDDP (horizontally double-density pixel) tech. NLT also claims their LCD reduces cross talk (when your brain’s visual system mixes right and left eye information).

NLT 3D LCD Tech

Speaking of display tech, I also played with Uneo’s Force Imaging Array System and 3D-Touch Module. The force array was not combined with a screen, and I’m not sure exactly what the killer app(s) would be—they claim it could be used for some unspecified medical, automotive, industrial apps. But I tried it and it works, and they told me that they would have one with even higher resolution soon (the current one has 2500 elements).

The 3D-Touch module was embedded in a tablet, and that also worked pretty well. The example app was of course a paint program, where you can see how your finger’s pressure affects the brush width as you paint. This doesn’t use the array—instead it uses sensors at the corners of the screen. That means you should be able to add it to any existing screen—it doesn’t have to be layered into the display stack. I certainly could imagine this being useful, at least occasionally, in various apps on my phone. Uneo has demoed it with Android devices so far but plans on getting support from the other mobile OSes.

Uneo 3D Touch example (photo from Uneo)

Microsoft was there. Nothing amazingly new…they had the Xbox 360 Wireless Speed Wheel, which ships in October as far as I know. It seems like such an obvious controller that I was surprised that it didn’t come out until 2011.

Xbox 360 Wireless Speed Wheel (stock photo)

They had a Kinect there, of course, and that’s always fun to play with—I spent about 10 minutes chopping flying fruit with my sword-hands. For those that are excited by this prospect, Fruit Ninja is available as of last month. For those living under rocks, Kinect is a super massively best selling controller for the XBox 360 which tracks the movement of your body as input for games. When it came out, people immediately started hacking it and using the sensor for robot applications. Microsoft didn’t like that at first, but now they’ve given in and offer a legit SDK (Software Development Kit) for it.

Fruit Ninja Kinect (stock screenshot)

I was pleased to see that one attendee teleconned in with a VGo telepresence robot. Note this photo is of the back of robot.

VGo robot in use at ESC 2011

This is a belated post from April. I live near MIT, so when they held an open house on April 30 I felt it was my duty to attend.

However, the most surprising thing was not the technology on display so much as the vast swarms of yuppie larvae—aka virus vectors, aka children. After awhile (about 5 minutes) my perception of their presence incremented from “cute” to “horrific.” Even worse were the parents of said children, whose method for navigating crowds consisted of crashing into other people like a bunch of semi-autonomous pinballs. So I departed, but not without taking a few photos first.

El Cheapo Multi-Touch Table

Innards of the Student-Built Multi-Touch Table

Cars That Won't Crash

Supervisory Control of Cyberphysical Systems (poster)

A Wearable Vital Signs Monitor at the Ear (poster)

And now photos of human children engaged with robots:

Children with Robots

Children with Robots

Children with Robots

Children with Robots

Children with Robots

And just for shits and giggles, here are some ancient computing artifacts that were on display in the Stata Center.  The first is an Atari 2600 “video computer system” (nowadays, a “console”) with a Space Invaders cartridge, right underneath a sign about Moore’s Law.

Atari 2600 console (released in 1977)

And one of the first cell phones, being fondled by me:

Motorola DynaTAC 8000X (circa 1983)

I conclude with a video I took of a good ol’ floating electromagnet:

Popular Science [1] has reported a tidbit of information: Marc Christensen’s team at SMU is supposed to start testing if they can stimulate a rat’s leg with optical fibers.

Fiber optic to nervous system interface

This is the same DARPA-funded project I mentioned last September in my article “Softer, Better, Faster, Stronger” [2].  DARPA held a related “Reliable Neural Interface Technology (RE-NET)” workshop back in 2009 [3]:

A well-meaning motor prosthesis with even 90% reliability, such as a prosthetic leg that fails once every 10 steps, would quickly be traded for a less capable but more reliable alternative (e.g., a wheelchair). The functionality of any viable prostheses using recorded neural signals must be maintained while the patient is engaged in or has their attention directed to unrelated activities (e.g., moving, talking, eating, etc.). Since the neural-prosthesis-research community has yet to demonstrate the control of even a simple 1-bit switch with a long-term high level of speed and reliability, the success of more ambitious goals (e.g., artificial limbs) are placed in doubt.

DARPA is interested in identifying the specific fundamental challenges preventing clinical deployment of Reliable Neural Technology (RE-NET), where new agency funding might be able to advance neural-interface technology, thus facilitating its great potential to enhance the recovery of our injured servicemembers and assist them in returning to active duty.

Neurophotonics

Technology comparison

Some of the challenges listed for the optical (neurophotonic sensing) approach are [4][5]:

• Transduce action potential into optically measurable quantity
• Modes: ionic concentration / flux vs. electromagnetic field
• Field Overlap
• Can’t go straight from voltage (indirect detection)
• Sensitivity, Parallelism
• Packaging, Size
• Untested
• “What is the minimum level of control-signal information required to recover a range of activities of daily living in both military and civilian situations?”
• “Need a method for characterizing tissue near implant to better understand long term degradation.”

Some of those challenges probably apply to all forms of neuro sensing.  Likewise, the metrics for neurophotonic interfaces—resolution, signal-to-noise ratio, and density—probably apply to other methods as well.

The Need for Better Neural Interfaces

Future prosthetics

Maybe the neurophotonic approach won’t work in the end, or it will only work in combination with another method.  Whatever the case, a lot of money should be put into this kind of project.  We are in desperate need for more advanced neural interfaces.  As Dr. Principe of the University of Florida writes [6]:

Just Picture yourself being blindfolded in a noisy and cluttered night club that you need to navigate by receiving a voice command once a second…And you will understand the problem faced by engineers designing a BMI [Brain Machine Interface].

Present systems are signal translators and will not be the blue print for clinical applications.  Current decoding methods use kinematic training signals – not available in the paralyzed. I/O models cannot contend with new environments without retraining.  BMIs should not be simply a passive decoder – incorporate cognitive abilities of the user.

Interfaces to the nervous systems are the key enablers for all of future prosthetics—and of course other exotic devices that don’t even exist yet.  Without overcoming this interface hurdle, we’ll be stuck in the stone age of prosthetics and nervous system repair.

References:
[1] M. Peck, “Talk To The Hand: A New Interface For Bionic Limbs,” Popular Science, Feb 24, 2011.
[3] J.W. Judy & M.B. Wolfson, RE-NET website.
[2] “Softer, Better, Faster, Stronger: The Coming of Soft Cybernetics,” H+ Magazine, Sept 21, 2010.
[4] M.P. Christensen, “Neuro-photonic Sensing: Possibilities & Directions”, DARPA RE-NET Workshop, Nov 19, 2009.
[5] Optical Breakout Session Report, DARPA RE-NET Workshop, Nov 20, 2009.
[6] J.C. Principe, “Architectures for Brain-Machine Interfaces,”  DARPA RE-NET Workshop, Nov 19, 2009.

Image Credits:
[1] Rajeev Doshi, PopSci
[2] DARPA / CIPhER via Physorg
[3] scan of book cover, art by John Berkey