SynapticNulship

Now published on h+ magazine: my article “Five Ways Machines Could Fix Themselves.” Check it out!

As I see cooling fans die and chips fry, as I see half the machines in a laundry room decay into despondent malfunctioning relics, as my car invents new threats every day along the theme of catastrophic failure, and as I hear the horrific clunk of a “smart” phone diving into the sidewalk with a wonderful chance of breakage, I wonder why we put up with it. And why can’t this junk fix itself?

Design guru and psychologist Donald A. Norman has pointed out how most modern machines hide their internal workings from users. Any natural indicators, such as mechanical sounds, and certainly the view of mechanical parts, are muffled and covered. As much machinery as possible has been replaced by electronics which are silent except for the sound of fans whirring. And electronics are even more mysterious to most users than mechanical systems are.

Our interfaces to machines are primarily composed of various kinds of transducers (like buttons), LEDs (those little glowing lights), and display screens. We are, at the very least, one—if not a dozen—degrees removed from the implementation model. As someone who listens to user feedback, I can assure you that a user’s imagining of how a system works is often radically different than how it really works.

Yet with all this hiding away of the dirty reality of machinery, we have not had a proportional increase in machine self support.

Argument: Software, in some cases, does fix itself. Specifically I am thinking about automatic or pushed software updates. And, because that software runs on a box, it is by default also fixing a machine. For instance, console game platforms like XBox 360 and Playstation 3 receive numerous updates for bug fixes, enhancements, and game specific updates. Likewise, with some manual effort from the user, smart phones and even cars can have their firmware updated to get bug fixes and new features (or third-party hacks).

Counterargument: Most machines don’t update their software anywhere close to “automatically.” And none of those software updates actually fix physical problems. Software updates also require a minimal subset of the system to be operational, which is not always the case. The famous Red Ring of Death on the early XBox 360 units could not be fixed except via replacement of hardware. You might be able to flash your car’s engine control unit with new software, but that won’t fix mechanical parts that are already broken. And so on.

Another argument: Many programs and machines can “fail gracefully.” This phrase comforts a user like the phrase “controlled descent into the terrain” comforts the passenger of an airplane. However, it’s certainly the minimum bar that our contraptions should aim for. For example, if the software fails in your car, it should not default to maximum throttle, and preferably it would be able to limp to the nearest garage just in case your cell phone is dead. Another example: I expect my laptop to warn me, and then shutdown, if the internal temperature is too hot, as opposed to igniting the battery into a fireball.

The extreme solution to our modern mechatronic woes is to turn everything into software. If we made our machines out of programmable matter or nanobots that might be possible. Or we could all move into virtual realities, in which we have hooks for the meta—so a software update would actually update the code and data used to generate the representation of a machine (or any object) in our virtual world.

However, even if those technologies become mature, there won’t necessarily be one that is a monopoly or ubiquitous. A solution that is closer and could be integrated into current culture would be a drop-in replacement that utilizes existing infrastructures.

Some ideas that come close:

1. The device fixes itself without any external help. This has the shortcoming that it might be too broken to fix itself, or might not realize it’s broken. In some cases, we already have this in the form of redundant systems as used in aircraft, the Segway, etc.

2. Software updating (via the Internet) combined with 3D printing machines: the 3D printers would produce replacement parts. However, the printer of course needs the raw material but that could be as easy as putting paper in a printer. Perhaps in the future, that raw printer material will become some kind of basic utility, like water and Internet access.

3. Telepresence combined with built-in repair arms (aka “waldoes”). Many companies are currently trying to productize office-compatible telepresence robots. Doctors already use teleoperated robots such as Da Vinci to do remote, minimally-invasive surgery. Why not operate on machines? How to embed this into a room and/or within a machine is another—quite major—problem. Fortunately, with miniaturization of electronics, there might be room for new repair devices embedded in some products. And certainly not all products need general purpose manipulator arms. They could be machine specific devices, designed to repair the highest probability failures.

4. Autonomous telepresence combined with built-in repair arms: A remote server connects to the local machine via the Internet, using the built-in repair arms or device-specific repair mechanism. However, we also might need an automatic meta-repair mechanism. In other words, the fixer itself might break, or the remote server might crash. Now we enter endless recursions. However, this need not go on infinitely. It’s just a matter of having enough self-repair capacity to achieve some threshold of reliability.

5. Nothing is ever repaired, just installed. A FedEx robot appears within fifteen minutes with a replacement device and for an extra fee will set it up for you.

Now published on h+ magazine: my article “Softer, Better, Faster, Stronger: The Coming of Soft Cybernetics.” Check it out!

I have a titanium screw in my head.  It is a dental implant (root-form endosseous) covered with a crown.

Note: This is a representative photo from Wikipedia, not my personal implant

Osseointegration (fusing implants with bone) is used for many things these days, such as bone-anchored hearing aids and bone-anchored leg prostheses.

This is cool, but there’s a major interface problem if you have a metal rod poking out of your skin–it’s basically an open wound.  Researchers have found a solution however based on deer antlers, called ITAP (Intraosseous Transcutaneous Amputation Prosthesis), in which they can get the skin to actually grow into the titanium.

Deer antlers go through the skin, like bone-anchored prosthetics

They do this by carefully shaping the titanium and putting lots of tiny holes in it.  ITAPs are what the momentarily famous “bionic” cat Oscar received last June.

In these examples, biology is doing most of the work.  Sure, the chemical properties of titanium make it compatible, but when will the artificial technology pull its weight?  Where are the implants that integrate seamlessly with your body and with other implants?  Where are the computer interfaces that automatically and robustly integrate with any person’s nervous system?

Sure, there’s a lot of great medical technology which does successfully interface with human biology.  Let’s not forget the AbioCor artificial implantable replacement heart as featured in the illustrious film Crank: High Voltage.

AbioCor

Crank: High Voltage

But there’s a lot of things that don’t work well yet, such as direct neural interfaces–although there are glimmers of hope such as optical interfaces to the nervous system.  And besides medical technology, what about all machines–why are they so inflexible and high-maintenance?  And it’s not just hardware–the software realm seems to be particularly behind with “soft” and flexible interfaces.

In recent article called “Building Critical Systems as a Cyborg”, software architect Greg Ball compares the von Neumann algorithmic approach of most conventional software to the cybernetics approach.  He says:

Don’t assume those early cyberneticists would be impressed by our modern high-availability computer systems. They might even view our conventional approach to software as fatally arrogant, requiring a programmer to anticipate everything.

What if instead of fighting changes and new interactions, our software embraced them?  A cybernetic approach to software would more oriented around self regulation, including parts that are added in to the system from outside.

You might argue that regulation with feedback loops has been part of engineering systems for a long time.  But we still have a lot of brittleness in the interfaces.  It’s not easy to make systems out of components unless the interfaces match up perfectly.  In the software realm, things are pretty much the same.  Most of our technology behaves very differently from biology in terms of interfacing, adaptation, learning and growth.  Eventually we can do better than biology, but first we need to be as soft as biology.  This will help us not only for making machines that operate in the dynamic real world of humans, but will also help us make devices that directly attach to humans.

Do We Need Fuzzy Substrates?

Computers are embedded in almost all of our devices, and most of them are digital.  Information at the low levels is stored as binary.  Biology, in contrast, often makes use of analog systems.  But does that matter?  Take fuzzy logic for example.  Fuzzy logic techniques typically involve the concept of intermediate values between true and false.  It’s a way of dealing with vagueness.  But you don’t need a special computer for fuzzy logic–it’s just a program running on the digital computer like any other program.

Fuzzy logic, probability and other soft-computing approaches could go a long way to cover the role of adaptive interfaces in the computer code of a cyborg.  But are adaptive layers running on digital substrates enough?

USCD has been doing research with electronic neurons, which are made from analog computers.  So unlike most computers, the substrate does not represent information with discrete values.

Joseph Ayers and his lab members at Northeastern University were at one point attempting to use these electronic neurons in biomimetic lobster robots.  The electronic nervous system (ENS) would generate the behaviors of the robot, such as the pattern of signals to cause useful motion of the legs.  The legs are powered by nitinol (an alloy of titanium and nickel) wires, which expand and shrink thus causing movement.

biomimetic lobster robot from Northeastern University

The robots already had a digital control system, so the main point of moving to the ENS was for chaotic dynamics.  As Ayers described the situation:

The present controller is inherently deterministic, i.e., the robot does what we program it to do. A biological nervous system however is self-organizing in a stimulus-dependent manner and can use neurons with chaotic dynamics to make the behavior both robust and adaptive. It is in fact this capability that differentiates robotic from biological movements and the goal of ENS-based controllers.

Besides the dynamic chaos in nervous systems, the aforementioned USCD also researches synchronized chaos.  It sounds paradoxical, but it actually happens.  It could potentially be used for certain kinds of adaptable interfaces.  For instance, synchronized chaos can achieve “asymptotic stability,” which means that two systems can recover synchronization quickly after an external force messes up their sync.

I have given you a mere taste of soft cybernetics.  Its usage may have to increase, although it is not clear yet whether we need new information substrates such as analog computers.

Image Credits:

1. DRosenbach at en.wikipedia
2. Elizabeth Banuelos-Totman, University of Utah
3. Marieke IJsendoorn-Kuijpers
4. ABIOMED
5. Crank: High Voltage (2009), Lionsgate
6. Mostaque Chowdhury
7. Jan Witting

Christopher Nolan’s Inception has spawned some interesting things on the intertubes.

Behold this infographic by Rick Slusher showing the paths of the characters through the nested dreams (and who hosted each dream):

Surely that would make Tufte proud and warrants as much attention as Charles Joseph Minard’s famous graphic showing the time-place losses of Napoleon’s army during the Russian campaign of 1812…

…and more than matches xkcd’s movie chart excellence:

As a curious comparison, here is director Christopher Nolan’s own diagram:

Here is another Inception graphic–it shows the same info in a different way, although it doesn’t get across the time duration increases as the first graphic does.  It also states that Fischer hosted Limbo which disagrees (I think) with Nolan’s diagram.

And now, just for fun:

Image Credits:

1. Rick Slusher via Co.Design
2. Minard via Edward Tufte
3. Randall Munroe (xkcd)
4. Inception: The Shooting Script via io9
5. Sean Mort
6. fredvanlente via jasonpollock.tv

I just realized that Eadweard Muybridge, in the 1800s, could have achieved the time-slice effect as it was used for bullet time in the movie The Matrix.

Eadweard Muybridge

For instance, he could have arranged his trip-wire cameras in a circle around a subject to get the effect.

From wikipedia:

The method used for creating these effects involved a technically expanded version of an old art photography technique known as time-slice photography, in which a large number of cameras are placed around an object and triggered nearly simultaneously.^[14] Each camera is a still-picture camera, and not a motion picture camera, and it contributes just one frame to the video sequence. When the sequence of shots is viewed as in a movie, the viewer sees what are in effect two-dimensional “slices” of a three-dimensional moment. Watching such a “time slice” movie is akin to the real-life experience of walking around a statue to see how it looks from different angles.

Walking and turning around rapidly with a satchel in one hand, a cane in the other

The only problem with Muybridge’s technique is you need a Zoopraxiscope to watch the movie.  Somebody make a digital interface…

The Zoopraxiscope

Image credits:

1. Virtual Museum of the City of San Francisco
2. Random Kikiness
3. UK Screen Heritage Network