SynapticNulship

Now published on h+ magazine: my article “Five Ways Machines Could Fix Themselves.” Check it out!

As I see cooling fans die and chips fry, as I see half the machines in a laundry room decay into despondent malfunctioning relics, as my car invents new threats every day along the theme of catastrophic failure, and as I hear the horrific clunk of a “smart” phone diving into the sidewalk with a wonderful chance of breakage, I wonder why we put up with it. And why can’t this junk fix itself?

Design guru and psychologist Donald A. Norman has pointed out how most modern machines hide their internal workings from users. Any natural indicators, such as mechanical sounds, and certainly the view of mechanical parts, are muffled and covered. As much machinery as possible has been replaced by electronics which are silent except for the sound of fans whirring. And electronics are even more mysterious to most users than mechanical systems are.

Our interfaces to machines are primarily composed of various kinds of transducers (like buttons), LEDs (those little glowing lights), and display screens. We are, at the very least, one—if not a dozen—degrees removed from the implementation model. As someone who listens to user feedback, I can assure you that a user’s imagining of how a system works is often radically different than how it really works.

Yet with all this hiding away of the dirty reality of machinery, we have not had a proportional increase in machine self support.

Argument: Software, in some cases, does fix itself. Specifically I am thinking about automatic or pushed software updates. And, because that software runs on a box, it is by default also fixing a machine. For instance, console game platforms like XBox 360 and Playstation 3 receive numerous updates for bug fixes, enhancements, and game specific updates. Likewise, with some manual effort from the user, smart phones and even cars can have their firmware updated to get bug fixes and new features (or third-party hacks).

Counterargument: Most machines don’t update their software anywhere close to “automatically.” And none of those software updates actually fix physical problems. Software updates also require a minimal subset of the system to be operational, which is not always the case. The famous Red Ring of Death on the early XBox 360 units could not be fixed except via replacement of hardware. You might be able to flash your car’s engine control unit with new software, but that won’t fix mechanical parts that are already broken. And so on.

Another argument: Many programs and machines can “fail gracefully.” This phrase comforts a user like the phrase “controlled descent into the terrain” comforts the passenger of an airplane. However, it’s certainly the minimum bar that our contraptions should aim for. For example, if the software fails in your car, it should not default to maximum throttle, and preferably it would be able to limp to the nearest garage just in case your cell phone is dead. Another example: I expect my laptop to warn me, and then shutdown, if the internal temperature is too hot, as opposed to igniting the battery into a fireball.

The extreme solution to our modern mechatronic woes is to turn everything into software. If we made our machines out of programmable matter or nanobots that might be possible. Or we could all move into virtual realities, in which we have hooks for the meta—so a software update would actually update the code and data used to generate the representation of a machine (or any object) in our virtual world.

However, even if those technologies become mature, there won’t necessarily be one that is a monopoly or ubiquitous. A solution that is closer and could be integrated into current culture would be a drop-in replacement that utilizes existing infrastructures.

Some ideas that come close:

1. The device fixes itself without any external help. This has the shortcoming that it might be too broken to fix itself, or might not realize it’s broken. In some cases, we already have this in the form of redundant systems as used in aircraft, the Segway, etc.

2. Software updating (via the Internet) combined with 3D printing machines: the 3D printers would produce replacement parts. However, the printer of course needs the raw material but that could be as easy as putting paper in a printer. Perhaps in the future, that raw printer material will become some kind of basic utility, like water and Internet access.

3. Telepresence combined with built-in repair arms (aka “waldoes”). Many companies are currently trying to productize office-compatible telepresence robots. Doctors already use teleoperated robots such as Da Vinci to do remote, minimally-invasive surgery. Why not operate on machines? How to embed this into a room and/or within a machine is another—quite major—problem. Fortunately, with miniaturization of electronics, there might be room for new repair devices embedded in some products. And certainly not all products need general purpose manipulator arms. They could be machine specific devices, designed to repair the highest probability failures.

4. Autonomous telepresence combined with built-in repair arms: A remote server connects to the local machine via the Internet, using the built-in repair arms or device-specific repair mechanism. However, we also might need an automatic meta-repair mechanism. In other words, the fixer itself might break, or the remote server might crash. Now we enter endless recursions. However, this need not go on infinitely. It’s just a matter of having enough self-repair capacity to achieve some threshold of reliability.

5. Nothing is ever repaired, just installed. A FedEx robot appears within fifteen minutes with a replacement device and for an extra fee will set it up for you.

The great drama of the next few decades will unfold under the crossed stars of the analog and the digital.

—Steven Johnson, Interface Culture

Credit: urbanartcore.eu, CC by-nc-sa 2.0

Credit: Brian Despain

Credit: E. Benyaminso via A Journey Round My Skull, CC by- 2.0

Credit: J (mtonic.com), CC by- 2.0

Credit: J (mtonic.com), CC by- 2.0

Credit: Roberto Rizzato, CC by-nc 2.0

Credit: ARE MOKKELBOST

UPDATE 2011: There is a new/better version of this essay:  “Enactive Interface Perception and Affordances”.

There are two theories of perception which are very interesting to me not just for AI, but also from a point of view of interfaces, interactions, and affordances.  The first one is Alva Noë’s enactive approach to perception.  The second one is Donald D. Hoffman’s interface theory of perception.

Enactive Perception vs. Interface Perception

Enactive Perception

The key element of the enactive approach to perception is that sensorimotor knowledge and skills are a required part of perception.

A lot of artificial perception schemes, e.g. for robots, run algorithms on camera video frames.  Some programs also use the time dimension, e.g. structure from motion.  They can find certain objects and even extract 3D data (especially if they also use a range scanner such as LIDAR, ultrasound, or radar).  But the enactive approach suggests that animal visual perception is not simply a transformation of 2-D pictures into a 3-D (or any kind) of representation.

Example of optical flow (one of the ways to get structure from motion). Credit: Naoya Ohta.

My interpretation of the enactive approach is that it suggests perception co-evolved with motor skills such as how our bodies move and how our sensors, for instance, eyes, move.  A static 2D image can not tell you what color blobs are objects and what are merely artifacts of the sensor or environment (e.g. light effects).  But if you walk around this scene, and take into account how you are moving, you get a lot more data to figure out what is stable and what is not.  We have evolved to have constant motion in our eyes via saccades, so even without walking around or moving our heads, we are getting this motion data for our visual perception system.

Of course, there are some major issues that need to be resolved, at least in my mind, about enactive perception (and related theories).  As Aaron Sloman has pointed out repeatedly, we need to fix or remove dependence on symbol grounding.  Do all concepts, even abstract ones, exist in a mental skyscraper built on a foundation of sensorimotor concepts?  I won’t get into that here, but I will return to it in a later blog post.

The enactive approach says that you should be careful about making assumptions that perception (and consciousness) can be isolated on one side of an arbitrary interface.  For instance, it may not be alright to study perception (or consciousness) by looking just at the brain.  It may be necessary to include much more of the mind-environment system—a system which is not limited to one side of the arbitrary interface of the skull.

Perception as a User Interface

The Interface Theory of Perception says that “our perceptions constitute a species-specific user interface that guides behavior in a niche.”

Evolution has provided us with icons and widgets to hide the true complexity of reality.  This reality user interface allows organisms to survive better in particular environments, hence the selection for it.

Perception as an interface

Hoffman’s colorful example describes how male jewel beetles use a reality user interface to find females.  This perceptual interface is composed of simple rules involving the color and shininess of female wing cases.  Unfortunately, it evolved for a niche which could not have predicted the trash dropped by humans that lead to false positives—which results in male jewel beetles humping empty beer bottles.

Male Australian jewel beetle attempting to mate with a discarded "stubby" (beer bottle). Credit: Trevor J. Hawkeswood.

All perception, including of humans, evolved for adaptation to niches.  There is no reason or evidence to suspect that our reality interfaces provide “faithful depictions” of the objective world.  Fitness trumps truth.  Hoffman says that Noë supports a version of faithful depiction within enactive perception, although I don’t see how that is necessary for enactive perception.

Of course, the organism itself is part of the environment.

True Perception is Right Out the Window

How do we know what we know about reality?  There seems to be a consistency at our macroscopic scale of operation.  One consistency is due to natural genetic programs—and programs they in turn cause—which result in humans having shared knowledge bases and shared kinds of experience.  If you’ve ever not been on the same page as somebody before, then you can imagine how it would be like if we didn’t have anything in common conceptually.  Communication would be very difficult.  For every other entity you want to communicate with, you’d have to establish communication interfaces, translators, interpreters, etc.  And how would you even know who to communicate with in the first place?  Maybe you wouldn’t have even evolved communication.

So humans (and probably many other related animals) have experiences and concepts that are similar enough that we can communicate with each other via speech, writing, physical contact, gestures, art, etc.

But for all that shared experience and ability to generate interfaces, we have no inkling of reality.

Since the UI theory says that our perception is not necessarily realistic, and is most likely not even close to being realistic, does this conflict with the enactive theory?

Noë chants the mantra that the world makes itself available to us (echoing some of the 80s/90s era Brooksian “world as its own model”).  If representation is distributed in a human-environment system, doesn’t that mean it must be a pretty accurate representation?  No.  I don’t see why that has to be true.  So it seems we can combine the two theories together.

There may be some mutation to enactive theories if we have to slant or expand perception more towards what happens in the environment and away from the morphology-dependent properties.  In other words, we may have to emphasize the far environment (everything you can observe or interact with) even more than the near environment (the body).  As I think about this and conduct experiments, I will report on how this merging of theories is working out.
————

References

Noë, A., Action in Perception, Cambridge, MA: MIT Press, 2004.

Hoffman, D.D, “The interface theory of perception: Natural selection drives true perception to swift extinction” in Dickinson, S., Leonardis, A., Schiele, B., & Tarr, M.J. (Eds.), Object categorization: Computer and human vision perspectives. Cambridge, UK: Cambridge University Press, 2009, pp.148-166.  PDF.

Hawkeswood, T., “Review of the biology and host-plants of the Australian jewel beetle Julodimorpha bakewelli,” Calodema, vol. 3, 2005.  PDF.

According to philosopher Alva Noë, “Consciousness is more like dancing than it is like digestion.” I.e., consciousness happens while you are interfacing with the world.

But is it ballroom dancing, techno dancing, break dancing…the robot?

Consciousness

Sure, you can dance by yourself—especially to industrial and EBM—but you need music. And music is at least partially external. And there’s probably people around you. Crazy people. And a disco ball, and strobe lights…it’s an environment.

Does this mean your consciousness is dependent on interactions?

If you were in a sensory deprivation chamber would you be unconscious? I think you would have hallucinations and eventually go insane. But you would probably still be self aware and relatively conscious, at least until the black hole of strange loop madness consumed you.

However, that is in lifetime space (ontogeny). Consciousness might be fully dependent on interactions in evolution space (phylogeny).

So can this metaphor stretch to unconsciousness? Maybe unconsciousness is like doing the fish stick.

Unconsciousness