Uh-oh, it looks like your Internet Explorer is out of date.
For a better shopping experience, please upgrade now.
As user interface designers, software developers, and yes-as users, we all know the frustration that comes with using "one size fits all" software from off the shelf. Repeating the same commands over and over again, putting up with an unfriendly graphical interface, being unable to program a new application that you thought of yourself-these are all common complaints. The inflexibility of today's computer interfaces makes many people feel like they are slaves to their computers. Shouldn't it be the other way around? Why can't technology give us more "custom-fitting" software?
On the horizon is a new technology that promises to give ordinary users the power to create and modify their own programs. Programming by example (PBE) is a technique in which a software agent records a user's behavior in an interactive graphical interface, then automatically writes a program that will perform that behavior for the user.
Your Wish is My Command: Programming by Example takes a broad look at this new technology. In these nineteen chapters, programming experts describe implemented systems showing that PBE can work in a wide variety of application fields. They include the following:
- Text and graphical editing
- Web browsing
- Computer-aided design
- Teaching programming to children
- Programming computer games
- Geographical information systems
The renowned authors and their editor believe that PBE will some day make it possible for interfaces to effectively say to the user, "Your wish is my command!"
|Publisher:||Elsevier Science & Technology Books|
|Series:||The Morgan Kaufmann Series in Interactive Technologies|
|Edition description:||New Edition|
|Product dimensions:||7.38(w) x 9.24(h) x 1.01(d)|
About the Author
Henry Lieberman has been a Research Scientist at the MIT Media Laboratory since 1987. From 1972 until 1987, he was a researcher at the MIT Artificial Intelligence Laboratory. His work focuses on the intersection of artificial intelligence and the human interface. Dr. Lieberman began his career with Seymour Papert and the group behind the educational language Logo. A member of the Software Agents group, he holds a doctoral-equivalent degree from the University of Paris-VI and has published over fifty papers on a wide variety of research topics.
Read an Excerpt
Chapter 1: Novice Programming Comes of Age
Since the late 1960s, programming language designers have been trying to develop approaches to programming computers that would succeed with novices. None has gained widespread acceptance. We have taken a different approach. We eliminate traditional programming languages in favor of a combination of two other technologies: programming by demonstration (PBD) and visual before-after rules. Our approach is now available as a product named Stagecast Creator, which was introduced in March 1999. It is one of the first commercial uses of PBD. Stagecast Creator enables even children to create their own interactive stories, games, and simulations. Here, we describe our approach, offer independent evidence that it works for novices, and discuss why it works when other approaches haven't and, more important, can't.
The computer is the most powerful tool ever devised for processing information, promising to make people's lives richer (in several senses). But much of this potential is unrealized. Today, the only way most people are able to interact with computers is through programs (applications) written by other people. This limited interaction represents a myopic and procrustean view of computers, like Alice looking at the garden in Wonderland through a keyhole. Until people can program computers themselves, they'll be able to exploit only a fraction of a computer's power.
The limits of conventional interaction have long motivated researchers in end-user programming. An end user in this context uses a computer but has never taken a programming class-a definition describingthe vast majority of computer users. We use the term novice programmer to describe end users who want to program computers. Is novice programmer an oxymoron? Is it a reasonable goal? Certainly there are "novice document writers," "novice spreadsheet modelers," and even "novice Internet surfers." But in more than thirty years of trying, no one has come up with an approach that enables novices to program computers. Elliot Soloway, director of the Highly Interactive Computing Project at the University of Michigan, estimates that even for novices who do take a programming class, less than 1 percent continue to program when the class ends. We'll explore the reasons for this, but first we explore our own new approach to programming that seems to work for novices.
Stagecast Creator, a novice programming system for constructing simulations from Stagecast Software, Inc., founded by the authors and others in 1997, is the culmination of a seven-year research and development effort, the first five at Apple Computer (Smith, Cypher, and Spohrer 1994; Cypher and Smith 1995; Smith and Cypher 1995; Smith, Cypher, and Schmucker 1996). The project, initially called KidSim, was later renamed Cocoa, and finally became Creator. The goal was to make computers more useful in education. The co-inventors of Creator-the authors Smith and Cypher-decided to focus on simulations because simulations are a powerful teaching tool. They can make abstract ideas concrete and understandable. Interaction with them is unstructured and explanatory in nature. Students can conduct experiments. They can formulate hypotheses as to what will happen, then run a simulation and observe the results (the scientific method). When their hypotheses are refuted, students really get engaged. They lean toward the monitor and begin talking animatedly with each other. Even better, with Creator children learn by building simulations, encouraging the constructivist approach to learning. Children learn sequential, causal reasoning. In view of this potential, the goal of the Creator project evolved into empowering end users (teachers and students) to construct and modify simulations through programming.
Our initial approach was much like that of other language developers: to invent a programming language that would be acceptable to end users. We tried a variety of syntaxes; all failed dismally. That experience, together with the history of programming languages during the past thirty years-from Basic to Pascal, from Logo to Smalltalk, from HyperTalk to Lingo-convinced us we could never come up with a language that would work for novices.
Our first insight was that language itself is the problem and that any textual computer language represents an inherent barrier to user understanding. Learning a new language is difficult for most people. Consider the years of study required to learn a foreign language, and such languages are natural languages. A programming language is an artificial language that deals with the arcane world of algorithms and data structures. We concluded that no conventional programming language would ever be widely accepted by end users.
Programming without a Textual Programming Language
How can a computer be programmed without a textual programming language? Our solution combined two existing techniques: PBD and visual before-after rules. In PBD, users demonstrate algorithms to the computer by operating the computer's interface just as they would if they weren't programming. The computer records the user's actions and can reexecute them later on different inputs. PBD's most important characteristic is that everyone can do it. PBD is not much different from or more difficult than using the computer normally. This characteristic led us to consider PBD as an alternative approach to syntactic languages.
A problem with PBD has always been how to represent a recorded program to users. It's no good allowing users to create a program easily and then require them to learn a difficult syntactic language to view and modify it, as with most PBD systems. In Creator, we first sought to show the recorded program by representing each step in some form, either graphically or textually. Some of the representations were, in our opinion, elegant, but all tested terribly. Children would almost visibly shrink from their complexity. We eventually concluded that no one wanted to see all the steps; they were just too complicated.
Our second insight was not to represent each step in a program; instead, Creator displays only the beginning and ending states. Creator does in fact have a syntax-the lists of tests and actions in a rule-but people can create programs for a long time without even being aware of this syntax. This feature is dramatically different from conventional languages, in which users must know whether a routine is called "move" or "go," the order of the parameters, and where all the various quotation marks, semicolons, and parentheses belong. As an example of the Creator approach, suppose we want the engine of a train simulation to move to the right. We move the engine by defining a visual before-after rule for the engine. Rules are the Creator equivalent of subroutines in other languages. Each rule represents an arbitrary number of primitive operations, or statements in other languages. Visually, Creator shows a picture of a small portion of the simulation on the left, then an arrow, and then a picture of what we want the simulation to look like after the rule executes. Figure 1.1 shows the interactive, visual process of creating a rule by demonstration.
First, we define the initial rule. Notice that the left and right sides start out the same; all rules begin as identity transformations. Users define the behavior of the rule by demonstrating changes to the right side. Here, we grab the engine with the mouse and drag it to the right. When we drop the engine, it snaps to the grid square it is over. That's all there is to it. Nowhere did we type b e g i n -end, i f -then - e 1 se, semicolons, parentheses, or any other language syntax...
Table of Contents
|Chapter 1||Novice Programming Comes of Age||7|
|1.2||Programming without a Textual Programming Language||9|
|Chapter 2||Generalizing by Removing Detail: How Any Program Can Be Created by Working with Examples||21|
|2.2||A Brief Introduction to ToonTalk||24|
|2.3||An Example of Programming by Example||26|
|Chapter 3||Demonstrational Interfaces: Sometimes You Need a Little Intelligence, Sometimes You Need a Lot||45|
|3.2||Our Demonstrational Systems||47|
|3.3||Level of Intelligence||49|
|Chapter 4||Web Browsing by Example||61|
|4.2||Underlying Problems of PBE||63|
|4.3||Web Browsing: Good Domain for PBE||64|
|4.5||SmallBrowse: Web-Browsing Interface for SmallScreen Computers||73|
|Appendix||Copying HTML Data from Web Browser to Scrapbook||84|
|Chapter 5||Trainable Information Agents for the Web||87|
|5.2||An Application Scenario||89|
|5.3||The HyQL Query Language||91|
|5.4||The Training Dialogue||96|
|5.6||The Communication Problem||105|
|5.7||Another Application Scenario||109|
|5.8||Related Work (Non-PBE)||110|
|Chapter 6||End Users and GIS: A Demonstration IS Worth a Thousand Words||115|
|6.2||A Story of End Users and GIS||116|
|6.3||Why Is GIS Software So Hard to Use?||118|
|6.4||Are Things Improving for GIS Users?||120|
|6.5||How Can Programming by Demonstration Help?||121|
|6.6||A Programming-by-Demonstration Approach for GIS: C-SPRL||123|
|Chapter 7||Bringing Programming by Demonstration to CAD Users||135|
|7.2||PBD and CAD||137|
|7.3||Toward a Complete Solution||143|
|7.4||True Explicit PBD Solutions||155|
|Chapter 8||Demonstrating the Hidden Features that Make an Application Work||163|
|8.2||The Perils of Plain Demonstration||165|
|8.3||Who Is Actually Programming?||166|
|8.4||Giving the System Hints||167|
|8.5||The Programming Environment Matters||171|
|Chapter 9||A Reporting Tool Using Programming by Example for Format Designation||175|
|9.3||User Interface of Format Editor||179|
|9.4||Extracting Formatting Rules||182|
|9.6||Example of the Process||183|
|Chapter 10||Composition by Example||191|
|10.2||PBE-Based Text Editing Systems||193|
|10.3||Dynamic Macro: A PBE-Based Text Editing System||193|
|10.4||POBox: A PBE-Based Text Input System||197|
|Chapter 11||Learning Repetitive Text-Editing Procedures with SMARTedit||209|
|11.2||The SMARTedit User Interface||212|
|11.3||The Smarts behind SMARTedit||215|
|11.4||Choosing the Most Likely Action||219|
|11.5||Making SMARTedit a More Intelligent Student||221|
|11.6||Other Directions for SMARTedit||223|
|11.7||Comparison with Other Text-Editing PBD Systems||223|
|Chapter 12||Training Agents to Recognize Text by Example||227|
|12.1||Text Recognition Agents||228|
|12.2||Writing Conventional Grammars as Text||230|
|12.3||Programming Grammars by Example for More Accessibility||231|
|12.4||Grammex: A Demonstrational Interface for Grammar Definition||232|
|12.5||An Example: Defining a Grammar for Email Addresses||233|
|12.6||Rule Definitions from Multiple Examples||236|
|12.7||Future Work: Using Grammar Induction to Speed Up the Definition Process||241|
|Chapter 13||SWYN: A Visual Representation for Regular Expressions||245|
|13.2||Other PBE Systems for Inferring Regular Expressions||250|
|13.3||A User Interface for Creating Regular Expressions from Examples||251|
|13.4||A Heuristic Algorithm for Regular Expression Inference||255|
|13.5||A Visual Notation for Regular Expressions||258|
|13.6||An Integrated Facility for Regular Expression Creation||265|
|Chapter 14||Learning Users' Habits to Automate Repetitive Tasks||271|
|14.2||Overview of APE||274|
|14.4||Detecting Repetitive Tasks||284|
|14.5||Learning a User's Habits||286|
|14.6||Use and Experimental Results||290|
|Chapter 15||Domain-Independent Programming by Demonstration in Existing Applications||297|
|15.2||What Familiar Does||300|
|15.4||AppleScript: A Commercial Platform||313|
|Chapter 16||Stimulus-Response PBD: Demonstrating "When" as well as "What"||321|
|16.2||The Syntax of Stimulus-Response||326|
|16.3||The Semantics of Stimulus-Response||336|
|16.4||Feedback and Editing||340|
|Chapter 17||Pavlov: Where PBD Meets Macromedia's Director||345|
|17.3||Objects that React Asynchronously to Events||347|
|Chapter 18||Programming by Analogous Examples||351|
|18.2||The GUI to Program Chasm||354|
|18.3||Programming by Analogous Examples||356|
|Chapter 19||Visual Generalization in Programming by Example||371|
|19.1||If You Can See It, You Should Be Able to Program It||372|
|19.2||What Does Visual Generalization Buy Us?||374|
|19.3||Low-Level Visual Generalization||376|
|19.4||High-Level Visual Generalization||378|
|19.5||Introducing Novel Generalizations: Generalizing on Grids||381|