Today, "end-user programming" brings to mind spreadsheet formulae and macros, SQL database queries, and Rapid Application Development (RAD). Commercial products such as Microsoft Access provide "builders" for queries, data entry forms, and reports, intended for use by persons with no formal training in the art of programming. Demonstrably successful (I myself have witnessed a marketing person create her own contact manager from scratch - though why she bothered, I cannot tell), these tools make no use of PBD: all those clever ideas devised by researchers over the past thirty years seem to have had no influence on these products. Today, the dominant paradigm for end-user programming is form filling, perhaps under the guidance of a Wizard: the programmer selects functional components and sets property values.
RAD tools, especially Visual Basic, have engendered a new class of programmers, whom the organizers of this workshop call "Blended-User-Programmers." This phenomenon is interesting in itself, and efforts directed toward helping them program have high economic value. My concern, however, remains with the common end-user, who seems to have been left behind by all this highly empowering yet nonetheless highly involving technology. As the Call for Participation points out, office software has become bloatware, and support for accomplishing tasks more efficiently comes at the price of a complicated user interface. Meanwhile, users still perform far too many tedious tasks, fail to take advantage of the computer’s ability to formalize their data, and, all too often, driven to perfectionism by their computer’s marvelous capabilities, become slaves to its need for input, until innumerable micromotions and incessant concentration on the screen ruin their hands and eyes. Automating more of their work, especially where such automation affords them a pause in their physical involvement with the computer, or extends their exploitation of computer (and internet) resources beyond their direct physical involvement, promises to make computer work less stressful and more rewarding.
These early end-user programmers were, of course, serious programmers. It was natural for them to think about automating their work, and they enjoyed the challenge of creating robust, generalized macros, the more complex the better. The end-uses that concern me are by no means "programmers" in the traditional sense. By training, and perhaps by disposition, they are not inclined to recognize opportunities for automation, nor to understand the difference between a macro and a generalized macro.
In the course of my research on agents, I ran a Wizard of Oz study, in which end-users of Microsoft Word taught repetitive text editing tasks to a simulated agent named Turvy that watched their actions and understood simple verbal hints. I tested both programmers and non-programmers on working with Turvy, and interviewed them afterward to glean their insights into the process of training a not-very-intelligent agent. While running through the tasks, programmers tended to worry about whether Turvy would generalize correctly; non-programmers did not worry. When giving verbal hints, programmers used terms like "if… then"; non-programmers would say "look for…" Programmers tended to say as little as possible, until Turvy asked them a question; non-programmers tried to explain the task - even its purpose – until they discovered that Turvy did not understand most of the words they used. Several tasks were procedural, involving several steps and some notion of state. Programmers were obviously careful to perform actions in the same order on each example they demonstrated; some even wondered aloud what Turvy would do if they violated this. Non-programmers, on the other hand, paid no heed to this issue, performing actions in freely varying order.
Such differences notwithstanding, by the end of their sessions, both programmers and non-programmers had built up similar mental models of the agent’s capabilities. They recognized that Turvy could pick out repetitions in a stream of actions, that it could generalize from examples and its own mistakes, and that it understood a small vocabulary, which users learned from the questions and feedback Turvy gave them. But during the debriefing, most programmers speculated on the limits of Turvy’s ability to generalize; non-programmers had little to say about this. Several programmers suggested that Turvy could optimize procedures it had learned; no non-programmer thought of this. When asked to evaluate Turvy’s usefulness, programmers gave it lower marks than non-programmers, mainly because they were concerned that it would err.
Among the programmers in this study was a multimedia artist who had recently learned how to program in HyperCard and found himself to be a natural "blended-user-programmer." Indeed, this artist evinced the characteristic responses of programmers to Turvy in the most extreme degree.
A programming system that takes care of the issues listed above must be very different from office applications like Excel or Visual Basic, or from PBD systems like AgentSheets or StageCast Creator. Some of these functions, such as recognizing opportunities for automation, appear impossible to accomplish without using inference. The metaphor that springs to mind for this kind of programming system is an intelligent agent – or what I and my colleagues have called an "instructible" agent, to de-emphasize the need for built-in knowledge. In my own research, I devised an instructible agent model that blends instruction and programming, for a general task model known as Find & Do. Put simply, Find & Do involves searching for objects (e.g. graphics, files or text strings) that match a pattern, and modifying those objects with a macro. The agent instruction protocol I devised enables users teach at their desired level of detail and control. They may perform their work, allowing the agent to predict actions it has inferred from watching, and giving it verbal hints if they wish. They may announce a lesson and teach with examples and hints. They may annotate examples using a property sheet. And they may examine and edit scripts. I built and tested a prototype, which I found very useful for studying interaction and discourse issues, but which failed to convince commercial interests that PBD was in their future, because the application (setting graphic properties and text styles) was too trivial.
Progress on the development of instructible agents has been slow and fitful. The current state of the art is embodied in Rich McDaniel’s GAMUT system for programming games, but GAMUT falls short of Turvy’s learning capabilities and user interface simplicity. Some key components of an instructible agent, for recognizing patterns in user activities, generalizing data descriptions, and interpreting user hints, are available in various research systems (including my own), but the necessary operating system infrastructure, such as access to application data, is mostly missing from "real" computer applications. But the deepest problems and most vehement objections concern interaction between agents and users. As the detractors have oft pointed out in public debates and magazine articles, agents will…
On the "technology" side of research, I see an opportunity to address the issue of identifying repetitious actions in large-scale streams of activity, for inferring tasks and for invoking them. We now have enough computer speed and memory to scan streams at several levels of abstraction, and to measure the relative information in alternative sequence models. Thanks to the Java JDK 1.2 event model, and to the free availability of browser source code, we could experiment with such algorithms in real world applications.
The outlook (pardon the pun) for end-user programming is not so clear. If, as I argued above, there are no end-user programmers, then the fate of this endeavor rests upon user acceptance of instructible agents. We have yet to overcome the known objections to these, let alone test the idea broadly enough to declare with confidence that agents are useful and usable. Unfortunately, certain premature commercial tests of agents, under the trade names "Bob" and "Office Assistant," have antagonized the public. Overcoming this negative bias is one more challenge to research, but at least we need no longer worry about the Hawthorne effect when testing our ideas!
I have no doubt that the ranks of blended-user-programmers will keep growing, especially as the first generation of humans who learned to use computers before they could read matures beyond game-playing and enters the workforce. I believe, however, that even blended-users are end-users much of the time, and will need help to recognize opportunities for automation, and to automate simple repetitive tasks. PBD is the best-known way to solve these problems. I foresee a blending of RAD with PBD: RAD used to design data models and user interfaces; PBD to script prototypes of behavior; and RAD used to polish the scripts. How to blend or supercede the metaphors of programming and teaching, relying as they do on incompatible mental models, remains a great challenge for research.