When is the Pen Mightier Than the Keyboard?

Consider a not-too-distant scenario in which scientists use tablets with digital pens as extensible electronic lab notebooks. Each of these 'notebooks' is the size and weight of a pad of paper but is actually an "infinite" virtual piece of paper that permits sketching ideas for annotated graphical, diagrammatic, and mathematical designs, including proof-of-concept simulations, and engaging in interactions with collaborators on shared portions of the workspace. As they seamlessly move among domains, scientists can have the recognition software assign domain-specific semantic interpretation to the digital ink as appropriate or leave the ink as design or annotation marks.

There is still a large distance between such a vision and what we have today: siloed applications operating in their own windows and relying on a 30-year-old keyboard-and-mouse interaction paradigm. While keyboarding is a fast and natural means of linear text entry, many other tasks such as the entry of mathematical or chemical notations require a cognitively demanding encoding of 2D notations into linear ones that is distracting at best, unnatural at worst. Current technical advances provide exciting new opportunities to interpret digital ink as commands and application objects in appropriate ways. Such a context-sensitive interpretation often involves inferencing and extensive use of gestures fluidly intermixed with notations natural to the domain.

This talk will demo three education-oriented prototypes in math sketching, chemistry, and a diagram creation tool, and then describe some of the research issues that the Microsoft Research Center in Pen-Centric Computing at Brown University is addressing.

---

Andries van Dam is the Thomas J. Watson Jr. Professor of Technology and Education and Professor of Computer Science at Brown University and was Brown’s first Vice President for Research. He has been a member of Brown's faculty since 1965, is a co-founder of Brown’s Computer Science Department, and was its first Chairman, from 1979 to 1985. His research includes work on computer graphics, hypermedia systems, post-WIMP user interfaces, including pen-centric computing, and educational software. He has been working for nearly four decades on systems for creating and reading electronic books with interactive illustrations for use in teaching and research.

He is the co-author of nearly a dozen books, including, Computer Graphics: Principles and Practice, with James D. Foley, Steven K. Feiner, and John F. Hughes (Addison-Wesley 1990) and his more recent, Object Oriented Programming in Java, with Katherine Sanders (Addison-Wesley 2006). He received a B.S. degree, with honors, in Engineering Sciences from Swarthmore College in 1960 and Ph.D. (1966) from the University of Pennsylvania. He is a Fellow of ACM, IEEE, and AAAS, is a member of National Academy of Engineering and the American Academy of Arts & Sciences. He has received honorary doctorates from Darmstadt Technical University in Germany, Swarthmore College and most recently from University of Waterloo in Canada.