Sketch Understanding

Home Research CrossJam Classes Toolbox CPOF SI Info Analogical Processing Command Post Laboratory CPOF Proposal Summary Multi-Dimensional Presentation Nemesis

DARPA CPOF SI Support Information Co-PIs: Ken Forbus Brian M. Dennis Sketch Understanding for Command Post Operations Support The commander and his staff must be able to communicate with the CPoF software directly via sketching. This interaction should be similar to how they communicate with each other, via a combination of gestures and speech, using a shared workspace to create a joint understanding. Traditional COA sketches are often acetate overlays on the operations map. For the CPoF, the shared workspace will use a geographic information system (GIS) as its substrate, with electronic overlays representing information from different sources (e.g., terrain analyses, including COO, OCOKA), and different COA analyses (e.g., initial array of friendly forces, endstate, deception plan, branches and sequels). By understanding these overlays in battlespace terms, the software can participate in the development, evaluation, and analysis of COAs. Achieving this vision requires interpreting digital ink in terms of glyphs registered with a GIS, and using geometric processing, consistent with human spatial reasoning, to extract conceptual relationships from the GIS and from sketch contents. It requires speech interpretation, to capture the verbal annotations of the drawn entities and the narrative aspects of the COA and its analysis. It requires domain theories that express the knowledge underlying operational concepts, so that the system can understand a commander's meaning and can participate in generating and evaluating COAs. We will be working on all four problems, using our HPKB work on qualitative representations of GIS data, our terrain and trafficability domain theories, and our work on diagram understanding and analogical encoding to help interpret digital ink in operational terms, and off-the-shelf speech software for ASR and TTS to integrate drawings and speech for natural interactive sketching. The deliverables for this task include a library of domain theories for operations and multimodal interactive systems for creating and understanding concept sketches and for creating, evaluating, comparing, and monitoring courses of action.

---

This page was last built on 9/21/98; 1:30:20 AM by Brian M. Dennis