the finite string newsletter: site report the esprit projecy loki

the finite string newsletter: site report the esprit projecy loki

THE F I N I T E S T R I N G NEWSLETrER

SITE REPORT

THE ESPRIT PROJECT LOKI

Since the beginning o f 1985, the Research Unit for Infor- mation Science and Artificial Intelligence at the Universi- ty of Hamburg has been participating in a new project:

LOKI - A Logic Oriented Approach to Knowledge and

Data Bases. Supporting a Natural User Interface, LOKI derives its funds from the ESPRIT program of the CEC.

The main contractors are the Belgian Institute of

Management SA (BIM, Bruxelles), the Fraunhofer Insti- tute IAO (Stuttgart), the Cretan Computer Institute

(CCI), and Scicon Limited (London). The research unit is only one of several groups participating in the LOKI project.

The following institutions are involved with other parts of the project: SCS Hamburg, the Technical Insti- tute of Munich (TUM), the Fraunhofer Institute IAO

(Stuttgart), the Cranfield Institute of Technology (Cran- field), Scicon Limited (London), BIM (BruxeUes), and the Cretan Computer Institute (CCI).

The goal of our work here in Hamburg is the design and implementation of a natural user interface for know- ledge and data bases. This interface currently bears the working title "LOQUI". The staff in Hamburg consists of: Walther von H a h n (project leader), Helmut Horaeek,

Claudius Pyka, Martin Schroeder, and T o m Wachtel.

The duration of the first phase is 1 August 1984 - 31

January 1987. Preparations are now being made to apply for a second phase, which is planned for the period from 1 February 1987 through 31 July 1988.

The framework of our work may be sketched as follows:

- Programming in Prolog (BIMProlog).

- P r o g r a m m i n g is taking place on a SUN Workstation

(SUN 2 / 1 2 0 ) by using the operating system UNIX bsd

4.2 version 2.0.

- The natural language interface (NLI) will be dialogue- oriented, and will have a kernel that is independent of application.

- There are plans for a project management system as a

-

pilot application.

We are developing a semantic representation language

LOLA (LOqui LAnguage) for use in analysis and gener- ation.

- A s a support for global dialogue strategies, we are planning an explicit dialogue structure with speech act recognition, taking focus into account.

- W e

are using unification grammar for analysis and generation (in particular, Lexical Functional Grammar, or a version of LFG modified for our purposes).

Presently, we are working on the implementation of the first version of the NLI, which will be completed in early

1986.

The LOQUI group publishes reports and memos, giving information about the state of our work and the research activities of our staff.

More information, including our published material, may be obtained from:

Research Unit for Information Science and Artificial

Intelligence

University of Hamburg

Mittelweg 179

D-2000 Hamburg 13

West G e r m a n y

Tel. (040) 4123 - 4 5 2 9 / 2 5 7 3 / 2 5 7 4 / 3 3 1 5

SITE REPORTS

FROM SEVERAL NATURAL LANGUAGE TECHNOLOGY

BASE CONTRACTS WITH

DARPA'S STRATEGIC COMPUTING PROGRAM

OVERVIEW

Lt. Col. Robert L. Simpson

Information Science Technology Office

D e f e n s e Advanced Research Projects Agency

The overall objective of the Strategic Computing (SC)

Program of the Defense Advanced Research Projects

Agency (DARPA) is to develop and demonstrate a new generation of machine intelligence technology

that

can form the basis for more capable military systems in the future and also maintain a position of world leadership for the US in computer technology. Begun in 1983, SC represents a focused research strategy for accelerating the evolution of new technology and its rapid prototyping in realistic military contexts. The more specific top level goals supporting this single broad objective are to produce technology that will:

1. enable the operation of military systems under critical constraints such as time, information overload, etc.,

2. enable the management of forces/resources under constraints of information overload, geographic distribution, cost of operation, etc., and

3. facilitate the design, manufacture, and maintenance of defense systems within time, performance, quality, reliability, and cost constraints.

Even though capabilities for man-machine interaction will ultimately form an important component of systems in all of these areas, the second of those goals has been selected as the initial area to include emphasis on deci- sion-making aids, including natural language processing.

Subgoals of these top level goals include:

1. T o strengthen/develop areas of science and technolo- gy that enable the building of computer systems need- ed to attain the top level goals. The technologies identified are:

• Artificial Intelligence,

Computational Linguistics, Volume 12, Number 2, April-June 1986 131

The FINITE STRING Newsletter Site Reports

• Software development and machine architectures,

Micro-electronics, and related infrastructure.

2. To build demonstration systems in specific military areas that:

• Provide focus for technology development,

• Provide means for exercising technology in real environments,

• Facilitate manpower training,

• Facilitate development of industrial capability, and

• Facilitate technology transfer to the military.

There are four very ambitious demonstration proto- types being developed within the SC Program. They are:

1. the Pilot's Associate, which will aid the pilot in route planning, aerial target prioritization, evasion of missile threats, and aircraft emergency safety proce- dures during flight;

2. the Autonomous Land Vehicle (ALV), which inte- grates in a major robotic test bed the technologies for dynamic image understanding, knowledge-based route planning with replanning during execution, hosted on new advanced parallel architectures;

3. two battle management projects: one for the Army, which is just getting started, called the AirLand Battle

Management program (ALBM), which will use know- ledge-based systems technology to assist in the gener- ation and evaluation of tactical options and plans at the Corps level;

4. and the other more established program for the Navy, the Fleet Command Center Battle Management

Program (FCCBMP) at Pearl Harbor. The FCCBMP is employing knowledge-based systems and natural language technology in an evolutionary test bed situ- ated in an operational command center to demon- strate and evaluate intelligent decision aids

that

can assist in the evaluation of fleet readiness and explore alternatives during contingencies. It is within this context that the natural language contractors are currently demonstrating the potential of natural language technology.

Competitive awards were made to seven contractors in

1984. Four (BBN Laboratories, USC/Information

Sciences Institute, the University of Pennsylvania, and the University of Massachusetts) are involved in research and development in natural language interfaces; three

(New York University, Systems Development Corpo- ration, and SRI International) are involved in research and development in text processing.

The work focuses on producing and demonstrating two "new generation systems": one for natural language interfaces and another for processing text in free form from military messages. BBN Laboratories serves as the integration contractor in natural language interfaces;

New York University serves as the integration contractor in message processing. The remaining contractors are working on various component technologies, directly or indirectly contributing to the two new generation systems.

BBN LABORATORIES

R e s e a r c h and D e v e l o p m e n t in Natural Language P r o c e s s - ing in the Strategic Computing Program

BBN Laboratories, Inc.

Cambridge, M A 0 2 2 3 8

Staff: Ralph Weischedel (Principal Investigator),

Remko Scha, Edward Walker, Damaris Ayuso,

Andrew Haas, Erhard Hinrichs, R o b e r t Ingria,

Lance Ramshaw, Varda Shaked, David Stallard

1 BACKGROUND

BBN's responsibility is to conduct research and develop- ment in natural language interface technology. This responsibility has three aspects:

• to demonstrate state-of-the-art technology in a Strate- gic Computing application, collecting data regarding the effectiveness of the demonstrated heuristics,

• to conduct research in natural language interface tech- nology, as itemized in the description of JANUS later in this note, and

• to integrate technology from other natural language interface contractors, including USC/Information

Sciences Institute, the University of Pennsylvania, and the University of Massachusetts.

Of the three initial applications described in the over- view, the Fleet Command Center Battle Management

Program (FCCBMP) has been the application providing the domain in which our work is being carried out. The

FCCBMP encompasses the development of expert system capabilities at the Pacific Fleet C o m m a n d Center in

Hawaii, and the development of an integrated natural language interface to these new capabilities as well as to the existing data bases and graphic display facilities. BBN is developing a series of increasingly sophisticated natural language understanding systems which will serve as an integrated interface to several facilities at the Pacific

Fleet Command Center: the Integrated Data Base (IDB), which contains information about ships, their readiness states, their capabilities, etc.; the Operations Support

Group Prototype (OSGP), a graphics system, which can display locations and itineraries of ships on maps; and the

Force Requirements Expert System (FRESH), which is being built by Texas Instruments.

The target users for this application are naval officers involved in decision making at the Pacific Fleet

C o m m a n d Center; these are executives whose effort is better spent on navy problems and decision making than on the details of whiah software system offers a given information capability, how a problem should be divided to make use of the various systems, or how to synthesize the results from several sources into the desired answer.

Currently they do not access the data base or OSGP application programs themselves; instead, on a round- the-clock basis, two operators act as intermediaries between the Navy staff and the computers. The utility of

132 Computational Linguistics, Volume 12, Number 2, April-June 1986

The FINITE S T R I N G Newsletter

Site Reports

a natural language interface in such an environment is clear.

The starting point for development of the natural language interface system at the Pacific Fleet C o m m a n d

Center was the IRUS system, which has been under development at BBN for a n u m b e r of years. A new version of this system, IRUS-86, has been installed in the

FCCBMP testbed area at the Pacific Fleet C o m m a n d

Center for demonstration. Further basic research on the problems of natural language interfacing is continuing, and the results of this and future research will be incor- porated into a next generation natural language interface system called JANUS to be delivered to the Pacific Fleet

C o m m a n d Center at a later date. JANUS will share most of its domain-dependent data with IRUS-86, and it will share other modules as well; IRUS-86 will therefore be able to evolve gradually into the final version of JANUS.

2 I R U S - 8 6 : T H E INITIAL T E S T B E D S Y S T E M

The architecture of IRUS (Bates and B o b r o w 1983) is a cascade consisting of a sequence of translation modules:

• An ATN parser, which produces a syntactic tree.

• A semantic interpreter, which produces a formula of the meaning representation language MRL.

• A postprocessor for resolving anaphora and ellipsis.

• A translation module, which produces a formula of the relational database language ERL (Extended Relational

Language).

• A translation module, which produces a sequence of commands for the underlying database access system.

N o w installed at the Pacific Fleet C o m m a n d Center,

IRUS-86 is a version of IRUS that has b e e n extended in several ways. T w o of these extensions are especially worth mentioning:

• IRUS-86 uses the NIKL system (Maser 1983) to repre- sent its domain model, i.e., the relationships b e t w e e n the predicates and relations of the meaning represen-

tation language MRL. The NIKL domain model supports the system's treatment of semantic anomaly, anaphora, and nominal compounds.

IRUS-86 contains a new module

that

exploits this NIKL domain model to simplify MRL expressions; this makes it possible to translate complex MRL-expressions into

ERL constraints, thus allowing for significant diver- gences between the input English and the structure of the underlying data base (StaUard 1986).

In addition to accessing the NIKL domain model, the parser, semantic interpreter, and MRL-to-ERL translator access other knowledge sources

that

contain domain-de- pendent information:

• the lexicon,

• the semantic interpretation rules for individual concepts,

• the MRL-to-ERL mapping rules for individual MRL constants, which introduce the details of underlying system structure, such as file and field names.

To port IRUS to the N a v y domain, the relevant domain-dependent data had to be supplied to the system.

This task is being accomplished b y personnel at the

Naval Ocean Systems C e n t e r (NOSC). In August 1985,

BBN provided NOSC with an initial p r o t o t y p e system containing small example sets of lexical entries, semantic interpretation rules, and MRL-to-ERL rules; using acqui- sition tools provided b y BBN, NOSC personnel have b e e n entering the rest of the data.

IRUS-86 was delivered to the FRESH developers at

Texas Instruments in J a n u a r y 1986, was installed in a testbed area of the Pacific Fleet C o m m a n d Center in

April 1986, and will be demonstrated in June 1986.

Currently, the lexicon and the d o m a i n - d e p e n d e n t rules of the system only cover a relatively small part of the OSGP capabilities and the files and attributes of the Integrated

D a t a Base. Once enough d a t a have been entered so that the system covers a sufficiently large part of the data base, it will be tried out in actual use by N a v y personnel.

This will enable us to gather data about the w a y the system performs in h real environment, and to fine-tune the system in various respects. F o r instance, IRUS-86 makes use of shallow heuristic methods to address some aspects of natural language understanding such as anaphora and ellipsis for which general solutions are still research issues. The FCCBMP application provides a test bed in which such heuristic methods can be evaluated, and enhancements to them developed and tested, as part of the evolutionary technological growth intended to continue throughout the Natural Language Technology effort of the Strategic Computing Program.

3 F U N C T I O N A L G O A L S F O R J A N U S

The IRUS-86 system excels by its clean, modular struc- ture, its broad s y n t a c t i c / s e m a n t i c coverage, its sophisti- cated domain model, and its systematic treatment of discrepancies b e t w e e n the English lexicon and the data- base structure. We thus expect that it will demonstrate considerable utility as an interface c o m p o n e n t in the

FCCBMP application. Nevertheless, IRUS-86 shares with other current systems several limitations that should be overcome if natural language interfaces are to b e c o m e truly "natural". In developing JANUS, the successor of

IRUS-86, we shall attempt to o v e r c o m e some of those limitations. The areas of increased functionality we are considering are: semantics and knowledge represen- tation, ill-formedness, discourse, cooperativeness, multi- ple underlying systems, and knowledge acquisition.

3.1 S E M A N T I C S AND K N O W L E D G E R E P R E S E N T A T I O N

IRUS-86, like most other current systems, represents sentence meanings as formulas of a logical language

that

is a slight extension of first-order logic. As a conse- quence, m a n y important p h e n o m e n a in English have no equivalent in the meaning representation language, and cannot be dealt with correctly, e.g., modalities, proposi- tional attitudes, generics, collective quantification, and context-dependence. Thus, one foregoes one of the m o s t important potential assets of a natural language interface:

Computational Linguistics, Volume 12, Number 2, April-June 1986 1 3 3

The FINITE STRING Newsletter

Site Reports

the capacity of expressing complex semantic structures in a succinct and comfortable way.

In JANUS, therefore, we will adopt a new meaning representation language

that

combines features from

PHLIQAI's enriched lambda-calculus (Scha 1976) with ideas underlying Montague's Intensional Logic

(Montague 1970), and possibly a distributed quote-oper- ator (Haas 1986). It will have sufficient expressive power to incorporate a version of Carlson's treatment of gener- ics (Carlson 1979), a version of Scha's treatment of quantification (Scha 1981), Montague's treatment of modality, and various possible approaches to proposi- tional attitudes and context-dependence.

In adopting a higher order logic as proposed, one confronts problems of formula simplification and the need to apply meaning postulates to reduce the semantic representation of an input sentence to an expression appropriate to the underlying system, e.g., a relational algebra expression in the case that the underlying system is a data base. To do this, we will investigate the limited inference mechanisms of KL-TWO (Moser 1983, Vilain

1985), following up on our previous work (Stallard

1986). The advantage of these inference mechanisms is their tractability; discovering their power and limitations in this complex problem domain should be an interesting result.

3.2 DISCOURSE

The meaning of a sentence depends in many ways on the context set up by the preceding discourse. IRUS and other systems, however, currently ignore most 0f these dependencies, and employ a rather shallow i~odel of discourse structure. To allow the user to exploit the full expressive potential of a natural language interaction, the system must track topics, reference times, possible ante- cedents for anaphora, etc.; it must be able to recognize the constituent units of a discourse and the subordination or coordination relations obtaining between them. A substantial amount of work has been done already on several of these issues, much of it by BBN researchers

(Sidner 1985, Hinrichs 1981, Polanyi 1984, Grosz and

Sidner 1986). Research in this area continues under a separate DARPA-funded contract. We expect to be able to integrate some of the results of that research in the

J A N U S system.

3.3 ILL-FORMEDNESS

A natural interface system should be forgiving of a user's deviations from its expectations, be they misspellings, typographical errors, unknown words, poor syntax, incor- rect presuppositions, fragmentary forms, or violated selection restrictions. Empirical studies show that as much as 2 5 % of the input to database query systems is ill-formed.

IRUS currently~ handles some classes of ill-formedness by using a combin~/tion of shallow heuristics and user interaction. It can correct for typographical misspellings, for omitted determiners or prepositions, and for some ungrammaticalities, like determiner-noun and subject- verb disagreement. The JANUS system will employ a more general approach to ill-formedness that will handle a larger class of ungrammatical constructions, a larger class of word selection problems, and that will also explore correcting several types of semantic ill-formed- ness.

These capabilities have major implications for the control of the understanding process, since considering such possibilities can exponentially expand the search space. Maintaining control will require care in integrating the ill-formedness capability into the rest of the system, and also in making maximal use of the guidance that can be derived from a model of the discourse and user's goals to constrain the search.

3.4 COOPERATIVENESS

A truly helpful system should react not to the literal meaning of a sentence but to its perceived intent. If in the context of a given application it is possible to charac- terize the goals that a user may be expected to be pursu- ing through his interaction with the system, the system should try to infer from the user-input what the underly- ing goal could be. A system can do this by accessing a goal-subgoal hierarchy

that

links the speech acts expressed by individual utterances to the global goals that the user may have. This strategy has been applied successfully to rather small domains (Allen 1983, Sidner

1985). We wish to investigate whether it carries over to the FCCBMP applications.

3.5 MODELLING THE CAPABILITIES OF MULTIPLE SYSTEMS

The way in which IRUS-86 decides whether an input sentence translates into an IDB query or an

O S G P

command may be refined. There is a need for work on what kind of knowledge would be necessary to interface smoothly and intelligently to multiple underlying systems.

A reasoning component is needed that can determine which underlying system or systems can best fulfill a user's request. Such a reasoning c o m p o n e n t would have to combine a model of the capabilities of the underlying systems with a model of the user goals and current intentions in the discourse context in order to choose the correct system(s). Such a model would also be useful for providing supporting information to the user.

3.6 KNOWLEDGE ACQUISITION

Further research is also called for to expand the power of the knowledge acquisition tools used in adding to the lexicon, the set of case frame rules, the model of domain predicates, and the set of transformation rules between the Meaning Representation Language and the languages of the underlying systems. The acquisition tools available in IRUS, unlike those in some other systems, are not tied to the specific fields and relations in the underlying data base. The acquisition tools should work on the higher level of the domain model, since that provides a more general and transportable result. The knowledge acquisi-

134 Computational Linguistics, Volume 12, Number 2, April-June 1986

The FINITE STRING Newsletter Site Reports

tion facilities f o r JANUS will also n e e d to be redesigned to support and to m a k e maximal use o f the p o w e r of the n e w m e a n i n g representation language b a s e d o n inten- sional logic.

4 NEW UNDERLYING TECHNOLOGIES

4.1 COPING WITH AMBIGUITY

T h e n e w functionalities we described in the previous section, a n d the techniques we intend to use to achieve them, raise an issue

t h a t

has i m p o r t a n t c o n s e q u e n c e s f o r the design of JANUS: we will be faced with an explosion in the n u m b e r of interpretations that the system will have to process; every sentence will be manifold ambiguous.

O n e source of this p h e n o m e n o n is the i m p r o v e m e n t o f the semantic coverage a n d the b r o a d e n i n g o f the discourse context. Distinctions a n d ambiguities which so far were ignored will be dealt with: for instance, different interpretation a n d scopes of quantifiers will be consid- ered, and different a n t e c e d e n t s f o r p r o n o u n s . E v e n m o r e serious is the processing o f ill-formed sentences, which m a y require that some constraint be relaxed, while the only w a y to find o u t which o n e is to try all partial inter- pretations to see which one c a n be e x t e n d e d to a complete interpretation after relaxing o n e or m o r e constraints.

T o cut d o w n on the processing of spurious interpreta- tions, it is very i m p o r t a n t that interpretations of sentences and their constituents be tested f o r plausibility at an early stage. Different techniques m u s t p r o b a b l y be used in c o n j u n c t i o n :

• Simplification t r a n s f o r m a t i o n s m a y s h o w t h a t an inter- pretation is absurd, b y reducing it to

true

or

f a l s e

or the e m p t y set.

• T h e discourse c o n t e x t and the m o d e l of the user's goals impose constraints o n expected sentences.

4.2 PARALLEL PARSING

Since some of the techniques we intend to use to fight the ambiguity explosion are themselves r a t h e r c o m p u t a - tion-intensive, it is clearly unavoidable that the i m p r o v e d system functionality we aim f o r will lead to a consider- able increase in the a m o u n t of processing required. T o avoid a serious decrease of the n e w systeni's response times, we will therefore m o v e it to a suitable parallel machine such as B B N ' s Butterfly or M o n a r c h , r u n n i n g a parallel COMMON LISP. This in itself has rather serious c o n s e q u e n c e s f o r the software design. It means t h a t f r o m the outset we will keep parallelizability o f the software in mind.

W e have b e g u n to address this issue in the area o f syntax. A n e w declarative g r a m m a r is being written, which will ultimately have a c o v e r a g e o f English larger than the current RUS grammar. T h e g r a m m a r is written in a side-effect-free formalism (a c o n t e x t - f r e e g r a m m a r with variables) so that we m a y explore different parsing algorithms that are easily parallelizable. T h e first such algorithm was i m p l e m e n t e d in M a y 1986 o n BBN's

Butterfly.

5 CONTRIBUTIONS FROM

OTHER SITES

5.1 ISI/UMASS: GENERATION

W e should n o t expect t h a t JANUS will always be able to assess correctly which interpretation of a sentence is the i n t e n d e d one. I n light o f such situations, it is v e r y i m p o r - t a n t that the s y s t e m c a n give a p a r a p h r a s e of the input to the user, which shows the s y s t e m ' s interpretation. This m a y be d o n e either explicitly or as part of the answer.

T o be able to develop such capabilities, w o r k o n N a t u r a l

L a n g u a g e G e n e r a t i o n is needed. A t U S C / I S I a project directed b y William M a n n a n d N o r m a n S o n d h e i m e r is u n d e r w a y to develop the g e n e r a t i o n s y s t e m PENMAN, using the N I G E L systemic g r a m m a r . P E N M A N will be integrated to b e c o m e the g e n e r a t i o n c o m p o n e n t of

JANUS. PENMAN itself consists of several s u b c o m p o - nents. Some o f these, specifically the " t e x t p l a n n i n g " c o m p o n e n t , will be d e v e l o p e d t h r o u g h joint w o r k b e t w e e n U S C / I S I a n d D a v i d M c D o n a l d at the University o f Massachusetts, b a s e d o n the latter's experience with the MUMBLE system.

5.2 UPENN: COOPERATION AND CLARIFICATION

U n d e r the direction of A r a v i n d Joshi a n d B o n n i e W e b b e r of the University o f Pennsylvania, several f o c u s s e d studies have b e e n carried o u t at U P e n n to investigate various aspects o f c o o p e r a t i v e s y s t e m b e h a v i o u r and clar- ification interactions. ( F o r m o r e detail, see their r e p o r t below.) As part o f the Strategic C o m p u t i n g N a t u r a l

L a n g u a g e effort, U P e n n will eventually d e v e l o p this into a m o d u l e

t h a t

c a n be integrated into JANUS to further e n h a n c e its capabilities.

REFERENCES

Allen, J.F. 1 9 8 3 Recognizing Intentions from Natural Language

Utterances. In Brady, M. and Berwiek, R.C., Eds,

Computational

Models of Discourse.

Massachusetts Institute Technology Press,

Cambridge, Massachusetts: 107-166.

Bates, M. and Bobrow, R.J. 1983 A Transportable Natural Language

Interface for Information Retrieval. In

Proceedings o f the 6th Annual

International ACM SIGIR Conference.

ACM Special Interest Group on Information Retrieval and American Society for Information

Science, Washington, D. C. (June).

Carlson, G. 1979

Reference to Kinds in English.

Garland Press, New

York.

Grosz, B.J. and Sidner, C.L. 1986 The Structures of Discourse Struc- ture. In Polanyi, L., Ed.,

Discourse Structure.

Ablex Publishers,

Norwoods, New Jersey.

Haas, A.R. 1986 A Syntactic Theory of Belief and Action.

Artificial

Intelligence.

Hinrichs, E. 1981 Temporale Anaphora im Englischen. Unpublished ms., University of Tuebingen.

Montague, R. 1970 Pragmatics and Intensional Logic.

Synthese

22:

68-94.

Moser, M.G. 1983 An Overview of NIKL, the New Implementation of

KL-ONE. In Sidner, C. L., et al., Eds.,

Research in Knowledge Repre- sentation for Natural Language Understanding - Annual Report, 1

September 1982 - 31 August 1983.

5421: 7-26.

BBN Laboratories Report No.

Polanyi, L. and Scha, R. 1984 A Syntactic Approach to Discourse

Semantics. In

Proceedings of International Conference on Computa- tional Linguistics.

Stanford University, Stanford, California.

Scha, R.J.H. 1976 Semantic Types in PHLIQA1. In

Proceedings o f the

6th International Conference on Computational Linguistics.

Computational Linguistics, Volume 12, Number 2, April-June 1986 135

The FINITE STRING Newsletter

Site Reports

Scha, R.J.H. 1981 Distributive, Collective and Cumulative Quantifica- tion. Formal Methods in the Study of Language, Part 2. Mathema- tisch Centrum, Amsterdam: 483-512. Reprinted in Groenedijk,

J.A.G.; Janssen, T.M.V.; and Stokhof, M.B.J., Eds.,

Truth, Interpre-

tation and Information. GRASS 3. Dordrecht, Foils.

Sidner, C.L. 1985 Plan Parsing for Intended Response Recognition in

Discourse. Computational Intelligence 1 (1): 1-10.

Stallard, D.G. 1986 A Terminological Simplification Transformation for Natural Language Question-Answering Systems. In

Proceedings of the 24th Annual Meeting of the Association for Computational

Linguistics. Association for Computational Linguistics (July).

Vilain, M. 1985 The Restricted Language Architecture of a Hybrid

Representation System. In

Proceedings of IJCAI85, International

Joint Conference on Artificial Intelligence, Inc. Morgan Kaufmann

Publishers, Inc., Los Angeles, California: 547-551.

UNIVERSITY OF PENNSYLVANIA

Research

in Natural Language

Processing

University of Pennsylvania

Department of Computer and Information Science

Faculty: Aravind Joshi, Tim Finin, Dale Miller, Lokendra

Shastri, and Bonnie Webber

Students: Brant Cheikes, John Dowding, Amy Felty,

Ellen Hays, Robert Kass, Ron Katriel, Sitaram

Lanka, Megan Moser, Gopalan Nadathur,

MaryAngela Papalaskaris, Martha Pollack, Robert

Rubinoff, Yves Schabes, Ethel Schuster, Sunil

Shende, Jill Smudski, Vijayshankar, David Weir,

Blair Whitaker

Facilities: L'INC (Langauge, Information, and Computa- tion) laboratory, which consists of a dedicated VAX

11/785, 10 Symbolics Lisp machines, 7 HP

68020-based AI workstations, a SUN workstation, several Macintoshes, and a laser printer. These machines are networked together and to other research facilities in the department.

This is a brief report summarizing our work to date and our intermediate and long term goals. See

Abstracts of

Current Literature in this issue for a summary of some of our publications on this work.

1 MAJOR THRUST

Natural language interfaces providing support for many different communicative functions:

• Providing definitions of concepts;

• Recognizing and correcting user misconceptions;

• Providing explanations;

• Offering to provide information later, when known;

• Verifying and demonstrating understanding;

• Exploiting and enriching the context of natural language discourse between user and system.

1.1 WORK TO DATE

• Integration of RUS-TEXT-MUMBLE (RTM) - This effort involves integrating three natural language system components (BBN's RUS parser-interpreter,

McKeown's TEXT system (developed at Penn), and

McDonald's MUMBLE system (received from U. Mass in January 1985). This integration of three independ- ently developed systems has required substantial effort.

This version of RTM (to be completed in May 1986)

- a c c e p t s a limited number of English language requests for definitions of, descriptions of, or comparisons between terms in the ONR database used by Kathy M c K e o w n in her development of

TEXT;

- f o r m u l a t e s appropriate responses using TEXT and outputs those responses in English using MUMBLE; and

- runs on a SYMBOLICS Lisp machine.

This work has been done by Moser, Whitaker and

Rubinoff.

• Initial work on incorporating a sense of relevance in monitor offers. Mays's dissertation work on monitor offers was limited to issues of competency. This work is being done by Cheikes and Schabes.

• Completion of McCoy's dissertation work on correct- ing certain types of object-related misconceptions and implementation of a system called ROMPER, which generates such corrections. (MUMBLE is used as the tactical generation component of this system as well.)

• Completion of Hirschberg's dissertation work on scalar impllcatures and their use in constructing non-mislead- ing responses.

• Completion of Pollack's dissertation work on plan inference in which user and system beliefs about actions and plans are decoupled.

• Continuation of work on integrating scalar-implica- tures-based reasoning within a general framework of circumscription-based non-monotonic reasoning.

• Development of methods for converting proofs in a system akin to first-order resolution into natural deduction (ND) proofs, which are then reorganized into cohesive paragraphs using Chester's 1976 algo- rithm.

• Development of methods of converting modal resolu- tion proofs into modal ND proofs and higher-order resolution proofs into higher-order ND proofs.

• Initial development of domain-independent tools for expressing and reasoning about user models - in partic- ular, for defining hierarchies of stereotypical users, representing individual users, and drawing inferences about them using a default logic.

• Continuation of basic research on local coherence of discourse using the notions of centering and syntax, semantics, and parsing of tree adjoining grammars.

2 FUTURE PLANS

Having gained the experience of integrating three natural language systems and carrying out some of the basic research as. described in the previous section, we have now developed the plan described below, which summa- rizes the near- and long-term goals.

136 Computational Linguistics, Volume 12, Number 2, April-June 1986

The FINITE STRING Newsletter

Site Reports

2.1 NEAR-TERM GOALS

We have three tangible goals for the next year:

• Completing the RTM demonstration system (using the existing domain and knowledge representation) and producing a videotape

that

explains and demonstrates it.

• Developing TEXT into a more modular tool for defin- ing and comparing terms, on the order of RUS and

MUMBLE. This will eliminate its tie to a particular knowledge representation and increase its portability.

• Acquiring familiarity with the PENMAN approach to

NL generation through acting as a beta-test site for

NIGEL.

2.2 L O N G - T E R M GOALS

2.2.1 SUPPORT FOR NL D E F I N I T I O N S -

ENRICHED K N O W L E D G E REPRESENTATION

In our original proposal, we stated our intention of employing a richer knowledge representation as the basis for our work on text generation, especially for construct- ing definitions. Our original idea was to m a k e use of

BBN's NIKL system. In the past year though, we have become aware of some of NIKL's limitations, which essentially make it non-optimal, even as a next step, for our text generation work. On the other hand, we have identified several features with which a NIKL-like language could be enriched to make it more suitable for our work:

associating non-definitional information with concepts in a way that maintains the underlying structure of that information, without interfering with NIKL's automatic classification mechanism;

• associating "evidential" information with concepts, especially frequency information - h o w often the concept is known to display particular features;

• allowing for what appears to be conflicting information coming down through inheritance - e.g., information that is contrary to expectations grounded in an alterna- tive perspective on a concept;

• allowing mutual definition of concepts - each being defined with reference to the others in a set;

incorporating notions of time and change

-

allowing the defining properties and evidential properties of concepts to include how they change over time;

• allowing assertions about usual relations b e t w e e n prop- erties of subtypes.

W o r k on an enriched knowledge representation that includes all these features in a well-motivated way will take several years. H o w e v e r one that includes at least the first three of them can p r o b a b l y be developed over the next two years, with work on employing it in text gener- ation beginning six months to a year after the start of that work.

2.2.2 S U P P O R T O F NL D E F I N I T I O N S -

USE O F D I S C O U R S E AND USER M O D E L S

The TEXT system, as it is currently structured, will produce the same definition for a concept (or comparison b e t w e e n two concepts) whenever it is asked. It does not take into account what the user m a y have already found out about the concept, or what it is implicitly being contrasted with (e.g., some other concept the user has recently asked about), or what the user's goal is in making the request. Hence, other directions in which we would like to take this definitional/clarificational capabil-

-

ity is to increase its sensitivity to

- t h e discourse history, to avoid repetition and possibly to take advantage of the additional clarity brought b y contrasting a new t e r m to one explained before; the user's level of expertise, to avoid either stating the obvious or going more deeply into a concept than the user can understand; and

- t h e

user's goals, to focus on those aspects of the concept being defined (or concepts being compared) that are significant to the current task. (The latter is related to the notion of " p e r s p e c t i v e " used in K a t h y

M c C o y ' s recent thesis here.)

F o r b o t h these aspects of user modelling (in contrast with the first point, which can be developed using the current discourse alone), we will draw on the other work being done here on domain-independent user-modelling mech- anisms. This proposed work must be done in a domain in which tasks can be characterized and recognized. Thus we plan to do this initially in an investment advising domain we have started to develop. W o r k on incorporat- ing and using discourse history will involve about a one- year effort, once the knowledge base is built. W o r k on incorporating and using a model of a user's expertise and goals will take more time, on the order of two to three years.

2.2.3 EXPLANATIONS

Again in our original proposal, we p r o p o s e d work on constructing natural language explanations - more specif- ically, on ways to loosen the current tight coupling b e t w e e n the f o r m of the system's proof of some state- ment to the f o r m of its explanation of why the statement is t r u e . This coupling has kept systems that should be able to explain their reasoning f r o m employing stronger proof methods that do not have a natural, understandable f o r m of presentation to their h u m a n users.

Our immediate goals involve:

• developing a demonstration system

that

responds to NL queries posed to RUS by doing an efficient first-order resolution-based proof, transforming that proof into an

ND proof, organizing that p r o o f according to an improved version of the Chester algorithm, and then producing an English version of the text using

MUMBLE or NIGEL.

• abstracting f r o m the three separate sets of proof conversion methods (noted under WORK TO DATE) into general methods of transforming any resolution-

Computational Linguistics, Volume 12, Number 2, April-June 1986 137

The FINITE STRING Newsletter

Site Reports

style proof in any logic into its corresponding ND proof.

• determining whether existing methods of organizing first-order ND proofs into paragraphs are applicable to

ND proofs in these stronger logics or whether m o r e must be done to produce high-quality, cohesive, under- standable text.

Our l o n g - t e r m goals remain as stated in our original proposal - the production of explanations sensitive to users' beliefs, expertise, desired level of detail and expec- tations. In this long-term research, we see taking exper- tise and desired level of detail into account in determining h o w much of the ND proof is made explicit.

O f more interest is h o w users' beliefs and expectations should affect the explanations. W o r k on scientific expla- nation has shown that central to the explanation of what is the case is a set of alternative situations

that

are not the case. One explains

what is

in contrast to

what is not.

However, this requires additional work, to prove of each of the alternatives (which m a y be given explicitly by the user -

"'why this and not that?" -

or inferred f r o m the system's model of the user's expectations) that it is not true. Our planned approach involves guiding the (failing) proof of each alternative against the successful proof.

The point is that although there m a y be m a n y failing proofs of each alternative, the m o s t relevant of these in the current situation is the one that is analogous - up to the point of failure - to the original successful proof.

This technique should not only provide relevant informa- tion but should also be efficient in reducing the search space. We expect this work to take on the order of two to three years, provided we have enough resources to pursue it in parallel with our more near-term goals.

2.2.4 NATURAL LANGUAGE PARSING AND GENERATION

While using the RUS system, we will continue our work on tree adjoining g r a m m a r (TAG) b o t h f r o m the parsing and generation points of view. TAGs lead to some attrac- tive approaches to parallelizing parsing and also seem to provide natural planning units for generation. This work will be integrated with our future work on parsing and generation. Our first language generator (used b y TEXT) was one based on K a y ' s Functional Unification G r a m - mar. While theoretically elegant, it was unacceptably slow (in its straightforward implementation), leading us last year to import the MUMBLE generator f r o m M c D o n - ald at University of Massachusetts and adapt it to w o r k with TEXT. Using MUMBLE has produced a 60-fold speed-up in generation time. However, adapting

M U M B L E t o w o r k with TEXT and, independently, with two other systems has made us aware of MUMBLE's limi- tations, primarily its lack of knowledge of words or gram- mar. Essentially, MUMBLE's knowledge is limited to how to realize particular message units (i.e., to choose an acceptable one f r o m an a priori specified set of choices), given constraint~ already imposed b y message units that have already been realized. The large amount of w o r k that must be invested in building a MUMBLE lexicon and the lack of inter-application portability of anything but the control structure comes f r o m this fact - that one has to completely specify each set of choices b e f o r e h a n d for each message unit and the sets are completely application specific. We propose to w o r k on the d e v e l o p m e n t of a new architecture, including our work on tags, that avoids these limitations b y having m o r e knowledge of syntax and words and hence is more portable b e t w e e n applica- tions. T h e time frame for this project is approximately three years.

2.2.5 ANAPHORA RESOLUTION

The RUS p a r s e r / i n t e r p r e t e r we received f r o m BBN uses a limited m e t h o d of resolving definite pronouns and noun phrases that is only a bit m o r e advanced than the one originally developed for BBN's LUNAR system b a c k in

1971. Since then, there have b e e n m a j o r theoretical advances in our understanding of discourse a n a p h o r a (in the works of G r o s z (at SRI), Joshi, Sidner (at BBN),

Webber, and Weinstein), but these theoretical advances have not yet found their w a y into natural language understanding systems. We feel strongly qualified to undertake this work, having two of the m a j o r participants

(Joshi and W e b b e r ) here at P e n n already, and w a n t to do so. F o r us, it is b o t h of research interest and of practical importance, since it can m e a n a m a j o r i m p r o v e m e n t in system's understanding abilities. We will also integrate our work on tags with this effort as it relates to parsing and generation. This w o r k will also c o m p l e m e n t addi- tional work being done here on a theoretical and c o m p u - tational account of anaphoric reference to actions and events. We see this work as taking a b o u t two to two and a half years.

2.2.6 USER MODELING

The need for systems to model the knowledge and beliefs of their users has already b e e n pointed out. We plan to address a n u m b e r of issues

that

underlie the successful development and incorporation of explicit user models.

O u r current domain-independent user-modelling system,

GUMS, provides mechanisms for defining hierarchies of stereotypical users, representing individual users, and drawing inferences a b o u t t h e m using a rich default logic.

We will continue to develop this system as a tool

that will

support the user modeling needs of various applications.

We also plan to study the p r o b l e m of h o w new know- ledge of individual users can be derived f r o m their regu- lar interaction; that is, h o w relevant information a b o u t users can be inferred f r o m their queries and responses. In other situations it m a y b e c o m e necessary for the system to explicitly pose a few crucial questions to the user to determine what he or she does and does not know.

2.2.7 SYSTEM INTEGRATION

Finally, we plan to begin work on system integration. In recent years, we have identified m a n y types of behavior that interfaces to database systems and expert systems should demonstrate. Beginning with K a p l a n ' s work on

138 Computational Linguistics, Volume 12, Number 2, April-June 1986

The FINITE STRING Newsletter Site Reports

recognizing and responding to existential presupposition failures in his COOP system, we have developed and produced several modules, each demonstrating another type of desired, behavior. These include the ability to recognize and respond to type failures, the ability to respond to object-related misconceptions, the ability to calculate and offer competent database monitors, the ability to use scalar implicatures to convey additional information, and the ability to respond to a class of

"inappropriate" queries, and various paraphrase abilities.

Following the publication of Kaplan's thesis, the features of his COOP~ system were soon incorporated into several database interfaces (both natural slanguage and formal query language). This gave the resulting systems the ability to produce two types of responses: either a direct answer, if there was one, or a statement concern- ing the absence of individuals satisfying some description in the given query. Now we plan to tackle the more significant problem related to this:

Given a system that is able to call upon a variety of response strategies, how does it decide what to do in a given circumstance? This is the issue we plan to explore by investigating the integration of multiple communica- tive behaviors. Given a system with several different types of useful behaviors, which can be combined in vari- ous ways, can one efficiently and effectively coordinate a response that is better (i.e., more useful, more helpful and more understandable) than simply a (direct) answer?

While we speculate that it will be the case that identify- ing what one might consider the best response might take complex reasoning about the user's goals, level of exper- tise, and need-to-know with respect to what the answer

(if any) actually is, we also plan to look at how, with more limited resources, we can still improve system behavior.

This aspect of our future plans is the most long term, involving both the actual component integration itself (in which, in many cases, it is only the basic ideas that can be carded over, where the component must be re-pro- grammed entirely to fit into the integrated system) and the development of that part of the total system that reasons about what kind of response(s) to give. The time frame here is approximately four years.

2.2.8 ARCHITECTURE

We plan to investigate parallel and connectionist archi- tectures and algorithms for realizing our systems, espe- cially those for knowledge representation, reasoning, explanations, and integrated parsing and generation.

UNIVERSITY OF MASSACHUSETTS

The COUNSELOR Project at the University o f M a s s a c h u - setts

David D. M c D o n a l d and J a m e s D. Pustejovsky

Department of Computer and Information S c i e n c e

University of Massachusetts,

Amherst, Massachusetts 0 1 0 0 3

Participants in the COUNSELOR Project,

Fall 1984 through Summer 1986:

Principal Investigators: Edwina L. Rissland~ David D.

McDonald, W e n d y G. L e h n e r t

Research Associates: Beverly Woolf, James D.

Pustejovsky

Graduate Students: Marie M. Vaughan, Brian Stucky,

Penelope Sibun, Seth Rosenberg, Kelly Murray,

Kevin Gallagher, J o A n n M. Brooks, John Brolio,

Sabine Bergler, Kevin D. Ashley, Scott D.

Anderson

1

INTRODUCTION

The COUNSELOR Project began in the fall of 1984 with the goal of exploring basic problems in discourse struc- ture and text processing within an integrated interface to a strong expert system. The program we have developed,

COUNSELOR, integrates separately developed compo- nents for natural language generation (MUMBLE: see

McDonald and Pustejovsky 1985a,b,c), parsing (PLUM:

Lehnert and Rosenberg 1985), and case-based legal reasoning (HYPO: Ashley 1986, Ashley and Rissland

1985). It adds a newly developed component, CICERO

(Pustejovsky 1986), positioned between the two text processors and the expert system; CICERO is responsible for managing textual inferences ("reading between the lines") by using common sense models of legal events.

COUNSELOR can provide advise to an attorney about how to argue eases involving violations of trade secret law in the computer field. The attorney presents the facts of their case to the system, which may ask questions to elicit other facts that it knows to be relevant. The system then suggests lines of argument that the attorney might use, drawing on its library of litigated cases to find ones with analogous dimensions.

At its present state of development, COUNSELOR can handle simple variations on a single scenario, exemplified by the following dialog:

User: I represent a client named HackInc, who wants to sue SwipeInc and L e r o y Soleil for misappropriating trade secrets in connection with software developed by my client. Hacklnc markets the soft- ware, known as Autoteli, a program to automate some of a bank teller's func- tions, to the banking industry.

COUNSELOR: Did Soleil work for Hacklnc.?

User: Yes, he was a key employee on the

Autotell project.

COUNSELOR: Did he later work for SwipeInc.?

User: Yes.

COUNSELOR: You can argue that there is an implied agreement arising out of Soleil's employ- ment with Hacklnc that he not disclose any trade secret information to which he gained access by virtue of his employ- ment.

Computational Linguistics, Volume 12, Number 2, April-June 1986 139

The FINITE STRING Newsletter Site Reports

2 MOTIVATIONS

Consequential results in natural language research will only come from working with a strong underlying program whose communicative needs will challenge the capabilities of state of the art of language interfaces. As a group, we are not interested in building yet another question answering system: our goal is to understand the structure of discourse. We believe that an effective place to begin is with task specific, mixed initiative dialog where the participants' goals cannot be satisfied b y single utterances. Working with a legal reasoning system like

Kevin Ashley and Edwina Rissland's HYPO provides particular challenges to natural language research:

1. Legal text is structurally complex. The need to avoid ambiguity leads to d e e p l y - e m b e d d e d clauses and heavy noun phrases.

2. As both the user and the system have a thorough knowledge of the law, they communicate vastly more information in conversations about legal arguments than ever appears in their literal utterances.

3. HYPO's role as an advisory system creates a natural motivation to communicate t .hrough language.

4. Legal cases are large, complex objects that can be viewed f r o m m a n y alternative perspectives. The purpose for which a case is being described strongly influences which of its attributes are salient and h o w that information should be structured as a text.

3 COMPONENT PARTS

We began the project with three partially developed components, HYPO, MUMBLE, and PLUM, each designed with independent motivations. An initial tension was whether to convert aspects of these programs that did not seem apt in their new setting, or alternatively to interpose new components between them to smooth out the differ- ences. We concluded that the motivations underlying each c o m p o n e n t were strong enough that we should not change them just because they were n o w working togeth- er. HYPO reasons with cases and hypotheticals. Actually litigated legal cases are encoded and indexed b y

"dimensions", which capture the utility of a case for making a particular kind of argument. When evaluating new cases, HYPO first analyzes them in terms of the dimensions they involve. Relevant cases are then retrieved to guide the reasoning. The system m a y ask pertinent questions about facts n o w found to be relevant.

When the analysis is complete, HYPO describes the argu- ments available to the user, and responses and counter responses that m a y follow.

MUMBLE, the linguistic c o m p o n e n t for generation, is responsible for realizing conceptual specifications as grammatical text cohesive with the discourse that proceeds it. MUMBLE works within a description directed framework. Its input specification is a description of the message the underlying program wants to communicate. This description is executed incre- mentally, producing an intermediate linguistic represen- tation which defines the text's grammatical relations and imposes constraints on further realization. This surface structure description is concurrently executed, producing the actual text.

PLUM is a conceptual analyzer

t h a t

has b e e n given a well defined schematic structure so it can be easily extended. It parses b y doing prediction and completion over semantic concepts implied b y the words rather than over syntactic categories. As in other conceptual analyz- ers, no explicit surface structure is recovered. PLUM's output is the set of completed frames.

CICERO is a new component, a discourse and infer- ence m a n a g e r b e t w e e n the language c o m p o n e n t s and the expert system. F r o m the understanding side, CICERO must integrate the clause b y clause output of the parser into the larger discourse context, recognizing, for e x a m - ple, when noun phrases refer to the same object. In interpreting these small, lexically derived frames,

CICERO draws on its o w n representation of events which bridges the gap b e t w e e n the w a y such information is expressed in language and the w a y it is organized for expert legal reasoning. F o r generation, CICERO is responsible for planning the message that is given to the generator. In particular, it determines what information should be included and what m a y be omitted as inferable, and it selects pivotal lexicai items with appropriate perspective and rhetorical force.

4 FUTURE DIRECTIONS

While the accomplishments of the individual c o m p o n e n t s of COUNSELOR are interesting in their o w n right, the greatest effect of the project has b e e n to provide a work- bench for studying the problems of language in an inte- grated context. Perennial problems in anaphora, lexical semantics, aspect, etc. b e c o m e m o r e tractable in an inte- grated system where there is a discourse context and intensional motivation. There are also semantic gener- alizations b e t w e e n the level at which the text processors operate and the level of the expert system which are more easily captured w h e n parsing and generation can be studied in unison. O n a larger scale, an explicit discourse manager, a requisite for m o r e complex dialogs, can only be developed once an integrated system exists.

REFERENCES

Ashley, Kevin D. 1986 Modelling Legal Argument: Reasoning with

Cases

and Hypotheticals - a Thesis Proposal. Technical Report 10,

The COUNSELOR Project, Department of Computer and Informa- tion Science, University of Massachusetts at Amherst.

Ashley, Kevin D. and Rissland, Edwina L. 1985 Toward Modelling

Legal Argument.

Proceedings o f the 2rid International Congress

LOGICA, INFORMATICA,

DIRITTO.

Instituto Per La Documentazione

Giuridica, Florence, Italy.

Brooks, JoAnn M. 1985 Themis: A Discourse Manager. Unpublished

Master's thesis, Department of Computer and Information Science,

University of Massachusetts at Amherst.

Gallagher, Kevin 1986 The Design and Implementation of CICERO.

Unpublished Master's thesis, Department of Computer and Infor- mation Science, University of Massachusetts at Amherst.

Lehnert, Wendy G. and Rosenberg, Seth 1985 The PLUM

User's

Manual. Technical Report 1, The COUNSELOR Project, Depart-

1 4 0 Computational Linguistics, Volume 12, Number 2, April-June 1986

The

FINITE STRING Newsletter

Site Reports

ment of Computer and Information Science, University of Massa- chusetts at Amherst.

McDonald, David D. 1986 Natural Language Generation: Complexi- ties and Techniques. To appear in Nirenburg, Ed.,

Theoretical and

Methodological Issues in Machine Translation.

Cambridge University

Press.

McDonald, David D. and Pustejovsky, James 1985a Description

Directed Natural Langauge Generation.

Proceedings of IJCAI-85:

799-805.

McDonald, David D. and Pustejovsky, James 1985b TAGs as a Gram- matical Formalism for Generation.

Proceedings of the 23rd Meeting of the Association for Computational Linguistics:

94-103.

McDonald, David D. and Pustejovsky, James 1985c SAMSON: A

Computational Theory of Prose Style for Natural Language Gener- ation.

Proceedings of the 1985 Meeting of the European Association for Computational Linguistics.

Pustejovsky, James 1986 An Integrated Theory Discourse Analysis.

Technical Report 11, The COUNSELOR Project, Department of

Computer and Information Science, University of Massachusetts at

Amherst.

Rissland Edwina L.; Valcarce, Edward; and Ashley, Kevin 1984

Explaining and Arguing with Examples.

Proceedings of AAAI-84.

Vaughan, Marie M, and McDonald, David D. 1986 A Model of

Revision in Natural Language Generation.

Proceedings of the 24th

Meeting of the Associaton for Computational Linguistics.

NEW YORK UNIVERSITY AND

SYSTEM DEVELOPMENT CORPORATION

PROTEUS

and PUNDIT: Research in Text Understanding

Department of Computer Science, New York University

System Development Corporation - A Burroughs Company

Prepared by Ralph Grishman (NYU) and

L y n e t t e Hirschman (SDC)

1 INTRODUCTION

We are engaged in the development of systems capable of analyzing short narrative messages dealing with a limited domain and extracting the information contained in the narrative. These systems are initially being applied to messages describing equipment failure. This work is a joint effort of N e w York University and the System

D e v e l o p m e n t Corporation for the Strategic Computing

Program. Our aim is to create a system reliable enough for use in an operational enviromnent. This is a formida- ble task, b o t h because the texts are unedited (and so contain various errors) and because the complexity of any real domain precludes us from assembling a

" c o m p l e t e " collection of the relationships and domain knowledge relevant to understanding texts in the domain.

A n u m b e r of laboratory prototypes have been devel- oped for the analysis of short narratives. N o n e of the systems we k n o w about, however, is reliable enough for use in an operational environment (the possible exceptions are expectation-driven systems, which simply ignore anything deviating f r o m these built-in expecta- tions). Typical success rates reported are that 7 5 - 8 0 % of sentences are correctly analyzed, and that m a n y errone- ous analyses pass the system undetected; this is not acceptable for most applications. We see the central task of the work to be described below as the construction of a substantially more reliable system for narrative analy- sis.

Our basic approach to increasing reliability will be to bring to bear on the analysis task as m a n y different types of constraints as possible. These include constraints related to syntax, semantics, domain knowledge, and discourse structure. In order to be able to capture the detailed knowledge about the domain that is needed for correct message analysis, we are initially limiting ourselves to messages about one particular piece of equipment (the "starting air c o m p r e s s o r " ) ; if we are successful in this narrow domain, we intend to apply the system to a b r o a d e r domain.

The risk with having a rich set of constraints is that m a n y of the sentences will violate one constraint or another. These violations m a y arise f r o m problems in the messages or in the knowledge base. O n the one hand, the messages frequently contain typographical or g r a m m a t - ical errors (in addition to the systematic use of fragments, which can be accounted for b y our grammar). O n the other hand, it is unlikely that we will be able to build a

" c o m p l e t e " model of domain knowledge; gaps in the knowledge base will lead to constraint violations for some sentences. To cope with these violations, we intend to develop a "forgiving" or flexible analyzer which will find a best analysis (one violating the fewest constraints) if no " p e r f e c t " analysis is possible. O n e aspect of this is the use of syntactic and semantic information on an equal footing in assembling an analysis, so that neither a syntactic nor a semantic error would, b y itself, block an analysis.

2 APPLICATION

This work is work is a c o m p o n e n t of the Fleet C o m m a n d

Center Battle M a n a g e m e n t P r o g r a m (FCCBMP), which is part of the Strategic Computing Program. The FCCBMP has two natural language components: one for interac- tive natural language access, the other for message proc- essing. The interactive c o m p o n e n t - which is to provide access to a data base and multiple expert systems - is being integrated b y Bolt Beranek and N e w m a n . The message processing c o m p o n e n t is being integrated as a

Computational Linguistics, Volume 12, Number 2, April-June 1986 141

The FINITE STRING Newsletter

Site Reports

joint effort of New York University and the System

Development Corporation.

Much of the information received by the Fleet

Command Center is in the form of messages. Some of these messages have a substantial natural language component. Consequently, natural language analysis is required if the information in these messages is to be recorded in a data base in a form usable by other programs. The specific class of messages we are studying are CASREPs, which are reports of equipment failures on board ships. These messages contain a brief narrative, typically 3 to 10 sentences in length, describing the symptoms, diagnosis, and possibly the attempts at repair of the failure. A typical narrative is shown in Figure 1.

The problems we face in analyzing these messages are similar to those in analyzing short messages and reports in other technical domains, and we therefore expect that the solutions we develop will be widely applicable.

3 PROJECT ORGANIZATION

This work is a joint research effort of New York Univer- sity and the System Development Corporation. NYU has principal responsibility for development of the domain knowledge base; SDC has principal responsibility for development of the flexible parser and for the domain-in- dependent discourse components. The division of the other tasks is noted in the detailed component descriptions below. We will also be integrating work on the knowledge base being done by SRI, which is a component technology developer for the FCCBMP natural language work.

The work by NYU is being done in LISP (primarily in

COMMON

LISP), as is most of the Strategic Computing research. SDC is doing its development in

PROLOG

because PROLOG provides a powerful framework for writing grammars; it also provides the inference engine necessary for knowledge structuring and reasoning about the discourse structures in text processing. This division will permit us to make some valuable comparisons between the LISP and PROLOG development environ- ments, and between the resulting systems.

The system being developed in LISP by NYU is called

PROTEUS (PROtotype TExt Understanding System;

Grishman et al., submitted for publication); the SDC system is called PUNDIT (Prolog UNDerstander of Inte- grated Text; Palmer et al. 1986). Notwithstanding the difference in implementation languages, we have tried to maintain a high level of compatibility between the two systems. We use essentially the same grammar and have agreed on common representations for the output of the syntactic analyzer (the regularized s~Tntactic structure) and the output of the semantic analyzer. This common- ality makes it possible to assign primary responsibility for the design of a component to one group, and then to take

A Sample CASREP

about a SAC (Starting Air Compressor)

142

DURING NORMAL START CYCLE OF 1A GAS TURBINE,

APPROX 90 SEC AFTER CLUTCH ENGAGEMENT, LOW

LUBE OIL AND FAIL TO ENGAGE ALARM WERE

RECEIVED ON THE ACC. (ALL CONDITIONS WERE

NORMAL INITIALLY). SAC WAS REMOVED AND

METAL CHUNKS FOUND IN OIL PAN. LUBE OIL PUMP

WAS REMOVED AND WAS FOUND TO BE SEIZED.

DRIVEN GEAR WAS SHEARED ON PUMP SHAFT.

Figure 1

Computational Linguistics, Volume 12, Number 2, April-June 1986

The FINITE STRING Newsletter Site Reports

the design developed for one system and port it to the other in a straightforward way.

We are currently developing baseline systems

that

incorporate substantial domain knowledge but use a traditional sequential processing organization. When these systems are complete, we will begin experimenting with flexible parsing algorithms. The systems currently being developed (Figure 2) process input in the following stages: lexical look-up, parsing, syntactic regularization, semantic analysis, integration with the domain knowledge representation, and discourse analysis. These compo- nents, and other tasks that are part of our research program, are described individually below.

PROTEUS/PUNDIT SYSTEM STRUCTURE

MESSAGETEXT

"~J WD LOOKUP I

LEXICON ]

CATEGORY/ATTRB. LISTS

GRAMMAR

(RESTRICTION

LANGUAGE)

PARSE TREES

SYNTACTIC

REGU LARIZATION

RULES

OPERATOR-OPERAND TREES

SEMANTIC AND I

ANAPHORIiANALYSIS I

ANTIC CASiMARKED TREES

DOMAIN INFORMATION:

• SEMAN. MAPPING RULES

• PROTOTYPE FRAMES

(for equipment structure and function, discourse stru ctu re)

NSTANT;A

O FRAMES

~ ' J l DISCOURSE ANALYSIS.

I

I

- CAUSALITY

-TIME

ANALYZED MESSAGE

Figure 2

Computational Linguistics, Volume 12, Number 2, April-June 1986 143

The FINITE STRING Newsletter Site Reports

4. SYSTEM COMPONENTS

4.1 LEXICON (SDC + NYU)

The lexicon consists of a modified version of the lexicon of the NYU Linguistic String Project, with words classi- fied as to part of speech and subcategorized for various grammatical properties (e.g., 'verbs and adjectives are subclassified for their complement types).

4.2. LEXICAL ACQUISITION (SDC)

The message vocabulary is large and will grow steadily as the system is modified to handle a wider range of equip- ment; several measures are planned to manage the growth of the lexicon. An interactive lexical entry program has been developed to facilitate adding words to the dictionary. Special constructions such as dates, times, and part numbers are processed using a small definite clause grammar defining special shapes. Future plans include addition of a component to use morphological analysis and selectional patterns to aid in classification of new lexical items.

4.3. SYNTAX ANALYSIS (NYU + SDC)

4.3.1. GRAMMAR

The syntactic component uses a grammar of BNF defi- nitions with associated restrictions that enforce context- sensitive constraints on the parse. This grammar is generally modelled after that developed by the NYU

Linguistic String Project (Sager 1981). The grammar has been expanded to cover the fragmentary constructions and complex noun phrases characteristic of the N a w message domain. A wide range of conjunction types is parsed by a set of conjunction rules which are automat- ically generated by metarules (Hirschman, in press). T o serve as an interface between the syntactic and semantic components, an additional set of rules produces a normalized intermediate representation of the syntax.

4.3.2.

TOP-DOWN PARSERS

Two top-down parsers have been implemented using the common grammar just described. In each case, the analyzer applies the BNF definitions and their associated constraints to produce explicit surface structure parses of the input; the analyzer also invokes the regularization rules

that

produce the normalized intermediate represen- tation.

In the NYU (LISP-based) system the basic algorithm is a chart parser, which provides goal-directed analysis along with the recording (for possible re-use) of all inter, mediate goals tried. The context sensitive constraints are expressed in a version of Restriction Language (Sager

1975) compiled into LISP. The SDC (PROLOG-based) system uses a top-down left-to-fight PROLOG implemen- tation of a version of the restriction grammar (Hirschman and Puder 1986).

4.4. FLEXIBLE ANALYZER (SDC)

A major research focus for SDC during the first two years will be to produce a flexible analyzer that integrates application of syntactic and semantic constraints. The flexible analyzer will focus more quickly on the correct analysis and will have recovery strategies to prevent syntactic analysis from becoming a bottleneck for subse- quent processing.

4.5. SEMANTIC ANALYSIS

The task of the semantic analyzer is to transform the regularized syntactic analysis into a semantic represen- tation. This representation provides unique identifiers for specific equipment components mentioned in the text. It consists of predicates describing states and events involv- ing the equipment, and higher-order predicates capturing the syntactically-expressed time and causal relations.

Roughly speaking, the clauses from the syntactic analysis map into states and events, while the noun phrases map into particular objects (there are several exceptions, including nominalizations, e.g., "loss of pressure", and adjectives of state, such as " b r o k e n valve"). Accordingly, the semantic analysis is divided into two major parts, clause semantics and noun phrase semantics. In addition to these two main parts, a time analysis component captures the time information which can be extracted from the input.

4.5.1. CLAUSE SEMANTICS (SDC)

Semantic analysis of clauses is performed by Inference

Driven Semantic Analysis (Palmer 1985), which analyzes verbs into their component meanings and fills their semantic roles, producing a semantic representation in predicate form. This representation includes information normally found in a case-frame representation, but is more detailed. The task of filling in the semantic roles is used to integrate the noun phrase analysis (described in the next section) with the clausal semantic analysis. In particular, the selection restriction information on the roles can be used to reject inappropriate referents for noun phrases.

The semantics also provide a filtering function, by checking selectional constraints on verbs and their argu- ments. The selectional constraints draw on domain knowledge for type and component information, as well as for information about possible relationships between objects in the domain. This function is currently used to accept or reject a completed parse. The goal for the flexi- ble analyzer is to apply selectional filtering composi- tionally to partial syntactic analyses to rule out semantically unacceptable phrases as soon as they are generated in the parse.

4.5.2. NOUN PHRASE SEMANTICS (SDC + NYU)

A noun phrase resolution component determines the reference of noun phrases, drawing on two sources: a detailed equipment model, and cumulative information regarding referents in previous sentences. SDC has

1 4 4 Computational Linguistics, Volume 12, Number 2, April-June 1986

The FINITE STRING Newsletter Call for Papers concentrated on the role of prior discourse, and has developed a procedure

t h a t

handles a wide variety of noun phrase types, including pronouns and missing noun phrases, using a focusing algorithm based on surface syntactic structure (Dahl, submitted for publication).

NYU, as part of its work on the domain model, has devel- oped a procedure

t h a t

can identify a c o m p o n e n t in the model f r o m any of the noun phrases

t h a t

can name that c o m p o n e n t (Ksiezyk and Grishman, submitted for publi- cation). After further development, these procedures will be integrated into a comprehensive noun phrase semantic analyzer.

4.5.3. TIME ANALYSIS (SDC)

SDC has started to develop a module to process time information. Sources of time information include verb tense, adverbial time expressions, prepositional phrases, co-ordinate and subordinate conjunctions. These are all m a p p e d into a small set of predicates expressing a partial time ordering among the states and events in the message.

4.6. DOMAIN MODEL (NYU)

The domain model captures the detailed information about the general class of equipment, and about the specific pieces of equipment involved in the messages; this information is needed in order to fully understand the messages. The model integrates p a r t / w h o l e informa- tion, t y p e / i n s t a n c e links, and functional information about the various components (Ksiezyk and Grishman, submitted for publication).

The knowledge base performs several functions:

• It provides the domain-specific constraints needed for the semantics to select the correct arguments for a predicate, so that modifiers are correctly attached to noun phrases.

• It enables noun phrase semantics to identify the correct referent for a phrase.

• It provides the prototype information structures which are instantiated in order to record the information in a particular message.

• It provides the information on equipment structure and function used b y the discourse rules in establishing probable causal links between the sentences. A n d finally, associated with the components in the know- ledge base are procedures for graphically displaying the status of the equipment as the message is interpreted.

These functions are performed b y a large network of frames implemented using the Symbolics Zetalisp flavors system.

4.7. DISCOURSE ANALYSIS (NYU)

The semantic analyzer generates separate semantic representations for the individual sentences of the message. F o r m a n y applications it is important to estab- lish the (normally implicit) intersentential relationships between the sentences. This is p e r f o r m e d by a set of inference rules

t h a t

(using the domain model) identify plausible causal and enabling relationships a m o n g the sentences. These relationships, once established, can serve to resolve some semantic ambiguities. T h e y can also supplement the time information extracted during semantic analysis and thus clarify temporal relations a m o n g the sentences.

4.8. DIAGNOSTICS (NYU)

The diagnostic procedures are intended to localize the cause of failure of the analysis and provide meaningful f e e d b a c k when some domain-specific constraint has b e e n violated. We are initially concentrating on violations of local (selectional) constraints, and have built a small c o m p o n e n t for diagnosing such violations and suggesting acceptable sentence forms; later work will study more global discourse constraints.

REFERENCES

Dahl, Deborah A. (submitted for publication) Focusing and Reference

Resolution in PUNDIT.

Grishman, Ralph; Ksiezyk, Tomasz, and Nhan, Ngo Thanh (submitted for publication) Model-based Analysis of Messages about Equip- ment.

Hirschman, Lynette and Puder, Karl 1986 Restriction Grammar: A

PROLOG Implementation. In Warren, D.H.D. and Van Caneghem,

M., Eds., Logic Programming and its Applications. Ablex Publishing

Company, Norwood, New Jersey: 244-261.

Hirschman, Lynette (in press) "Conjunction in Meta-Restriction

Grammar."

Journal of Logic Programming.

Ksiezyk, Tomasz and Grishman, Ralph (submitted for publication) An

Equipment Model and its Role in the Interpretation of Nominal

Compounds.

Palmer, Martha S. 1985 Driving Semantics for a Limited Domain.

Ph.D. thesis. University of Edinburgh.

Palmer, Martha; Dahl, Deborah; Schiffman, Rebecca; Hirschman,

Lynette; Linebarger, Marcia; and Dowding, John 1986 Recovering

Implicit Information. To appear in

Proceedings of the 24th Annual

Meeting of the Association for Computational Linguistics.

Sager, Naomi and Grishman, Ralph 1975 The Restriction Language for Computer Grammars of Natural Language.

Communications of

the ACM 18: 390-400.

Sager, Naomi 1981

Natural Language Information Processing: A

Computer Grammar of English and its Applications. Addison-Wesley,

Reading, Massachusetts.

C A L L FOR PAPERS

ESCOL 86

10-12 October 1986, University of Pittsburgh and

Carnegie-Mellon University

The 1986 Eastern States Conference on Linguistics will include demonstrations of natural language processing software. The invited speakers are Charles Fillmore and

Lily W a n g Fillmore f r o m the University of California at

Berkeley, Martin K a y f r o m the X e r o x Palo Alto

Research Center, and George Miller f r o m Princeton

University.

Original, unpublished papers on any topic of general linguistic interest are invited for the general sessions. F o r the special session, Linguistics at Work, we invite papers on applied linguistics, especially in the areas of language teaching and computational linguistics.

Computational Linguistics, Volume 12, Number 2, April-June 1986 145

Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement