Utility evaluation of cross-document information extraction

Heng Ji; Zheng Chen; Jonathan Feldman; Antonio Gonzalez; Ralph Grishman; Vivek Upadhyay

Utility evaluation of cross-document information extraction

Heng Ji, Zheng Chen, Jonathan Feldman, Antonio Gonzalez, Ralph Grishman, Vivek Upadhyay

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We describe a utility evaluation to determine whether cross-document information extraction (IE) techniques measurably improve user performance in news summary writing. Two groups of subjects were asked to perform the same time-restricted summary writing tasks, reading news under different conditions: with no IE results at all, with traditional singledocument IE results, and with cross-document IE results. Our results show that, in comparison to using source documents only, the quality of summary reports assembled using IE results, especially from cross-document IE, was significantly better and user satisfaction was higher. We also compare the impact of different user groups on the results.

Original language	English (US)
Title of host publication	NAACL HLT 2010 - Human Language Technologies
Subtitle of host publication	The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference
Pages	285-288
Number of pages	4
State	Published - 2010
Event	2010 Human Language Technologies Conference ofthe North American Chapter of the Association for Computational Linguistics, NAACL HLT 2010 - Los Angeles, CA, United States Duration: Jun 2 2010 → Jun 4 2010

Publication series

Name	NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference

Other

Other	2010 Human Language Technologies Conference ofthe North American Chapter of the Association for Computational Linguistics, NAACL HLT 2010
Country/Territory	United States
City	Los Angeles, CA
Period	6/2/10 → 6/4/10

ASJC Scopus subject areas

Language and Linguistics
Linguistics and Language

Cite this

Ji, H., Chen, Z., Feldman, J., Gonzalez, A., Grishman, R., & Upadhyay, V. (2010). Utility evaluation of cross-document information extraction. In NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference (pp. 285-288). (NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference).

Utility evaluation of cross-document information extraction. / Ji, Heng; Chen, Zheng; Feldman, Jonathan et al.
NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference. 2010. p. 285-288 (NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ji, H, Chen, Z, Feldman, J, Gonzalez, A, Grishman, R & Upadhyay, V 2010, Utility evaluation of cross-document information extraction. in NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference. NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference, pp. 285-288, 2010 Human Language Technologies Conference ofthe North American Chapter of the Association for Computational Linguistics, NAACL HLT 2010, Los Angeles, CA, United States, 6/2/10.

Ji H, Chen Z, Feldman J, Gonzalez A, Grishman R, Upadhyay V. Utility evaluation of cross-document information extraction. In NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference. 2010. p. 285-288. (NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference).

Ji, Heng ; Chen, Zheng ; Feldman, Jonathan et al. / Utility evaluation of cross-document information extraction. NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference. 2010. pp. 285-288 (NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference).

@inproceedings{fcdff15a1c3b4b3cb15d3c276783a5a1,

title = "Utility evaluation of cross-document information extraction",

abstract = "We describe a utility evaluation to determine whether cross-document information extraction (IE) techniques measurably improve user performance in news summary writing. Two groups of subjects were asked to perform the same time-restricted summary writing tasks, reading news under different conditions: with no IE results at all, with traditional singledocument IE results, and with cross-document IE results. Our results show that, in comparison to using source documents only, the quality of summary reports assembled using IE results, especially from cross-document IE, was significantly better and user satisfaction was higher. We also compare the impact of different user groups on the results.",

author = "Heng Ji and Zheng Chen and Jonathan Feldman and Antonio Gonzalez and Ralph Grishman and Vivek Upadhyay",

year = "2010",

language = "English (US)",

isbn = "1932432655",

series = "NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference",

pages = "285--288",

booktitle = "NAACL HLT 2010 - Human Language Technologies",

note = "2010 Human Language Technologies Conference ofthe North American Chapter of the Association for Computational Linguistics, NAACL HLT 2010 ; Conference date: 02-06-2010 Through 04-06-2010",

}

TY - GEN

T1 - Utility evaluation of cross-document information extraction

AU - Ji, Heng

AU - Chen, Zheng

AU - Feldman, Jonathan

AU - Gonzalez, Antonio

AU - Grishman, Ralph

AU - Upadhyay, Vivek

PY - 2010

Y1 - 2010

N2 - We describe a utility evaluation to determine whether cross-document information extraction (IE) techniques measurably improve user performance in news summary writing. Two groups of subjects were asked to perform the same time-restricted summary writing tasks, reading news under different conditions: with no IE results at all, with traditional singledocument IE results, and with cross-document IE results. Our results show that, in comparison to using source documents only, the quality of summary reports assembled using IE results, especially from cross-document IE, was significantly better and user satisfaction was higher. We also compare the impact of different user groups on the results.

AB - We describe a utility evaluation to determine whether cross-document information extraction (IE) techniques measurably improve user performance in news summary writing. Two groups of subjects were asked to perform the same time-restricted summary writing tasks, reading news under different conditions: with no IE results at all, with traditional singledocument IE results, and with cross-document IE results. Our results show that, in comparison to using source documents only, the quality of summary reports assembled using IE results, especially from cross-document IE, was significantly better and user satisfaction was higher. We also compare the impact of different user groups on the results.

UR - http://www.scopus.com/inward/record.url?scp=84863337594&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863337594&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84863337594

SN - 1932432655

SN - 9781932432657

T3 - NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference

SP - 285

EP - 288

BT - NAACL HLT 2010 - Human Language Technologies

T2 - 2010 Human Language Technologies Conference ofthe North American Chapter of the Association for Computational Linguistics, NAACL HLT 2010

Y2 - 2 June 2010 through 4 June 2010

ER -

Utility evaluation of cross-document information extraction

Abstract

Publication series

Other

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this