Efficient query subscription processing for prospective search engines

Utku Irmak; Svilen Mihaylov; Torsten Suel; Samrat Ganguly; Rauf Izmailov

Efficient query subscription processing for prospective search engines

Utku Irmak, Svilen Mihaylov, Torsten Suel, Samrat Ganguly, Rauf Izmailov

Research output: Contribution to conference › Paper › peer-review

Abstract

Current web search engines are retrospective in that they limit users to searches against already existing pages. Prospective search engines, on the other hand, allow users to upload queries that will be applied to newly discovered pages in the future. Some examples of prospective search are the subscription features in Google News and in RSS-based blog search engines. In this paper, we study the problem of efficiently processing large numbers of keyword query subscriptions against a stream of newly discovered documents, and propose several query processing optimizations for prospective search. Our experimental evaluation shows that these techniques can improve the throughput of a well known algorithm by more than a factor of 20, and allow matching hundreds or thousands of incoming documents per second against millions of subscription queries per node.

Original language	English (US)
Pages	375-380
Number of pages	6
State	Published - 2006
Event	2006 USENIX Annual Technical Conference - Boston, United States Duration: May 30 2006 → Jun 3 2006

Conference

Conference	2006 USENIX Annual Technical Conference
Country	United States
City	Boston
Period	5/30/06 → 6/3/06

ASJC Scopus subject areas

Computer Science(all)

Access to Document

Fingerprint Dive into the research topics of 'Efficient query subscription processing for prospective search engines'. Together they form a unique fingerprint.

Cite this

@conference{64d7b016fbc848cf9ba9dc856dd112f5,

title = "Efficient query subscription processing for prospective search engines",

abstract = "Current web search engines are retrospective in that they limit users to searches against already existing pages. Prospective search engines, on the other hand, allow users to upload queries that will be applied to newly discovered pages in the future. Some examples of prospective search are the subscription features in Google News and in RSS-based blog search engines. In this paper, we study the problem of efficiently processing large numbers of keyword query subscriptions against a stream of newly discovered documents, and propose several query processing optimizations for prospective search. Our experimental evaluation shows that these techniques can improve the throughput of a well known algorithm by more than a factor of 20, and allow matching hundreds or thousands of incoming documents per second against millions of subscription queries per node.",

author = "Utku Irmak and Svilen Mihaylov and Torsten Suel and Samrat Ganguly and Rauf Izmailov",

year = "2006",

language = "English (US)",

pages = "375--380",

note = "2006 USENIX Annual Technical Conference ; Conference date: 30-05-2006 Through 03-06-2006",

}

TY - CONF

T1 - Efficient query subscription processing for prospective search engines

AU - Irmak, Utku

AU - Mihaylov, Svilen

AU - Suel, Torsten

AU - Ganguly, Samrat

AU - Izmailov, Rauf

PY - 2006

Y1 - 2006

N2 - Current web search engines are retrospective in that they limit users to searches against already existing pages. Prospective search engines, on the other hand, allow users to upload queries that will be applied to newly discovered pages in the future. Some examples of prospective search are the subscription features in Google News and in RSS-based blog search engines. In this paper, we study the problem of efficiently processing large numbers of keyword query subscriptions against a stream of newly discovered documents, and propose several query processing optimizations for prospective search. Our experimental evaluation shows that these techniques can improve the throughput of a well known algorithm by more than a factor of 20, and allow matching hundreds or thousands of incoming documents per second against millions of subscription queries per node.

AB - Current web search engines are retrospective in that they limit users to searches against already existing pages. Prospective search engines, on the other hand, allow users to upload queries that will be applied to newly discovered pages in the future. Some examples of prospective search are the subscription features in Google News and in RSS-based blog search engines. In this paper, we study the problem of efficiently processing large numbers of keyword query subscriptions against a stream of newly discovered documents, and propose several query processing optimizations for prospective search. Our experimental evaluation shows that these techniques can improve the throughput of a well known algorithm by more than a factor of 20, and allow matching hundreds or thousands of incoming documents per second against millions of subscription queries per node.

UR - http://www.scopus.com/inward/record.url?scp=80052126667&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80052126667&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:80052126667

SP - 375

EP - 380

T2 - 2006 USENIX Annual Technical Conference

Y2 - 30 May 2006 through 3 June 2006

ER -