Approximate string joins in a database (almost) for free

Luis Gravano, Panagiotis G. Ipeirotis, H. V. Jagadish, Nick Koudas, S. Muthukrishnan, Divesh Srivastava

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    String data is ubiquitous, and its management has taken on particular importance in the past few years. Approximate queries are very important on string data especially for more complex queries involving joins. This is due, for example, to the prevalence of typographical errors in data, and multiple conventions for recording attributes such as name and address. Commercial databases do not support approximate string joins directly, and it is a challenge to implement this functionality efficiently with user-defined functions (UDFs). In this paper, we develop a technique for building approximate string join capabilities on top of commercial databases by exploiting facilities already available in them. At the core, our technique relies on matching short substrings of length q, called q-grams, and taking into account both positions of individual matches and the total number of such matches. Our approach applies to both approximate full string matching and approximate substring matching, with a variety of possible edit distance functions. The approximate string match predicate, with a suitable edit distance threshold, can be mapped into a vanilla relational expression and optimized by conventional relational optimizers. We demonstrate experimentally the benefits of our technique over the direct use of UDFs, using commercial database systems and real data. To study the I/O and CPU behavior of approximate string join algorithms with variations in edit distance and q-gram length, we also describe detailed experiments based on a prototype implementation.

    Original languageEnglish (US)
    Title of host publicationVLDB 2001 - Proceedings of 27th International Conference on Very Large Data Bases
    EditorsPeter M. G. Apers, Paolo Atzeni, Richard T. Snodgrass, Stefano Ceri, Kotagiri Ramamohanarao, Stefano Paraboschi
    PublisherMorgan Kaufmann
    Pages491-500
    Number of pages10
    ISBN (Electronic)1558608044, 9781558608047
    StatePublished - 2001
    Event27th International Conference on Very Large Data Bases, VLDB 2001 - Roma, Italy
    Duration: Sep 11 2001Sep 14 2001

    Publication series

    NameVLDB 2001 - Proceedings of 27th International Conference on Very Large Data Bases

    Other

    Other27th International Conference on Very Large Data Bases, VLDB 2001
    Country/TerritoryItaly
    CityRoma
    Period9/11/019/14/01

    ASJC Scopus subject areas

    • Information Systems and Management
    • Computer Science Applications
    • Hardware and Architecture
    • Software
    • Computer Networks and Communications
    • Information Systems

    Fingerprint

    Dive into the research topics of 'Approximate string joins in a database (almost) for free'. Together they form a unique fingerprint.

    Cite this