General suffix automaton construction algorithm and space bounds

Mehryar Mohri, Pedro Moreno, Eugene Weinstein

Research output: Contribution to journalArticle

Abstract

Suffix automata and factor automata are efficient data structures for representing the full index of a set of strings. They are minimal deterministic automata representing the set of all suffixes or substrings of a set of strings. This paper presents a novel analysis of the size of the suffix automaton or factor automaton of a set of strings. It shows that the suffix automaton or factor automaton of a set of strings U has at most 2 Q - 2 states, where Q is the number of nodes of a prefix-tree representing the strings in U. This bound significantly improves over 2 {norm of matrix} U {norm of matrix} - 1, the bound given by Blumer et al. [A. Blumer, J. Blumer, D. Haussler, R.M. McConnell, A. Ehrenfeucht, Complete inverted files for efficient text retrieval and analysis, Journal of the ACM 34 (1987) 578-589], where {norm of matrix} U {norm of matrix} is the sum of the lengths of all strings in U. More generally, we give novel and general bounds for the size of the suffix or factor automaton of an automaton as a function of the size of the original automaton and the maximal length of a suffix shared by the strings it accepts. We also describe in detail a linear-time algorithm for constructing the suffix automaton S or factor automaton F of U in time O (| S |). Our algorithm applies in fact to any input suffix-unique automaton and strictly generalizes the standard on-line construction of a suffix automaton for a single input string. Our algorithm can also be used straightforwardly to generate the suffix oracle or factor oracle of a set of strings, which has been shown to have various useful properties in string-matching. Our analysis suggests that the use of factor automata of automata can be practical for large-scale applications, a fact that is further supported by the results of our experiments applying factor automata to a music identification task with more than 15,000 songs.

Original languageEnglish (US)
Pages (from-to)3553-3562
Number of pages10
JournalTheoretical Computer Science
Volume410
Issue number37
DOIs
StatePublished - Sep 1 2009

Keywords

  • Factor automata
  • Finite automata
  • Indexing
  • Inverted text
  • Music identification
  • Pattern-matching
  • String-matching
  • Suffix automata
  • Suffix trees

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'General suffix automaton construction algorithm and space bounds'. Together they form a unique fingerprint.

  • Cite this