Scanning the Technology on the Applications of Multimedia Processing to Communications

The challenge of multimedia processing is to provide services that seamlessly integrate text, sound, image, and video information and to do it in a way that preserves the ease of use and interactivity of conventional plain old telephone service (POTS) telephony, irrelevant of the bandwidth or means of access of the connection to the service. To achieve this goal, there are a number of technological problems that must be considered, including: • compression and coding of multimedia signals, including algorithmic issues, standards issues, and transmission issues; • synthesis and recognition of multimedia signals, including speech, images, handwriting, and text; • organization, storage, and retrieval of multimedia signals, including the appropriate method and speed of delivery (e.g., streaming versus full downloading), resolution (including layering or embedded versions of the signal), and quality of service, i.e., perceived quality of the resulting signal; • access methods to the multimedia signal (i.e., matching the user to the machine), including spoken natural language interfaces, agent interfaces, and media conversion tools; • searching (i.e., based on machine intelligence) by text, speech, and image queries; • browsing (i.e., based on human intelligence) by accessing the text, by voice, or by indexed images. In each of these areas, a great deal of progress has been made in the past few years, driven in part by the relentless growth in multimedia personal computers and in part by the promise of broad-band access from the home and from wireless connections. Standards have also played a key role in driving new multimedia services, both on the POTS network and on the Internet. It is the purpose of this paper to review the status of the technology in each of the areas listed above and to illustrate current capabilities by describing several multimedia applications that have been implemented at AT&T Labs over the past several years.