Message-Id: <199612180344.VAA25166@library.wustl.edu> Date: Tue, 17 Dec 1996 22:25:53 -0500 From: Shih-Fu Chang <mailto:sfchang@CTR.COLUMBIA.EDU> Subject: Re: FINDING IMAGES IN A DATABASE To: mailto:IMAGELIB@LISTSERV.ARIZONA.EDU
Hi,We have developed several automatic content-based image search engines. Although they did not solve the difficult problem of automatic 3D shape indexing and matching (as Brian mentioned), you might find them to be of interest and use.
Among them, SaFe focuses on joint spatial/feature based image query, while WebSEEk focuses on using both image and text features for WWW image cataloging and search. Both have several online image/video collections for testing (from hundreds to 650,000 images). WebSEEk also utilizes a working image taxonomy to guide search. You are all welcome to take a look (URL enclosed below). Comments will be appreciated!
Best Regards,
----------------------------------------------------------- Shih-Fu Chang, Asst. Professor Dept. of Electrical Engineering & Center for Telecommunications Research Columbia University New York, NY 10027
TEL: (O) 212-854-6894 FAX: 212-316-9068, 212- 932-9421, (Network) (212) 854-2497 email: mailto:sfchang@ctr.columbia.edu www: http://www.ctr.columbia.edu/~sfchang -----------------------------------------------------------
------------ project 1 ------------------------------ SaFe - A Fully Automatic Joint Spatial/Feature Based Image Search System
Demo: http://disney.ctr.columbia.edu/SaFe
SaFe is a general tool for spatial and feature image search. It provides a framework for searching for and comparing images by the spatial arrangement of regions or objects.
In a SaFe query, objects or regions are assigned by the user. These are given properties of spatial location, size and visual features, such as color. The SaFe system finds the images that best match the query. SaFe uses fully automatic tools for region/feature extraction and indexing.
SaFe also resolves spatial relationships, which allows the user to position objects relative to each other in a query.
Example queries include "find images including a blue region on top and a wide green open region in the bottom (looking for images with blue sky and open grass field)", "use this spatial pattern of red, white, blue colors to find images containing American Flags."
The current SaFe Web demo includes several natural/synthetic test image collections.
Reference:
J. R. Smith and S.-F. Chang, "VisualSEEk: A Fully Automated Content-Based Image Query System," ACM Multimedia Conference, Boston, MA, Nov. 1996. (ftp://ftp.ctr.columbia.edu/CTR-Research/advent/public/papers/96/smith96f.ps)
-------------------- project 2 ------------------------
Project Name:
WebSEEk- Automatic Image/Video Searching and Cataloging on the World Wide Web
Columbia University, Department of Electrical Engineering, Image and Advanced TV Lab, New York, USA
Project Summary:
Webseek is a content-based image and video catalog and search tool for the World Wide Web. The system collects the images and videos using several autonomous Web agents which automatically analyze, index, and assign the images and videos to subject classes. The system is novel in that it utilizes text and visual information synergistically to provide for cataloging and searching for the images and videos. The complete system possesses several powerful functionalities, namely, searching using content-based techniques, query modification using content-based relevance feedback, automated collection of visual information, compact presentation of images and videos for displaying query results, image and video subject search and navigation, text-based searching, and search results lists manipulations such as intersection, subtraction and concatenation. At present, the system has catalogued over 650,000 images and 10,000 videos from the Web.
New algorithms are being developed for automatic mapping of new unconstrained images/video to semantic-level subject classes in the image taxonomy. A working image taxonomy has been constructed in a semi-automatic way in the current prototype of WebSEEk. The mapping algorithms explore visual features (such as color, texture, spatial layout, video object features), text features (such as associated html documents, transcript, caption), and intelligent clustering techniques in the feature space.
Demonstration or Prototype Access:
http://www.ctr.columbia.edu/webseek
Principal Investigators:
Prof. Shih-Fu Chang (mailto:sfchang@ctr.columbia.edu, http://www.ctr.columbia.edu/~sfchang)
and
John Smith (mailto:jrsmith@ctr.columbia.edu, http://www.ctr.columbia.edu/~jrsmith)
References
J. R. Smith and S.-F. Chang, "Searching for Images and Videos on the World-Wide Web," Columbia University/CTR Technical Report #459-96-25. also IS&T/SPIE Symposium on Electronic Imaging: Science and Technology - Storage & Retrieval for Image and Video Databases V, San Jose, CA, February 1997. (ftp://ftp.ctr.columbia.edu/CTR-Research/advent/public/papers/96/smith96g.ps)
S.-F. Chang, J. Smith, and H. Meng, "Efficient Techniques for Feature-Based Image/Video Access and Manipulation," Clinic on Library Applications of Data Processing: Digital Image Access and Retrieval, University of Illinois at Urbana-Champaign, March 1996. (http://www.ctr.columbia.edu/~sfchang/dpc96-2/title.html)