Search Engine Coverage of the OAI-PMH Corpus

Authors

Frank McCown Ph.D., Harding UniversityFollow
Xiaoming Liu
Michael L. Nelson, Old Dominion University
Mohammad Zubair, Old Dominion University

Document Type

Article

Publication Title

IEEE Internet Computing

Publication Date

3-2006

Volume

10

Issue

2

First Page

66

Last Page

73

Abstract

Having indexed much of the "surface" Web, search engines are now using various approaches to index the "deep" Web. At the same time, institutional repositories and digital libraries are adopting the open archives initiative protocol for metadata harvesting (OAI-PMH) to expose their holdings. The authors harvested nearly 10 million records from OAI-PMH repositories. From these records, they extracted 3.3 million unique resource URLs and then conducted searches on samples from this collection to determine how much of the OAI-PMH corpus the three major search engines have indexed.

Copyright held by

IEEE Computer Society

Recommended Citation

McCown, F., Liu, X., Nelson, M. L., & Zubair, M. (2006). Search Engine Coverage of the OAI-PMH Corpus. IEEE Internet Computing, 10 (2), 66-73. http://dx.doi.org/10.1109/MIC.2006.41