| Abstract | Objective: Despite the proliferation of electronic information retrieval (IR) systems for physicians, their
effectiveness has not been well assessed. The purpose of this review is to provide a conceptual framework
and to apply the results of previous studies to this framework.
Data Sources: All sources of medical informatics and information science literature, including MEDLINE,
along with bibliographies of textbooks in these areas, were searched from 1966 to January 1998.
Study Selection.All articles presenting either classifications of evaluation studies or their results, with an
emphasis on those studying use by physicians.
Data Extraction: A framework for evaluation was developed, consisting of frequency of use, purpose of use,
user satisfaction, searching utility, search failure, and outcomes. All studies were then assessed based on the
framework.
Data Synthesis: Due to the heterogeneity and simplistic study designs, no meta-analysis of studies could be
done. General conclusions were drawn from data where appropriate. A total of 47 articles were found to
include an evaluation component and were used to develop the framework. Of these, 21 articles met the
inclusion criteria for 1 or more of the categories in the framework. Most use of IR systems by physicians
still occurs with bibliographic rather than full-text databases. Overall use of IR systems occurs just 0.3 to 9
times per physician per month, whereas physicians have 2 unanswered questions for every 3 patients.
Conclusions: Studies comparing IR systems with different searching features have not shown that advanced
searching methods are significantly more effective than simple text word methods. Most searches retrieve
only one fourth to one half of the relevant articles on a given topic and, once retrieved, little is known about
how these articles are interpreted or applied. These studies imply that further research and development are
needed to improve system utility and performance. |