Ir Evaluation

888 Words4 Pages
Blair, D.C & Maron, M.E explains the meaning of evaluation as performance or value of a system, process, product, or policy. In any area such as (science, technology and social applications) evaluation is done in evaluation criteria, measures, methods, and related aspects. In IR research the evaluation is a major force in the progress of IR research and development (R&D). In his paper the analysis of basic and historical aspects related to evaluation of IR systems and processes assess the accomplishments and shortcomings of IR evaluation, raise critical issues and questions, and compare in a limited way IR evaluation with evaluation of related information systems, most notably expert systems and OPACS were used. The context of IR was the major…show more content…
W. States that evaluation of IR systems came shortly after the appearance of first design proposals and fist prototypes. Kent et al. (1955) were the first to propose the criterion of relevance and the measures [3] of precision and relevance (later renamed recall) for evaluation of IR systems; these became the staple of most IR evaluations to this day. Shortly after that, various US Government agencies began sponsoring IR evaluation efforts. The most famous of the pioneering IR evaluations were the Cranfield studies that begun in the late 1950s and ran till mid 1960’s. Cranfield is the grand-daddy of IR evaluations, setting the role for evaluation in IR, and the tone and approach that is used in most evaluation studies to this day. Many approaches could be used in evaluation. The choice of the approach depends on the intent of evaluation and in many ways it defines the type of results obtained. In addition, every approach has its limitations. It is hard to mix…show more content…
Different levels 2. Requirements 3. System and processes in IR evaluation 4. Criteria in IR evaluation 5. Measures in IR evaluation Cooper, W. S. Explains the relevant document [4] and precision measures how well the system retrieves only the relevant document. Relevance is a something quite straight – forward. In many respects they were successful. Evaluation is an integral and vital part of IR. This paper is an attempt to evaluate a broad range of evaluations in IR, and to identify major strengths and shortcomings of evaluations, to raise questions that ought to be considered in evaluation, and provide some comparisons with evaluation of related information systems. However, many issues and problems in evaluation are reappearing in historical cycles. Great many significant evaluative problems need to be addressed. Blair,D.C. & maron states the retrieval effectiveness which was most widely used as measures of document retrieval effectiveness as recall and precision [5]. Recall is a proportion of relevant documents which the systems

More about Ir Evaluation

Open Document