TREC (Text REtrieval Conference)

The TREC experiments may be seen as a representation of the mainline approach to information retrieval (IR). It is based on experiments with huge text collections and is based on the test collections approach and the measurement of recall and precision first used in the Cranfield experiments.

Robertson; Walker & Beaulieu (1997): "Although some interesting interactive experiments have been performed at TREC, the problems of reconciling the requirements of the laboratory context with the concerns of interactive retrieval are still largely unresolved.

Blair (2002) warns that some recall measures in these experiments may be overrated.

Literature:

Blair, D. C. (2002). Some thoughts on the reported results of TREC. Information Processing & Management, 38(3), 445-451.

Harman, D. K. & Voorhees, E. M. (2006). TREC: An overview. Annual review of information science and technology , 40, 113-155.

Robertson, S.E.; Walker, S. & Beaulieu, M. (1997). Laboratory experiments with Okapi: Participation in the TREC programme. Journal of Documentation, 53(1), 20-34.

Sparck Jones, K. (1995). Reflections on TREC. Information Processing & Management, 31(3), 291-314.

Sparck Jones, K. (2000). Further reflections on TREC. Information Processing & Management, 36(1), 37-85.

TREC-1,  1992. http://trec.nist.gov/pubs/trec1/t1_proceedings.html

TREC-2,  1993. http://trec.nist.gov/pubs/trec2/t2_proceedings.html

TREC-3,  1994. http://trec.nist.gov/pubs/trec3/t3_proceedings.html

TREC-4,  1995. http://trec.nist.gov/pubs/trec4/t4_proceedings.html

TREC-5,  1996. http://trec.nist.gov/pubs/trec5/t5_proceedings.html

TREC-6,  1997. http://trec.nist.gov/pubs/trec6/t6_proceedings.html

TREC-7,  1998. http://trec.nist.gov/pubs/trec7/t7_proceedings.html

TREC-8,  1999. http://trec.nist.gov/pubs/trec8/t8_proceedings.html

TREC-9,  2002  http://trec.nist.gov/pubs/trec9/t9_proceedings.html

TREC-10, 2001. http://trec.nist.gov/pubs/trec10/t10_proceedings.html

TREC-11, 2002. http://trec.nist.gov/pubs/trec11/t11_proceedings.html

TREC-12, 2003. http://trec.nist.gov/pubs/trec12/t12_proceedings.html

TREC-13, 2004. http://trec.nist.gov/pubs/trec13/t13_proceedings.html

Voorhees, E. M. (2005). TREC: Improving Information Access through Evaluation. Bulletin of the American Society for Information Science and Technology, 32(1). http://www.asis.org/Bulletin/Oct-05/voorhees.html

Voorhees, E. M. & Harman, D. K. (2000). Overview of the Sixth Text REtrieval Conference (TREC-6). Information Processing & Management, 36(1), 3-35.

Voorhees, E. M. & Harman, D. K. (Eds.). (2005). TREC: Experiment and Evaluation in Information Retrieval. The MIT Press. Table of contents and samples:  http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=10667&mode=toc

----

TREC Homepage: http://trec.nist.gov/

See also: Cranfield experiments; Information retrieval; Test Collections

Birger Hjørland

Last edited: 19-05-2006

Home