CSxxx Fundamentals of Information Retrieval
Lecture 0
Course Introduction
Krishnendu Ghosh
Department of Computer Science & Engineering
Indian Institute of Information Technology Dharwad
Course Details
Credit Structure: 1.5-0-0-2-2
Course Webpage
https://sites.google.com/view/krishnendughosh/teaching/ir
Course Details: Course Objectives
• Learn the theories and techniques behind Web search engines.
• Get hands on project experience by developing real-world applications, such as intelligent tools for improving search accuracy from user feedback, email spam detection, or scientific literature organization and mining.
• Learn tools and techniques to do cutting-edge research in the area of information retrieval or text mining.
• Open the door to the amazing job opportunities in Search Technology and E-commerce companies such as Google, Microsoft, Yahoo! and Amazon.
Course Details: Learning Outcomes
LO1: Learn to write code for text indexing and retrieval
LO2: Learn to evaluate information retrieval systems
LO3: Learn to analyze textual and semi-structured data sets
LO4: Learn about text similarity measure
LO5: Understanding about search engine
Course Syllabus
Unit 1: Foundations of Information Retrieval
Introduction to Information Retrieval, Information Retrieval vs Database Systems, Applications of IR, Components of an IR System, Document Collections and Corpora, Text Processing Pipeline, Tokenization, Stopword Removal, Stemming, Lemmatization, Inverted Index, Query Processing, Web Search Basics.
Course Syllabus
Unit 2: Data Compression and Indexing
Need for Compression in IR, Dictionary Compression, Blocked Storage, Front Coding, Posting List Compression, Variable Byte Encoding, Gamma Coding, Delta Coding, Skip Pointers, Efficient Index Construction, Dynamic Indexing, Distributed Indexing, Index Maintenance.
Course Syllabus
Unit 3: Implementation and Evaluation
IR System Architecture, Web Crawling, Document Acquisition, Building an Inverted Index, Query Execution, Ranked Retrieval, Relevance Feedback, Evaluation Methodology, Precision, Recall, F1-Score, Mean Average Precision (MAP), Mean Reciprocal Rank (MRR), NDCG, TREC Collections and Benchmarks, Lucene, Elasticsearch, Whoosh, PyTerrier.
Course Syllabus
Unit 4: Retrieval Models
Boolean Retrieval Model, Boolean Queries, Vector Space Model (VSM), Term Frequency (TF), Inverse Document Frequency (IDF), TF-IDF Weighting, Cosine Similarity, Probabilistic Retrieval Model, Binary Independence Model, BM25, Language Models for IR, Query Likelihood Model, Jelinek-Mercer Smoothing, Dirichlet Smoothing, Latent Semantic Analysis (LSA), Topic Models.
Textbooks
Introduction to Information Retrieval, by C. Manning, P. Raghavan, and H. Schütze (Cambridge University Press, 2008).
Search Engines: Information Retrieval in Practice. Croft, W. Bruce; Metzler, Donald; Strohman, Trevor. Addison Wesley (2008)
Information Retrieval: Implementing and Evaluating Search Engines, Stefan Buettcher, Charles L. A. Clarke, Gordon V. Cormack. MIT Press. (2010)
Modern Information Retrieval, Ricardo Baeza-Yates and Berthier Ribeiro-Neto, Addison-Wesley, (1999)
Course Evaluation Plan
Quiz 24%
Assignments 20%
Project 40% (Implementation 10%, Knowledge 10%, Analysis 20%)
Attendance 16%
Attendance policy:
85%, as per institute norms
Today’s Attendance
Thank You