Introduction -History of IR- Components of IR - Issues –Open source Search engine Frameworks - The impact of the web on IR - The role of artificial intelligence (AI) in IR – IR Versus Web Search - Components of a Search engine- Characterizing the web.
UNIT II INFORMATION RETRIEVAL
Boolean and vector-space retrieval models- Term weighting - TF-IDF weighting- cosine similarity – Preprocessing - Inverted indices - efficient processing with sparse vectors – Language Model based IR - Probabilistic IR –Latent Semantic Indexing - Relevance feedback and query expansion.
UNIT III WEB SEARCH ENGINE – INTRODUCTION AND CRAWLING
Web search overview, web structure, the user, paid placement, search engine optimization/ spam. Web size measurement - search engine optimization/spam – Web Search Architectures - crawling - meta-crawlers- Focused Crawling - web indexes –- Near-duplicate detection - Index Compression - XML retrieval.
UNIT IV WEB SEARCH – LINK ANALYSIS AND SPECIALIZED SEARCH
Link Analysis –hubs and authorities – Page Rank and HITS algorithms -Searching and Ranking – Relevance Scoring and ranking for Web – Similarity - Hadoop & Map Reduce - Evaluation - Personalized search - Collaborative filtering and content-based recommendation of documents and products – handling “invisible” Web - Snippet generation, Summarization, Question Answering, Cross- Lingual Retrieval.
UNIT V DOCUMENT TEXT MINING
Information filtering; organization and relevance feedback – Text Mining -Text classification and clustering - Categorization algorithms: naive Bayes; decision trees; and nearest neighbor - Clustering algorithms: agglomerative clustering; k-means; expectation maximization (EM).