Web Crawler Design Issues: A Review
Journal Title: International Journal of Management, IT and Engineering - Year 2012, Vol 2, Issue 8
Abstract
The large size and the dynamic nature of the Web increase the need for updating Web based information retrieval systems. Crawlers facilitate the process by following the hyperlinks in Web pages to automatically download a partial snapshot of the Web. While some systems rely on crawlers that exhaustively crawl the Web, others focus on topic specific collections. In present paper the various types of crawlers are discussed. The paper also discusses several web crawler design issues along with their solutions.
Authors and Affiliations
Deepika And Dr Ashutosh Dixit
Keywords
Related Articles
- EP ID EP18526
- DOI -
- Views 365
- Downloads 13
Multi Objective Multi Agent Based AccessPoint Selection Mechanism using Fuzzy Logic
The last few years have seen a tremendous increase in the deployment of 802.11 Wireless Local Area Networks (WLANs). The proliferation of wireless users and the promise of converged voice, data and video technology is...
EFFECT OFREWARD MANAGEMENT ON EMPLOYEE PERFORMANCE OF SELECTED OILS AND GAS INDUSTRIES: EVIDENCE FROM SOUTH-SOUTH NIGERIA
The purpose of this study focuses on effect of reward management on employee performance of selected oils and gas industries in south -South Nigeria. Specifically the study aimed to pursue the following objectives: to...
Construction of Optimal Portfolio Using Sharpe’S Single index Model: An Empirical Study on Indian Computer Software Industry Stocks
Indian IT services companies are establishing their presence in all major global markets. Accordingly the primary concern of this research is to build an optimal portfolio from Indian Computer Software Industry with...
slugThe current situation, future prospect of Poverty and inequality in Sudan
This research paper aims to address income poverty and inequality in Sudan. Poverty and inequality indicators were computed using both primary and secondary data sources. P-alpha equation, Povstat and Simsip models wer...
slugAn Integrated Agricultural Information and Communication System for Management and Extension: A Comprehensive Framework
Electronic culture (e-culture) and the applications of Information and Communication Technology (ICT) are continuously increasingly in every sphere of human life. It play much bigger role in industry and service sector...