Web Crawler Design Issues: A Review
Journal Title: International Journal of Management, IT and Engineering - Year 2012, Vol 2, Issue 8
Abstract
The large size and the dynamic nature of the Web increase the need for updating Web based information retrieval systems. Crawlers facilitate the process by following the hyperlinks in Web pages to automatically download a partial snapshot of the Web. While some systems rely on crawlers that exhaustively crawl the Web, others focus on topic specific collections. In present paper the various types of crawlers are discussed. The paper also discusses several web crawler design issues along with their solutions.
Authors and Affiliations
Deepika And Dr Ashutosh Dixit
Keywords
Related Articles
- EP ID EP18526
- DOI -
- Views 327
- Downloads 13
Web Crawler Design Issues: A Review
The large size and the dynamic nature of the Web increase the need for updating Web based information retrieval systems. Crawlers facilitate the process by following the hyperlinks in Web pages to automatically downloa...
SOIL RESISTIVITY AND EARTHING SYSTEM
Soil resistivity and earthing system plays a key role in generation, transmission and distribution for safe and proper operation of any electric power system. Soil resistivity directly affects the design of a grounding...
slugCAMPUS TO CORPORATE: A GAME OF TRANSITION
Every student has to face one of the biggest challenges i.e. the transition from campus (college) to corporate life. Corporat e life offers several challenges which young aspiring candidates often are unprepared...
Cloud- A Way to Uplift Traditional to Technological Education
Technology advancements have always had an important impact on development, affecting even the most important system of the country, the education system. Education plays an important role in maintaining the economic...
A STATISTICAL AND HISTORICAL ANALYSIS OF INDIA’S ECONOMY
During the last several years, a country that has been making serious moves towards becoming an industrialized country and changing its status is India. The authors’ purpose in this paper is to do an evaluation, histor...