Big Data Analysis: Challenges and Solutions
Journal Title: International Journal of Scientific Research and Management - Year 2015, Vol 3, Issue 2
Abstract
We live in on - demand, on - command Digital universe with data prolife ring by Institutions, Individuals and Machines at a very high rate. This data is categories as "Big Data " due to its sheer Volume, Variety, Velocity and Veracity. Most of this data is unstructured, quasi structured or semi structured and it is heterogeneous in nature. The volume and the heterogeneity of data with the speed it is generated, makes it difficult for the present computing infrastructure to manage Big Data. Traditional data management, warehousing and analysis systems fall short of tools to analyze this data. Due to its specific nature of Big Data, it is stored in distributed file system architectu res. Hadoop and HDFS by Apache is widely used for storing and managing Big Data. Analyzing Big Data is a challenging task as it involves large distributed file system s which should be fault tolerant, flexible and scalable. Map Reduce is widely been used fo r the efficient analysis of Big Data. Traditional DBMS techniques like Joins and Indexing and other techniques like graph search is used for classification and clustering of Big Da ta. These techniques are being adopted to be used in Map Reduce. In this res earch paper the authors suggest various methods for catering to the problems in hand through Map Reduce framework over Hadoop Distributed File System (HDFS). Map Reduce is a Minimization technique which makes use of file indexing with mapping, sorting, shu ffling and finally reducing. Map Reduce techniques have been studied at in this paper which is implemented for Big Data analysis using HDFS.
Authors and Affiliations
Dipak M. Durgude
Keywords
Related Articles
- EP ID EP214706
- DOI -
- Views 76
- Downloads 0
Impact of FDI on Employment Generation in India
Job creation is one of the main challenges for developing cou ntries. Many people believe that FDI can generate many benefits to help solve the capital shortage problem in developing countries. But in...
Women Entrepreneurs with New Age Media
Almost all the people in our country collects and gets information about government, entertainment, sports, market, employment and so on through media. Media in include print media, electronic media and new age media. Me...
Detection of Conjugate P oints on Pair of Overlapping Image using Epipolar Correlation
Automation of the entire image mapping system is the ultimate goal of current research efforts in imagemetrology. In this paper, author presents the design and implementation of a MATLAB progra...
Account Security using Randomized 3D Environment Image
3D environment image with random textual password is a new scheme of authentication. This scheme is based on a virtual three - dimensional env ironment . To be authenticated, I present a 3 - D virtual environment...
Spatial and Temporal distribution of Waterborne D iseases in Namanyonyi Sub - C ounty, Mbale D istrict, Uganda
W aterborne disease is of great concern all over the world. Waterborne diseases represent significant burden of diseases in the globe. Nearly 4% of diseases are attributable to water, sanitation and hygiene, and appr...