This article deals with ‘ Big Data .’ This is part of our series on ‘Science and Technology’ which is important pillar of GS-3 syllabus . For more articles , you can click here
What is Big Data
- Big data is a catchphrase used to denote a set of data, both structured and unstructured, so large that it is not possible to analyse and process it by using conventional database management systems. Such a huge surge in data has occurred due to burgeoning number of information capturing devices like mobiles, cameras, sensors, etc. and a less increase in our storage capacity despite the latter being digital now
- It is characterised by 4Vs
- Volume – Volume is huge
- Velocity – It is being created at a huge speed
- Variety – It is of large variety
- Veracity – There is uncertainity about their truth
- Big Data in itself is not of use unless it is structured and analyzed.
Use of Big Data Analytics
1.Business: Customisation of services
- Companies use Big Data to analyse their customers’ preferences like likes and dislikes and tailor their products accordingly . This bring more satisfaction to the customer
2. Science and Technology
- Research data can be captured at more depth & analysed in a better way. Eg data at Large Hadron Collider for atomic research.
- Analysis of fund transfer, emails, web accounts can enhance our preparedness to tackle terrorism (cyber trails)
- Operation Chakravyuh of IB
- Big data can be analysed for targeted delivery of schemes, maintain record of beneficiaries, analyse the response of the electorate to policies, predict future trends and demands of population.
- Big Data can be used in developing treatment of various complicated diseases
- Eg :canSAR Project (canSAR project is the biggest database of cancer response to various drugs
6. Charitable sector
- Some social organisations are using e open government data to improve advocacy and fundraising
7. Other uses (Sports etc)
- Big Data is used to analyze and improve the performance of individuals (at sports, at home or work) where data from sensors in equipment and wearable devices can be combined with video analytics to get insights that traditionally were impossible to see.
Big Data in India
- With a population of 1.2 billion, the relevance of Big Data becomes all the more pronounced for India.
- It is not only being used by private players but also government agencies for policymaking.
Big Data & Government
- NITI Aayog also echoed the idea of evidence-based policymaking guided by Big Data.
- Internal Security : NATGRID is example of Big Data Analytics + Operation Chakravyuh of IB
- Use of preliminary data from the Goods And Services Tax Network (GSTN) to understand the patterns of trade
- Project Insight will use Big Data to identify tax evaders.
- Aarogyashri Healthcare trust by Telangana government uses data analytics to identify disease trends
- Justice BN Srikrishna Committee : Personal Data can be processed for purposes that are lawful . Individual has right to withdraw consent to process his data
Issues (wrt India)
- Absence of good quality of datasets: Dataset are found to be
- Outdated and incomplete
- lacking in semantic interoperability
- Lack of competent Professional: According to NASSCOM, deficit of 1.4 lakh workers
- Privacy Concern: There are concerns about the misuse of Big Data by intruding in personal sphere of an individuals.
- Data Sovereignty Issue : Indian data collected for Data processing taken abroad and stored in servers situated outside India
- Lack of Coordination and Cooperation between different ministries as well as between public and private sector