Big Data

Big Data

This article deals with ‘ Big Data .’ This is part of our series on ‘Science and Technology which is an important pillar of the GS-3 syllabus. For more articles, you can click here


Big Data

What is Big Data?

  • Big data is a catchphrase used to denote a set of data, both structured and unstructured, so large that it is not possible to analyse and process it by using conventional database management systems. Such a huge surge in data has occurred due to a burgeoning number of information capturing devices like mobiles, cameras, sensors, etc. and a less increase in our storage capacity despite the latter being digital now.
  • It is characterised by 4Vs
    • Volume – Volume is huge
    • Velocity – It is being created at a huge speed.
    • Variety – It is of a large variety. 
    • Veracity – There is uncertainty about their truth.
4Vs of Big Data
  • Big Data in itself is not of use unless it is structured and analysed.

Use of Big Data Analytics

Business

  • Customisation of services: Companies use Big Data to analyse their customers’ preferences like likes and dislikes and tailor their products accordingly. This brings more satisfaction to the customer.
  • Big Data helps business in cost reductions, time reductions, new product development and optimised offerings.  Instead of focusing only on profit and loss, it integrates a wide range of insights, taking into account each and every factor that could possibly influence the business.

Science and Technology

  • Using Big Data, research data can be captured in more depth & analysed in a better way. Eg:  data at the Large Hadron Collider for atomic research.

Security

  • Big Data can be used to analyse fund transfer, emails, web accounts etc. to enhance our preparedness to tackle terrorism (by creating cyber trials).
  • Intelligence Bureau of India is also using Big Data analyses in its Operation Chakravyuh.

Governance

  • Big data can be analysed for targeted delivery of schemes, maintain a record of beneficiaries, analyse the response of the electorate to policies, predict future trends and demands of population etc.

Health

  • Big Data can be used in developing the treatment of various complicated diseases. Eg: canSAR Project (the canSAR project is the biggest database of cancer response to various drugs).  

Charitable sector

  • Some social organisations are using open government data to improve advocacy and fundraising. 

Sports

  • Big Data is used to analyse and improve the performance of individuals (at sports, at home or work) where data from sensors in equipment and wearable devices can be combined with video analytics to get insights that traditionally were impossible to observe.

Big Data in India

  • With a population of 1.2 billion, the relevance of Big Data Analysis becomes all the more pronounced for India.
  • It is not only being used by private players but also by government agencies for policymaking.

Big Data & Government

  • Big Data Management Policy, 2016: It was launched by CAG and led to the foundation of Data Analytics. Along with that National Informatics Centre (NIC) launched a website to share the data of various ministries with citizens.
  • NITI Aayog has also echoed the idea of evidence-based policymaking guided by Big Data.
  • Internal Security: NATGRID is an example of Big Data Analytics. Apart from that, the Intelligence Bureau of India is also using Big Data analyses in its Operation Chakravyuh.
  • The government is using large data from the Goods And Services Tax Network (GSTN) to understand the patterns of trade. 
  • Project Insight used Big Data to identify tax evaders.
  • The government of India is also working towards an Open Data Policy, to encourage sharing of information between departments and across ministries.
  • Aarogyashri Healthcare Trust by the Telangana government uses data analytics to identify disease trends.
  • Justice BN Srikrishna Committee recommended that personal data can be processed for purposes that are lawful. An individual has the right to withdraw consent to process his data. 

Issues (wrt India)

  • Absence of good quality of datasets as the dataset is found to be
    1. Outdated and incomplete.
    2. Lacking in semantic interoperability.
  • Lack of competent professionals/ data scientists: According to NASSCOM, there is a deficit of 1.4 lakh data scientists.
  • Privacy Concern: There are concerns about the misuse of Big Data by intruding into the personal sphere of individuals.
  • Data Sovereignty Issue: Indian data collected for Data processing is taken abroad and stored in servers situated outside India.
  • Lack of coordination and cooperation between different ministries as well as between public and private sector.
  • Ethics of big data: It comes into role as a huge amount of private data is available and how and where it should be put to use raises the question.