Big data means benefits and challenges for people, businesses and the environment to shape global, regional and industry agendas. Our use of electronic devices is increasing and production processes are becoming ever more digitalised. This means that vast quantities of digital data are generated daily in the economy and by people’s personal and social activities.
What Big Data is and why it matters?
Big Data is a collection of data that is huge in volume, yet growing exponentially with time. Big data is also a data but with huge size. Although the concept of big data itself is relatively new, the origins of large data sets go back to the 1960s and ’70s when the world of data was just getting started with the first data centers and the development of the relational database.
Big data is a combination of structured, semistructured and unstructured data collected by organizations that can be mined for information and used in machine learning projects, predictive modeling and other advanced analytics applications. Companies use the big data accumulated in their systems to improve operations, provide better customer service, create personalized marketing campaigns based on specific customer preferences and, ultimately, increase profitability. Businesses that utilize big data hold a potential competitive advantage over those that don’t since they’re able to make faster and more informed business decisions, provided they use the data effectively.
Social Media Big Data
Around 2005, people began to realize just how much data users generated through Facebook, YouTube, and other online services. The development of open-source frameworks, such as Hadoop (and more recently, Spark) was essential for the growth of big data because they make big data easier to work with and cheaper to store. In the years since then, the volume of big data has skyrocketed. Users are still generating huge amounts of data—but it’s not just humans who are doing it.
The European Commission forecasts the total global amount of data will grow 530% by 2025 compared to 2018.
Data is an important part of the EU’s digital transformation. Artificial intelligence, an EU priority, relies on data and its development depends on how data will be managed in Europe. Data is also an integral part of the digital services that shape our everyday lives and the economy. Parliament is working on data legislation to ensure people, businesses, recovery and the green transition benefit from an efficient data strategy.
Big Data in Europe
Big data refers to collected data sets that are so large and complex that they require new technologies, such as artificial intelligence, to process. The data comes from many different sources. Often they are of the same type, for example, GPS data from millions of mobile phones is used to mitigate traffic jams; but it can also be a combination, such as health records and patients’ app use. Technology enables this data to be collected very fast, in near real time, and get analysed to get new insights.
Where does big data come from?
Big data can be produced by people: in mobile apps, on the web, including social media and commercial transactions, e-government records…
It can also be generated by machines and collected through sensors in objects linked to the Internet of Things, including smart cars, factories, GPS satellites and satellites collecting weather data etc.
Opportunities created by the use big data
Big data presents great opportunities in a number of areas.
Volume is the most commonly cited characteristic of big data. A big data environment doesn’t have to contain a large amount of data, but most do because of the nature of the data being collected and stored in them. Big data can provide companies with valuable insights into their customers that can be used to refine marketing campaigns and techniques in order to increase customer engagement and conversion rates.
Historical and real-time data can be used to assess the evolving preferences of consumers, consequently enabling businesses to update and improve their marketing strategies and become more responsive to customer desires and needs. Manufacturers and transportation companies rely on big data to manage their supply chains and optimize delivery routes. Other government uses include emergency response, crime prevention and smart city initiatives.
Industry
Big data enables companies to innovate, either through better analysing people’s needs and wants or by offering entirely new products. While personal data is central for the operation of apps and platforms that have become an important part of our lives and economy, better exploitation of industrial data could bring a new wave of innovation in the EU. Data can also boost productivity and help cut costs, for example by predicting sales or maintenance in smart factories.
Environment
Satellite data can improve research and help the EU reduce greenhouse emissions and help in preventing and responding to natural disasters such as wildfires. Improving production efficiency in ithe ndustry should lower emissions and waste.
Healthcare
The analysis of large clinical data sets – for example: anonymised health records or data entered by patients in apps – can enable better diagnostics, treatment and development of medicine, while reducing costs.
Agriculture
Farmers can use data from satellites and sensors to better use resources such as water or sunlight and adapt crops to changing circumstances.
Public sector
Data and advanced analytics can increase the efficiency and effectiveness of public services, to improve transparency and offer better-tailored services.
Transportation
Big data collected from GPS and social media can help mitigate traffic jams. Better regulation of traffic flows also contributes to savings in time and fuel and lowers CO2 emissions.
European Data Strategy
Big data challenges and missed opportunities
If the EU misses the opportunity to exploit the potential of big data, it could hamper the implementation of major EU programmes, such as the Green Deal, and affect consumers, businesses and the economy. Making the data in big data systems accessible to data scientists and other analysts is also a challenge, especially in distributed environments that include a mix of different platforms and data stores.
Big data systems must be tailored to an organization’s particular needs, a DIY undertaking that requires IT teams and application developers to piece together a set of tools from all the available technologies. Deploying and managing big data systems also require new skills compared to the ones possessed by database administrators (DBAs) and developers focused on relational software.
Protection of rights
As digital service providers obtain more information about users, an unregulated power imbalance might cause preferences and even weaknesses to be exploited for commercial or political gain. Highly targeted advertisements raise concerns about possible manipulation, where the preferences and even weaknesses of consumers are exploited.
Automated data based assessments could mean individuals or groups are categorised, which could lead to them being excluded from, for example, professional opportunities or medical coverage.
