SEATTLE — Imagine what cell phone metadata can do besides helping developers fix software bugs – like mapping poverty. In recent years, real-world applications for “big data” have thrived in different fields, from weather forecasts to urban planning and sports management. Now big data technology is also serving as a valuable tool for researchers to tackle the challenges of global poverty.
“Big data” refers to the voluminous data streams that are too large for traditional data processing software to handle. The flood of data – usually coming with extreme variety and speed – often provides a much more comprehensive report on the subject than traditional data sets. While big data is not a novelty in many tech-friendly fields, as it offers valuable information for behavioral analysis, big data is also slowly revolutionizing researchers’ approaches to dealing with global poverty.
The Capabilities of Big Data Technology
Big data comes from all aspects of life. In a July 2017 interview with Thomson Reuters Foundation, Rutgers University Big Data Program Chair Paul Szyarto illustrated the great variety and the magic of big data with current projects.
Many developing countries like India and Kenya, said Szyarto, are utilizing historical meteorological record and statistical models to forecast weather patterns, which could help farmers to plan agricultural activities accordingly. An improved crop yield can then can reduce hunger.
By relying on the data from previous epidemics, such as Ebola, West African nations are able to predict the origin of future disease outbreaks and the most vulnerable regions, allowing these areas to carry out targeted prevention programs.
The condition and distribution of poverty are essential for policymaking and aid allocation. However, in many underdeveloped or war-torn countries, traditional survey methods are difficult to perform, and it is nearly impossible to acquire timely and comprehensive information. Instead of conducting time-consuming and expensive home-to-home surveys, big data technology makes poverty-mapping relatively quick and easy.
University of Washington researchers managed to estimate the wealth of a region using only cell phone metadata provided by the local telephone company. They extracted the times, locations and events of cell phone activities from cell phone metadata generated by texts and calls and labeled these data as “hallmarks of socioeconomic status”. The researchers then used computers to sort through these big data sets and infer patterns about living style and economic activities, eventually constructing an estimation of wealth distribution of the region.
Big data can be in the form of images as well. Led by assistant professor of earth system science Marshall Burke, a team at Stanford University is combining a machine learning algorithm and millions of daytime and nighttime satellite imagery of poorer regions in the world to predict village-level wealth.
Experts’ Opinions on Big Data
University of Washington Information School assistant professor Joshua Blumenstock, the supervisor of the “poverty estimation with cell phone metadata” project, expressed his optimism about the current progress and future practical uses of big data technology. He believes big data can eventually lead to better poverty relief policies.
While acknowledging the great potential of big data analytics for delivering environmental, agricultural and social solutions, Szyarto believed that the majority of developing countries do not have the necessary equipment and knowledge to gather and study the data being generated.
On the flip side, big data has also spawned many concerns, primarily the risk of data abuse. In her new book “Weapon of Math Destruction”, the former director of the Lede Program for Data Journalism at Columbia University, Cathy O’Neil, warns of the dark side of big data. Due to conflicts of interest, different groups of people may want different results from the same data sets, thus leading to intrinsically biased algorithms, which then could give policymakers false information and exacerbate the original social problem.
Big data might well be the future weapon that could eradicate long-lasting social issues like global poverty. If they use the data with discretion and responsibility, researchers and policymakers could optimize big data technology’s vast potential.
– Chaorong Wang