This is a guest post by Katy Pearce, an assistant professor in the University of Washington's Department of Communication.
Governments and policy pundits frequently cite technology-penetration rates -- Internet, mobile phone, or social media -- as meaningful proxies for deeper concepts. Despite this, in fact, penetrations rates are futile.
Why should anyone care? Because inferences into what social-media- or mobile-phone- or Internet-penetration rates represent are dangerous.
When pundits or government officials mention penetration rates, it is often in support of a bigger social or political goal: "84 percent of women in Country X have a mobile phone"..."[Thus] women's empowerment through technology is possible." Or: "With 80 percent of the country using the Internet, we can say that we've achieved our economic and technological goals" or “most African-American households have Internet access, so the digital divide is over” or 1 million Facebook users means that there is freedom of expression. Given the challenges in measuring technology penetration, it is astonishing that speakers continue to make such statements.
Why is technology penetration so difficult to measure?
1. The contributing factors to technology adoption are well-known, but become embedded within discussions of penetration rates.
Technologies tend to diffuse in similar patterns, with similar factors determining early versus later adopters. In nearly all societies, the wealthier, the better educated, the more urban, and the younger adopt new technology earlier than the poorer, the less educated, the more rural, and the older do. Thus, it is unsurprising that in societies with more wealthy people or better distributed education systems, there are higher technology-penetration rates. Similarly, societies with higher proportions of younger people will have higher technology-penetration rates.
At a macro level, telecommunications systems, competition, price, and national wealth can also influence penetration rates.
Thus when comparing countries or other entities based on percentage of individuals that have adopted a technology, analyses that do not control for these differences are misrepresenting the actual technology landscape.
2. Counts are notoriously inaccurate.
Counting technology users is difficult. In lieu of a properly sampled nationally representative survey (which is not perfect either -- social desirability and response biases abound), penetration rates generally come from the International Telecommunication Union (ITU), a United Nations agency that is responsible for information and communication technologies.
Where does the ITU get this data? From "administrative data sources" -- which are "mainly telecommunication operators, and are collected by governments at the national level (ministries or regulatory authorities)." Without a doubt, some governments have reason to inflate penetration rates and there are few checks on this by the ITU. In an interview with RFE/RL's Azerbaijani Service, a representative of the Azerbaijani Telecommunications Ministry cited the ITU data as an authoritative source of Internet penetration in the country, without acknowledging that it was his own ministry that supplied that data.
Another issue with measuring penetration rates is the nature of the technologies. Mobile-phone subscriptions, for example, are notoriously inaccurate. Most count SIM card subscriptions, without acknowledging that it is incredibly common in many countries for individuals to possess multiple SIM cards -- for business purposes or to economize on voice versus data rates. Therefore when one hears that 75 percent of the world has a mobile phone, it is more accurate to say that there are 75 SIM cards (some active, some inactive) per 100 people in the world. The number of people with an active phone in their hand is much less.
With social-media platforms it is very difficult to determine the true location of a user. With choices about listing location and proxy servers that can allow a user to appear to be located in a different country than s/he is actually in, accuracy is questionable. Further, these penetration rates include all of those social-media accounts that were opened and never used again. Additionally, sites that produce social-media penetration rates (like Socialbakers.com) are interesting to get a sense of over-time growth, but because these sites are for profit entities selling analytics to marketers without transparent (in the name of good science) methodologies for determining penetration rates, these too are unreliable.
3. "Ever used" is a fairly meaningless category.
Another issue with penetration rates is that they are all-inclusive. An Internet user includes not only the young woman with a smartphone, a tablet computer, and a laptop who is online for all of her waking hours, but also the older man who went to an Internet cafe once four years ago and never went back or the person who Skypes with a distant relative every few months. All of these individuals are experiencing a different Internet and benefitting from their technology use differently as well.
A more useful measure would be of Internet frequency, because daily users are different from monthly users. Even so, measuring perpetually connected people's Internet use is futile. Consider this: how many hours are you online each day? Does checking your e-mail while waiting at the dentist’s office count as five minutes of use? If your smartphone is constantly connected to the Internet are you online 24 hours a day?
4. Speed matters.
While for many Americans and Europeans, nonbroadband Internet is a distant memory, for many in the world dial-up Internet, mobile-based connectivity, or metered limitations are daily realities. Multimedia sites and videos are something that a resource-constrained user has to consider before opening. While more sites have low-bandwidth mobile versions, the Internet becomes more bandwidth-intensive all the time, which can leave resource-constrained users behind, essentially creating different Internets.
5. Activities matter.
People assume a lot about what others do with technology. The average North American may spend their day checking Facebook, reading news stories, texting, and e-mailing. This may not be the case for everyone. An Internet user concerned about resources, for example, may be more cautious in her use. A recent academic study, “'Facebook is a Luxury': An Exploratory Study of Social Media Use in Rural Kenya,” explains that there is a cost associated with every phase of signing up for and maintaining a Facebook account -- beyond the Internet cafe (and travel costs to get to the cafe) or mobile Internet fees (and fees associated with charging the phone’s battery), users felt compelled to scan their best photographs for profile pictures, then pay to upload them, and continue using the site. Certainly these users are experiencing a different Facebook than my American undergraduate students do.
It's important to acknowledge that people do different things with their technology and that these different activities are not only determined by access to resources, cultural norms, and personal characteristics, but also have different outcomes. News reading is enriching for some things, while a first-person-shooter game may not be. Watching videos of cats falling off tables has a different impact than taking an online course does. So when penetration rates are cited, it is entirely possibly that many of those users are watching pornography, playing Farmville, or reading celebrity blogs.
So what is to be done about this problem? There are a few options. One may be to stop caring about penetration rates. This is not a race. Given the numerous contributing factors to these rates, the actual percentage is essentially meaningless. Given the way that these are manipulated, deemphasizing penetration rates may be best. Another solution would be an attempt at better measurement. Surveys would be best. Nonetheless there are many things that can go wrong within a survey. Governmental control, poor capacity in some places, and difficulties in cross-cultural comparison would also need to be dealt with.
Perhaps the easiest solution is an educational campaign to train journalists, governments, and pundits to be more critical consumers of statistics. While a revitalization of the global high-school statistics curriculum is a lofty goal, promoting basic statistical literacy and specifically the issues surrounding these penetration rates, the ability for manipulation could be reduced.
-- Katy Pearce
Governments and policy pundits frequently cite technology-penetration rates -- Internet, mobile phone, or social media -- as meaningful proxies for deeper concepts. Despite this, in fact, penetrations rates are futile.
Why should anyone care? Because inferences into what social-media- or mobile-phone- or Internet-penetration rates represent are dangerous.
When pundits or government officials mention penetration rates, it is often in support of a bigger social or political goal: "84 percent of women in Country X have a mobile phone"..."[Thus] women's empowerment through technology is possible." Or: "With 80 percent of the country using the Internet, we can say that we've achieved our economic and technological goals" or “most African-American households have Internet access, so the digital divide is over” or 1 million Facebook users means that there is freedom of expression. Given the challenges in measuring technology penetration, it is astonishing that speakers continue to make such statements.
Why is technology penetration so difficult to measure?
1. The contributing factors to technology adoption are well-known, but become embedded within discussions of penetration rates.
Technologies tend to diffuse in similar patterns, with similar factors determining early versus later adopters. In nearly all societies, the wealthier, the better educated, the more urban, and the younger adopt new technology earlier than the poorer, the less educated, the more rural, and the older do. Thus, it is unsurprising that in societies with more wealthy people or better distributed education systems, there are higher technology-penetration rates. Similarly, societies with higher proportions of younger people will have higher technology-penetration rates.
At a macro level, telecommunications systems, competition, price, and national wealth can also influence penetration rates.
Thus when comparing countries or other entities based on percentage of individuals that have adopted a technology, analyses that do not control for these differences are misrepresenting the actual technology landscape.
2. Counts are notoriously inaccurate.
Counting technology users is difficult. In lieu of a properly sampled nationally representative survey (which is not perfect either -- social desirability and response biases abound), penetration rates generally come from the International Telecommunication Union (ITU), a United Nations agency that is responsible for information and communication technologies.
Where does the ITU get this data? From "administrative data sources" -- which are "mainly telecommunication operators, and are collected by governments at the national level (ministries or regulatory authorities)." Without a doubt, some governments have reason to inflate penetration rates and there are few checks on this by the ITU. In an interview with RFE/RL's Azerbaijani Service, a representative of the Azerbaijani Telecommunications Ministry cited the ITU data as an authoritative source of Internet penetration in the country, without acknowledging that it was his own ministry that supplied that data.
Another issue with measuring penetration rates is the nature of the technologies. Mobile-phone subscriptions, for example, are notoriously inaccurate. Most count SIM card subscriptions, without acknowledging that it is incredibly common in many countries for individuals to possess multiple SIM cards -- for business purposes or to economize on voice versus data rates. Therefore when one hears that 75 percent of the world has a mobile phone, it is more accurate to say that there are 75 SIM cards (some active, some inactive) per 100 people in the world. The number of people with an active phone in their hand is much less.
With social-media platforms it is very difficult to determine the true location of a user. With choices about listing location and proxy servers that can allow a user to appear to be located in a different country than s/he is actually in, accuracy is questionable. Further, these penetration rates include all of those social-media accounts that were opened and never used again. Additionally, sites that produce social-media penetration rates (like Socialbakers.com) are interesting to get a sense of over-time growth, but because these sites are for profit entities selling analytics to marketers without transparent (in the name of good science) methodologies for determining penetration rates, these too are unreliable.
3. "Ever used" is a fairly meaningless category.
Another issue with penetration rates is that they are all-inclusive. An Internet user includes not only the young woman with a smartphone, a tablet computer, and a laptop who is online for all of her waking hours, but also the older man who went to an Internet cafe once four years ago and never went back or the person who Skypes with a distant relative every few months. All of these individuals are experiencing a different Internet and benefitting from their technology use differently as well.
A more useful measure would be of Internet frequency, because daily users are different from monthly users. Even so, measuring perpetually connected people's Internet use is futile. Consider this: how many hours are you online each day? Does checking your e-mail while waiting at the dentist’s office count as five minutes of use? If your smartphone is constantly connected to the Internet are you online 24 hours a day?
4. Speed matters.
While for many Americans and Europeans, nonbroadband Internet is a distant memory, for many in the world dial-up Internet, mobile-based connectivity, or metered limitations are daily realities. Multimedia sites and videos are something that a resource-constrained user has to consider before opening. While more sites have low-bandwidth mobile versions, the Internet becomes more bandwidth-intensive all the time, which can leave resource-constrained users behind, essentially creating different Internets.
5. Activities matter.
People assume a lot about what others do with technology. The average North American may spend their day checking Facebook, reading news stories, texting, and e-mailing. This may not be the case for everyone. An Internet user concerned about resources, for example, may be more cautious in her use. A recent academic study, “'Facebook is a Luxury': An Exploratory Study of Social Media Use in Rural Kenya,” explains that there is a cost associated with every phase of signing up for and maintaining a Facebook account -- beyond the Internet cafe (and travel costs to get to the cafe) or mobile Internet fees (and fees associated with charging the phone’s battery), users felt compelled to scan their best photographs for profile pictures, then pay to upload them, and continue using the site. Certainly these users are experiencing a different Facebook than my American undergraduate students do.
It's important to acknowledge that people do different things with their technology and that these different activities are not only determined by access to resources, cultural norms, and personal characteristics, but also have different outcomes. News reading is enriching for some things, while a first-person-shooter game may not be. Watching videos of cats falling off tables has a different impact than taking an online course does. So when penetration rates are cited, it is entirely possibly that many of those users are watching pornography, playing Farmville, or reading celebrity blogs.
So what is to be done about this problem? There are a few options. One may be to stop caring about penetration rates. This is not a race. Given the numerous contributing factors to these rates, the actual percentage is essentially meaningless. Given the way that these are manipulated, deemphasizing penetration rates may be best. Another solution would be an attempt at better measurement. Surveys would be best. Nonetheless there are many things that can go wrong within a survey. Governmental control, poor capacity in some places, and difficulties in cross-cultural comparison would also need to be dealt with.
Perhaps the easiest solution is an educational campaign to train journalists, governments, and pundits to be more critical consumers of statistics. While a revitalization of the global high-school statistics curriculum is a lofty goal, promoting basic statistical literacy and specifically the issues surrounding these penetration rates, the ability for manipulation could be reduced.
-- Katy Pearce