Small data: What it is and how it differs from big data

• Author:
• Author name

  Quantumrun Foresight
• April 7, 2022

Insight summary

Small data is transforming the way small and medium-sized businesses operate, enabling them to make tactical decisions with insights that were once reserved for larger corporations. From novel mobile apps that enhance personal productivity to rural hospitals improving healthcare accessibility, small data is becoming a versatile tool across various sectors. The trend's long-term implications include shifts in consumer behavior, the development of cost-efficient tools for businesses, and government support for local economies.

Small data context

Small data is the division of data into small sets, volumes, or formats that can be analyzed by traditional software and which humans can easily understand. Big data, by comparison, is voluminous data sets that conventional data programs or statistical methods cannot manage, instead requiring specialized software (and even supercomputers) to be analyzed and processed.

The term small data was coined by IBM researchers in 2011, being data represented in data sets that are less than one thousand rows or columns. Small data sets are small enough that they can be analyzed by simple estimation and easy-to-access digital tools. Small data can also be big data sets that have been broken down to the degree that they become accessible, understandable, and actionable by human beings.

Small data is typically used to provide analysis and insights of a current situation so that a business can make immediate or short-term decisions. In comparison, big data can be structured and unstructured data sets that are large in size and can provide insights relating to long-term business strategy. Big data also requires more sophisticated software and skills to produce these insights, so as a result, it can be more costly to manage.

Disruptive impact

The utilization of small data in decision-making processes is becoming an essential tool for small and medium-sized businesses, such as restaurants, bars, and hair salons. These businesses often need to make tactical decisions on a daily or weekly basis, and small data provides them with valuable insights without the complexity or cost of big data. By analyzing customer behavior, sales trends, and other relevant information, small data can assist business leaders in determining workforce size, pricing strategies, and even the potential for opening new branches.

Technology companies are recognizing the potential of small data and are working to develop tools that are both cost-efficient and highly effective. The development of these tools can lead to a more level playing field, where small businesses can compete more effectively with their larger counterparts. However, the challenge lies in creating tools that are user-friendly and tailored to the specific needs of different industries, ensuring that they are not just affordable but also practical and relevant.

For governments, the rise of small data presents an opportunity to support local economies and foster growth within various sectors. By encouraging the use of small data and supporting the development of tools tailored to the needs of small businesses, governments can help create a more dynamic and responsive business environment. However, there may need to be considerations around privacy and security, ensuring that the collection and use of data are done responsibly. Educating businesses on best practices and providing guidelines can be essential in ensuring that this trend is harnessed effectively, without compromising the trust and integrity that are vital to business success.

Implications of small data 

Wider implications of small data may include:

• Novel mobile apps and virtual voice assistants helping individuals make more efficient time usage decisions, leading to enhanced personal productivity and a more balanced lifestyle.
• Businesses leveraging small data to streamline their payroll and inventory purchases, leading to optimized operational costs and a more responsive supply chain.
• Rural hospitals using small data to effectively manage patient data and provide medical services, leading to improved healthcare accessibility and quality in underserved areas.
• The development of user-friendly small data tools targeting specific industries, leading to a more competitive market where small businesses can make data-driven decisions on par with larger corporations.
• Governments supporting the growth of small data usage through incentives and regulations, leading to a more vibrant small business sector and potential economic growth in local communities.
• An increased focus on privacy and security in the collection and use of small data, leading to the establishment of new laws and standards that protect individual rights without hindering business innovation.
• A shift in consumer behavior as small businesses become more adept at personalizing services and products through small data insights, leading to a more tailored and satisfying shopping experience.

Questions to consider

• What examples have you experienced where small data has made businesses more efficient and profitable?
• What sectors do you think can benefit most from using small data instead of using big data?