Counting on Beans, Knots, and Notches

A Short, Thrilling History of Data Collection

By Oz Guner in musings

April 6, 2023

Every time I see an interesting graph relaying information about obscure facts or phenomena, the first question that comes to my mind is the data collection. How do you collect this data? How do you store it? How do you analyze it? Data collection and data quality is, and always will be, the central part of any collection, aggregation, and analytics process.

Antiquity

As every data collection process, census record started with a simple curiosity: How many of us are there? The agricultural surplus led to a higher quality of life and increased population. Consequently, we saw the need to count ourselves. The first known census dates back to ancient Egypt. The primary reason of counting heads was accurate taxation. Ancient Egyptians used the same method to plan for land allocation following the annual flooding of the Nile. Egyptian officials visited each household and recorded the population on papyrus scrolls, probably using the hieroglyphics, a system of symbols and pictures used to represent words and ideas. The data collection was nevertheless complicated, messy, and insanely expensive.

Middle Ages

In pre-Renaissance Florence, the nurses found a creative way to collect data on newborn babies: They would put in a jar a black bean for a boy and a white bean for a girl. I’d love to imagine that this started in one jar. Later, they must have used a second jar to count boys and girls differently to create an early form of data visualization: If the white beans are higher than black beans in a jar, then there must be more girls born that day. No need to even count. Yet there were still thousands of unattended and unrecorded births every year, which added to the rapidly growing population problem that Florence had suffered from.

Modernity

An example of a modern data data collection is punch cards. As a former restaurant employee, I wholeheartedly hate them and I’d love to thank Toast for introducing digital punch cards. Basically, each time a card is put in machine, the data iterates over itself for one time, creating one record. The “punch” would be stamped permanently on the card, indicating the proof of record. Punch cards are a creation of late industrial society but are also used in the military. They were extremely ineffective because of a simple reason: You wouldn’t know who punched them unless you witnessed that moment personally. Modern-day punching methods are more effective in that respect. Physical cards were also vulnerable to counterfeit, misuse, and loss.

This short history of data collection shows that we will always be prone to error no matter what we are counting, measuring, and analyzing. Even with modern technologies, one of the primary problems when working with data is the data quality: incorrectly filled out parameters, blank fields, failed automations, etc. While we are getting much better at data collection, we will always be chasing that perfectly clean data set.

Posted on:
April 6, 2023
Length:
3 minute read, 486 words
Categories:
musings
Tags:
data collection
See Also: