A ton of data, maybe more

I recently came across an interesting article [1] which discusses how much data is moved daily on Youtube. The bulk comes from 500h of video uploaded per minute. Yes, that is a pretty BIG number.

How much storage this river of data takes?

The typical video size for a medium quality video is about 0.5 GB per hour.

It follows that on Youtube people upload about 250 GB (gigabyte) per minute,

that is about 360 TB (terabyte, each terabyte is equal to 1000 GB) per day, that is about 130 PB (petabyte, each petabyte is equal to 1000000 GB) per year. Now it is getting even more impressive.

If I were to store these data on my own laptop I could not manage to withstand its flow for more than one single minute !

I was pretty excited and I talked about it with a friend over a glass of wine during a weekend. He questioned back, "Well, for how many minutes would you be able to store our data on your laptop?"

I could do a back-of-the-envelope for him, but I am lazy during weekends.. Luckily the answer was in CERN's Annual Report published in 2018 [2].

It surprised me that during the Large Hadron Collider (LHC) data taking we actually produce such a sizeable amount of data. It is comparable to what is being stored on Youtube per unit of time, about 30/50 PB per year ! If I were to store LHC data, I would saturate the whole hard disk of my laptop in about 3/4 minutes, not bad.

There are flaws in this statement however, although I thought it remains a funny coincidence. The logical flaw is that LHC data taking lasts for limited time over a day; moreover the detectors used to collect the data have to undergo long-shutdown periods when they are switched off get hardware upgrades to cope with varying data-taking conditions; they can last for multiple years. This is an interesting topic by itself but it will be for another weekend and another glass of wine.

Additional readings

[1] https://www.makeuseof.com/tag/how-much-data-does-youtube-use/.

[2] CERN Annual report 2017 https://cds.cern.ch/record/2624296.

[3] Figure on the side is copyright of CERN http://cdsweb.cern.ch/record/1103476#01.