YouTube’s Architecture and Scalability
July 23, 2008 Tech and Security No CommentsHigh Scalability has a great link to a video TechTalk with Cuong Do, YouTube’s engineering manager. He talks about the challenges YouTube faces (past and present) to meet it’s skyrocketing user demand, as well as the infrastructure that allows them to scale. I enjoyed the anecdotes: especially the frantic email sent at 2am alerting the dev team that they only had 3 days of storage left… I always thought Google/YouTube would be immune to emergencies like that… ignorance on my part 
(requires Adobe Flash plugin… click HERE to watch it on YouTube)
I found this information interesting:
- The application code is written mostly in Python (the web app is not the bottleneck… the database RPC is)
- They use Apache for page content and lighttpd for serving video
- Thumbnails are now served by Google’s BigTable
- They’re running SuSE Linux with MySQL
- HW RAID-10 across multiple disks was too slow. HW RAID-1 with SW RAID-0 was faster because the Linux I/O scheduler could see the multiple volumes and would therefore schedule more I/O
You can read a good summary of the talk HERE from the High Scalability website.
A friend at work has been pointing me to some interesting videos out of Iraq. Having always been fascinated with military technology, I found this video particularly intriguing. It is video from an Apache attack helicopter watching (and subsequently eliminating) insurgents as they place an RPG in a field. This is a true testiment to the awesome power of its 30mm gun. Pay attention to the radio communications between the pilot and the forward-air-controller.
