High Scalability has a great link to a video TechTalk with Cuong Do, YouTube’s engineering manager. He talks about the challenges YouTube faces (past and present) to meet it’s skyrocketing user demand, as well as the infrastructure that allows them to scale. I enjoyed the anecdotes: especially the frantic email sent at 2am alerting the dev team that they only had 3 days of storage left… I always thought Google/YouTube would be immune to emergencies like that… ignorance on my part

(requires Adobe Flash plugin… click HERE to watch it on YouTube)

I found this information interesting:

• The application code is written mostly in Python (the web app is not the bottleneck… the database RPC is)
• They use Apache for page content and lighttpd for serving video
• Thumbnails are now served by Google’s BigTable
• They’re running SuSE Linux with MySQL
• HW RAID-10 across multiple disks was too slow. HW RAID-1 with SW RAID-0 was faster because the Linux I/O scheduler could see the multiple volumes and would therefore schedule more I/O

You can read a good summary of the talk HERE from the High Scalability website.

TechCruch has a good article of the YouTube API.

Share/Save