YouTube’s Architecture and Scalability

Tech and Security No Comments

High Scalability has a great link to a video TechTalk with Cuong Do, YouTube’s engineering manager. He talks about the challenges YouTube faces (past and present) to meet it’s skyrocketing user demand, as well as the infrastructure that allows them to scale. I enjoyed the anecdotes: especially the frantic email sent at 2am alerting the dev team that they only had 3 days of storage left… I always thought Google/YouTube would be immune to emergencies like that… ignorance on my part :-)

(requires Adobe Flash plugin… click HERE to watch it on YouTube)

I found this information interesting:

  • The application code is written mostly in Python (the web app is not the bottleneck… the database RPC is)
  • They use Apache for page content and lighttpd for serving video
  • Thumbnails are now served by Google’s BigTable
  • They’re running SuSE Linux with MySQL
  • HW RAID-10 across multiple disks was too slow. HW RAID-1 with SW RAID-0 was faster because the Linux I/O scheduler could see the multiple volumes and would therefore schedule more I/O

You can read a good summary of the talk HERE from the High Scalability website.

TechCruch has a good article of the YouTube API.

Website validation and embedded YouTube videos

Code Monkey No Comments

W3CSo I took the plunge today and got my website to pass CSS and XHTML validation tests. This ensures that my site renders correctly in most browsers and on mobile devices. Check out UITest.com for links to several validation tools.

The hardest thing to get validated are embedded YouTube videos. The <embed> tag they use is depreciated in XHTML 1.0. As soon as you embed a video, your site will fail validation. I found several code samples for fixing this, but only one of them actually worked. I just had to tweak the size so the output matched that from the YouTube script.

Here’s the HTML:

<object type=”application/x-shockwave-flash” data=”http://www.youtube.com/v/sM_WEohK3bo&hl=en” width=”425″ height=”355″>
<param name=”movie” value=”http://www.youtube.com/v/sM_WEohK3bo&hl=en” />
<param name=”FlashVars” value=”playerMode=embedded” />
<param name=”wmode” value=”transparent” />
</object>

The original script came from HERE.

The only problem I’ve had is with the visual editor in Wordpress auto-replacing this code with the original YouTube code.