You may have noticed we extended our trends web coverage to include over 100 million domain names. We built a list of unique domain names, mostly .com, .net and .org’s and retrieved a total of 130 million domains. After we removed the ones that were redirects to others the list came down to around 90 million unique sites, this is up from the 11 million we used to report on.
So we downloaded the HTML for all of them! That was a fun learning curve for us, we used Amazon EC2 to help us out, without it we’d be waiting months for the results!
We then run some statistics on that and came up with this nice infographic –
We’ve also done some redesigning and will be continuing to stream-line all of the BuiltWith sites. The website optimizer has had a fresh lick of paint and so has Trends Pro. We’ve removed all of the parts that no one used and focused on what you actually want.
We’ve also achieved some good speed improvements on these sites. If you’d like to know what these sites have been (re)built with we used some awesome new technologies that really speeded up development, most important to this process was Modernizr, 960.gs, JQuery and .NET 4.
We’ll keep you updated on whats going on and if you have any new features you’d like to see let us know!