Web Statistics 2011...

You may have noticed we extended our trends web coverage to include over 100 million domain names. We built a list of unique domain names, mostly .com, .net and .org’s and retrieved a total of 130 million domains. After we removed the ones that were redirects to others the list came down to around 90 million unique sites, this is up from the 11 million we used to report on.

So we downloaded the HTML for all of them! That was a fun learning curve for us, we used Amazon EC2 to help us out, without it we’d be waiting months for the results!

We then run some statistics on that and came up with this nice infographic -


click to enlarge

We’ve also done some redesigning and will be continuing to stream-line all of the BuiltWith sites. The website optimizer has had a fresh lick of paint and so has Trends Pro. We’ve removed all of the parts that no one used and focused on what you actually want.

We’ve also achieved some good speed improvements on these sites. If you’d like to know what these sites have been (re)built with we used some awesome new technologies that really speeded up development, most important to this process was Modernizr, 960.gs, JQuery and .NET 4.

We’ll keep you updated on whats going on and if you have any new features you’d like to see let us know!

Posted in News, Trends Update

The Top Three Internet Web Technologies...

Ignoring all the web standards (meta descriptions etc..), here’s the top 3 web technologies that are dominating the web  -

Number 3

JQuery has had an amazing two years, going from just over 4% of the top sites to an amazing 40%. JQuery is a javascript library that makes writing JavaScript much easier. You don’t have to worry about cross-platform issues and there’s lots and lots of libraries that extend the platform, making this the number one choice for web developers.

http://jquery.comJQuery Trends

Number 2

Apache Web Server has been around for as long as I can remember using the web. Used by just under half of the top 10k sites on the internet Apache has matured to be a stable feature rich web server that is free (open source). It’s market dominance is now under attack with other web servers such as NGINX taking over.

http://httpd.apache.org -  Apache Trends


Number 1

Google Analytics is sitting almost at the 60% mark for the top 10k sites on the internet. This free analytics package is a must-have for anyone wanted to track their visitors. It doesn’t really have any competition because its very difficult to offer a free product as good as this one.

http://google.com/analyticsGoogle Analytics Trends



Posted in Buzz

Top Websites...

Hello Everyone! We added a new feature today that lets you see the top sites that are running each web technology we cover in a nice grid view -

Top view this, simply go to BuiltWith Trends, view a tech and click the link under Top Sites. Here’s an example for Google Analytics.

Merry Christmas and a Happy New Year to everyone.

Posted in Buzz

Web Trends Highlights for June to September...

Hello, this is an email that went out a week ago, you can subscribe to the trends update email (very low frequency) on the Trends homepage at http://trends.builtwith.com.

We’ve picked some of the more interesting bits and have rounded them up below.

Remember you can always see the latest web trends for free at http://trends.builtwith.com.



Twitter / TweetMeme Show Down

In August, Twitter announced the Tweet Button, the defacto button up until that time had been TweetMeme’s very popular button.

It took two weeks for Twitter to get to the same market share that TweetMeme took a year to reach which is a telling signal for anyone competing with Twitter as a third party.

Tweet Button vs. TweetMeme Top 10k
Twitter vs. TweetMeme

The Tweet Button rise in usage was very quick and looks to have started to cut into TweetMeme’s share of the “Tweet This!” market with the TweetMeme button now in decline.



Google Buzz more a wimper

Google Buzz‘s “Share This” feature, that lets users share information about the website they are on via Buzz, has failed to gain any traction, with only about 300 of the top 1,000,000 sites having a Buzz button or follow link on their homepage. It’s usage is actually increasing but at a very slow rate, with only a handful of sites implementing the technology each month. This evidence along with Leo Laporte’s recent problems with Google Buzz lets us to believe this could
be another recent Google feature not gaining traction. Facebook and Twitter’s social implementation are doing very well in comparison.

Google Buzz Usage Statistics
Google Buzz Kill

Google are already top of nearly every other category on BuiltWith Trends (analytics, advertising, content delivery networks), they have a little way to go before reaching the top of social, maybe Google Me will be their breakthrough!



Getting old browsers to behave

Modern web browsers are supporting more and more cool features, making it easier for developers to build exciting websites, but what about all those older browsers that don’t support this new functionality, how are they supposed
to work? Traditionally a developer would spend days, weeks, months writing hacks to get their website working in multiple browsers. Conditional Comments are now in use on a third of the top ten thousand sites and a fifth of the top million, and new frameworks like modernizr are being developed to make cross-browser compatibility easier and easier, it’s easy to see why these technologies are taking off.

Modernizr Usage Statistics

Modernizr Growth



Chartbeat an analytics player

Since the start of August, Chartbeat, an analytics provider, has really started taking off. It fills the gap that Google Analytics (the biggest analytics player on the internet) left in its product by not providing a “real-time” aspect to analytics reporting. Will the growth continue into the future and will Google improve its analytics product to complete with Chartbeat?

Chartbeat Usage Statistics

Chartbeat



Goodbye 8 bit

8 bit is sooooooo 1975! When we started tracking trends over two years ago ISO/IEC 8859 encoding, an 8 bit set of encoding series, was in use by nearly half of websites on the internet, it’s now in use on less than a third.
The encoding group looking after ISO/IEC 8859 ceased operating in 2004 in favour of concentrating on universal character sets such as UTF-8 and UTF-16. UTF-8 has now skyrocketed and is well on the way to being the defacto standard for the internet.

ISO/IEC 8859 Slippery Slope

ISO/IEC Slippery Slope

Posted in News

Apple Extensions Gallery...

Thanks Apple for adding us to the Safari Extensions Gallery! We’ve got pride of place within the “developer” section of the website.

If you are a Safari user checkout the BuiltWith Safari Extension a non-intrusive quick lookup tool for websites to get a BuiltWith profile straight away.

More updates coming soon!

Posted in Buzz

Web Apps We Like...

Newsberry

We’re not entirely sold on this one yet but it is cheaper than all the competitors and it does the job. The process of creating a email from an advanced template

wasn’t as easy as it could have been. If you want to send email newsletters give Newsberry a try though, once over that mole hill it was plain sailing.

Postmark

Used for delivering emails from web applications. This is from the same company as Newsberry.

It is really handy if you don’t trust your mail server and want some feedback on email delivery and

spam reports. They also claim to do ISP throttling to help prevent emails being sent to spam,

and once again, the pricing is competitive.

Flickr

A great place to store pictures and a great community. I pay for a pro account on here so I can

have access to all my pictures and store as many as I want!

WordPress

I couldn’t mention top web applications without mentioning the one I’m using to write this blog post.

WordPress is an awesome bit of software!

Website Optimizer

No list is complete without one of our own products! The BuiltWith Website Optimizer gives daily updates

on our website optimization process, we also use it for little side projects to make sure we’re doing them right, like retailwith.com.

Posted in Buzz

New Trends Homepage...

We’ve updated the layout of the BuiltWith Trends homepage a little bit, to make it a bit more interactive and relevant. The homepage now shows the changes in the top 10,000 sites and also the top million sites.

The top two technologies for the week are also featured.

We hope you like the changes, check them out for yourself at http://trends.builtwith.com.

Posted in Buzz

Mozilla Automated Website Screen Grabber...

We have a requirement to do some screen grabbing and whilst there’s plenty of solutions for manual screen grabs there’s not much in the way of availability for automated screen grabs, as in, you feed an application a list of websites you want to get screenshots of and walk away from the computer.



ScreenGrabber in Action

Most of the examples that do exist are Internet Explorer based solutions. We’d rather not use Internet Explorer as the automated system might come across a website that exploits vulnerabilities in IE, so we wanted to use Firefox/Mozilla or a Chrome based solution. There are plenty of screenshot widgets for Firefox which is a great place to start.

So to give something back to the community we’re offering up some code we use in-house for a new project we are working on and we believe it may be the only XUL based automated screen grabber available at the moment.

Issues we had to Overcome to create an Automated Screen Grabber

1. Browser vulnerabilities – we didn’t want to use an Internet Explorer based solution so we opted for a Mozilla based solution using XULRunner.

2. Prompts – alert boxes, confirm boxes all sorts of modal dialogs can stop the automated process in its tracks and require a user to push a button. Using XULRunner and a modified version of Selenium’s promptService.js we managed to overcome this problem.

3. Flash – Typically flash will appear as a white box when the screenshot render happens if it’s “wmode” is not set to “transparent” so we actively change this when the page loads.

Flash does not render unless there’s an active window running, which is annoying especially it you run this on a remote desktop session (if you close the RDP window flash will not render on the screenshot grabber). To get around this is not too difficult and is explained later.

4. Events not Firing – Waiting for the page load to complete is not always accurate, we’ve got around this by waiting a maximum amount of time for a page to load and then just skipping to the next entry if no page load event fires.

Overcoming Flash not Rendering

Flash requires a “screen” to render, if there’s no active session, such as disconnected remote desktop session, Flash will appear as white boxes in your screenshots. To get around this install VNC Server and VNC Viewer, connect to a remote desktop session on your server and then open VNC Viewer to the local machine and run the application in the VNC Viewer window. The VNC Viewer application will keep alive when you disconnect the RDP session and continue to render Flash.

How to Install and Licence Details

The code for this application is far from perfect, it is basically built from a trial and error process so don’t expect the source code to be amazingly formatted and documented. You will need to do the following to get the application to work -

1. Install Firefox (not sure if you actually need to do this but we do it anyway to Install Flash for Firefox)

2. Install Flash for Firefox (or not if you don’t want Flash to render)

3. Install XULRunner

4. Register XULRunner with – xulrunner -register-global

5. Create a directory at c:\imgout \

6. Create a list of domains you want to screengrab, seperate them each with a new line and save to c:\domains.txt, for example-

google.com

altavista.com

mcdonalds.com

7. Run xulrunner application.ini -file c:\domains.txt

8. Watch as c:\imgout\ starts filling up with PNG’s.

Note this guide is for Windows, however the code should be cross-platform with a bit of modification to file paths in the mybrowser.js file in the package.

Download ScreenGrabber-1.0.zip 10KB

This is released under Apache 2.0 License.

Posted in News

BuiltWith Safari Extension...

The Safari BuiltWith Extension lets you see what a page is BuiltWith by right clicking and selecting “BuiltWith Analysis” from the context menu.

Download the Safari Extension Here

Posted in Uncategorized

Chrome Web Extension Homepage...

I just noticed we are on the homepage for Chrome Featured Web Developer Extensions! Thanks Google.

Thanks Google! If you have not tried Chrome we recommend you give it a go, it is much faster and easier to use than many of the alternatives and you can use the BuiltWith Chrome Extension.

Posted in Buzz