Architecture Upgrade

The most up voted request for a feature on BuiltWith is to allow reports to be bigger than 500,000 results. We already added a method for exporting full technology lists which can contain millions and millions of records which we call “Mega Technology Reports” however they just contain the list of websites and not the additional meta data that we process.

moreThan500k

We are also about to add gigabytes of additional meta data for records, which includes telephone numbers, addresses and people names to records and realized we are going to hit a processing wall, where reports, although will still generate will take longer to build.

So we’ve embarked on a architecture renewal. Starting next week we will begin pushing our new database architecture into production and once fully deployed, being to increase the maximum size of a dashboard report on BuiltWith Pro.

 

 

Gary

Gary is co-founder of BuiltWith and uses a profile picture from 2006.