Librestock

Requirement

Our client has a number of photo gallery sites providing CCO-Licensed free photographs and videos.  They wanted to improve their SEO and develop a hub site to help bring everything together.

Web Scrapper

We developed a generic web scraper which daily crawls their current sites for new images.  When an image is identified, a thumbnail is of the image is generated and stored in a database with the image tags (to make search more efficient) and a link back to the full definition image.  Having scraped the sites the database is re-indexed to allow quick and effective searching.

Nightly batch jobs are set up to check all links and ensure they are still valid.  Any links failing are deleted to keep the database clean.

The result was so successful that we immediately built a video version of the site which can be found at video.librestock.com.