Ultimate ОLХ scraper

A client requested to fill up their fresh international free classifieds directory database with ads from ОLХ.com. My clients’ service is a non-commercial venture, so they’ve reassured me that this would not violate the ОLХ’ TOS, besides I considered it a nice challenging task. I chose Perl for this project, partly because I don’t get to program using Perl too often, and due to that it was originally designed for easier text manipulation and information extraction. And that’s what I usually use it for.

The scraper goes through all the local ОLХ versions, routinely grabbing data from 100+ countries. After the first iteration, the database contained ~1.5 million ads with phone numbers, which were not older than 45 days.

Product Details
Name: ОLХ scraper
Programming languages used: Perl
3d-party libraries used: TreeBuilder
Type: standalone
Budget: 40 hours
Release: November 2012.

Discuss