Data Index Profiles

Data Index Profiles


Job Description

Project Description:
We have a big project with many parts we would like to outsource as soon as possible. We would like to find a partner who would be available for multiple similar one-off projects such as this. We have an urgent need to scrape or index company information from a handful of sites for use in a directory-like application. We have 10+ target web sites in mind that we will share through private discussions.

Basically, the steps are as follows:
+ Gather data from target website (name, description, logos, links and other fields for people, companies, organizations)
+ Rearrange fields in a CSV or other to map to fields already defined by us in our drupal CMS/SQL

These target sites range from 150,000 to tens of millions in terms of number of records/profiles.
Interested in how many hours it would take to scrape 1 Million records for a website, clean and ready for import.

There will likely be ongoing work. We are breaking this down into multiple "simple projects", and open to discussions on approaches, timing, costs.

Skills required:
Drupal, Javascript, PHP, Web Scraping
Data Entry, Data Mining, Excel, Web Scraping, Web Search