We have a big project with many parts we would like to outsource as soon as possible. We would like to find a partner who would be available for multiple similar one-off projects such as this. We have an urgent need to scrape or index company information from a handful of sites for use in a directory-like application. We have 10+ target web sites in mind that we will share through private discussions.
Basically, the steps are as follows:
+ Gather data from target website (name, description, logos, links and other fields for people, companies, organizations)
+ Rearrange fields in a CSV or other to map to fields already defined by us in our drupal CMS/SQL
These target sites range from 150,000 to tens of millions in terms of number of records/profiles.
Interested in how many hours it would take to scrape 1 Million records for a website, clean and ready for import.
There will likely be ongoing work. We are breaking this down into multiple "simple projects", and open to discussions on approaches, timing, costs.
Data Entry, Data Mining, Excel, Web Scraping, Web Search