I need a few scripts built to scrape 2 websites.
The scraper is to get pieces of information and store that information in a database for data analysis.
There will be less than 10 pieces of information from each website that will need to be stored. Additiaonlly, the script will need to track which pages have been scraped already on the sites as new sites are built throughout the day.
The scraper will need to regularly review each site and scrape any new pages. New pages are easy to find via links on the main page.
Scripts must be written in python and data stored in mysql. I'd prefer BeautifulSoup or Scrapy but am open to any recommendations on methods.
When you apply, please provide me with samples of previous scraping projects.