Python Developer Needed to Scrape Data from a Website

Profile

Python Developer Needed to Scrape Data from a Website
Python Developer Needed to Scrape Data from a Website
We are looking for an experienced Python developer for web scraping.
The development environment is CentOS / Linux and the database is PostgreSQL.
The job entails the following:
Scraping data from website; inserting data into PostgreSQL database; writing an updating script that grabs periodic updates from the source; writing an export tool to export data from the database to a CSV file; testing, documenting, and deploying the solution to production. Youll use Sphinx to generate documentation and use our internal Git server for version control.
The code should perform three main functions: 1) fetch entire data history and populate PostgreSQL database with entire history data and any relevant metadata; 2) perform ongoing updates to the database as new data are released by the source; 3) exporting data from PostgreSQL database to an CSV file in the desired format. We will QA the CSV file using our in-house tools to verify data have been gathered correctly.
The ideal candidate will have:
Expert-level experience with Python
Expert level experience with SQL databases
Experience with git
If you find this opportunity intriguing and interesting, please send us your answers to the following questions:
Which types of data have you scraped in the past?
How many years of experience do you have with webscraping?
How many years of experience do you have with SQL databases?
Which text editor or IDE do you use for coding?
If this job goes well, we will have several other jobs for you. We are ideally looking for someone we can work with long-term.
Skills:
Python
, Web Scraping
, MySQL
, Selenium
, ETL

Company info

Sign Up Now - WashingtondcCrossing.com