Our work aims to improve safety and performance in the oil and gas industry through risk analysis. A major part of this involves creation and curation of our proprietary data store. On any given day, our data engineers work to understand the details of what new public and private data sources mean, build the software processes that scrape this data, and suggest creative ways in which that data can be used.
Should have worked on ETL Processes,
Data imports on databases,
Data Scrapping Web Data Extraction,
Crawler and Scraping skills
Familiar with Scrapy Framework
Familiar with GitLab,
Experience with PostgreSQL
DB design, DB architecture, someone who can and has designed DB from scratch – should be able to write schemas and tables etc.
- Research meaning of data fields, files, and webpages associated with external public & private data sources identified as relevant to Suventure products.
- Create Python scrapers to programmatically fetch relevant data from external data sources, parse data, and store in database
- Determine plans of action of regular data updates and sequence of operations for scraping linked data sources
- Implement above plans
- Document oddities, edge cases, and other interesting qualities of data sources as scrapers are built and tested
Interested candidates can send their updated resume to email@example.com