Data Engineer

Corsearch | Amsterdam

Date listed

1 week ago

Employment Type

Full time

Employees

251-500

Total Funding

$135.9 billion

Keywords: python aws etl sql

Responsibilities

  • Data Pipelines development for online and batch processing from hundreds of data sources using the unified ETL framework;
  • ETL framework design and development for online and batch processing;
  • Optimization of ETL: unification, decrease of development TTM, increase of code readability;
  • Perform code reviews;
  • Physical and logical database design;
  • System/components/modules documentation;
  • Develop data feeds for external data consumers;
  • Adoption of the new technologies, used to improve the performance of the data platform, indexing engines, APIs;

Requirements

  • 5+ years in enterprise data platforms implementation projects;
  • Strong experience with proprietary and open-source ETL engines (AbInitio, Informatica, SAP DataServices, Pentaho, Talend, Airflow, etc);
  • Exceptional SQL knowledge and optimization skills;
  • Strong experience working with cloud-based solutions (AWS preferably);
  • Strong, proven experience with performance database/ETL optimization;
  • Strong scripting languages experience, preferably Python;
  • Web scraping technologies experience is a plus;
  • Self-starter capability, ability to acquire new skills autonomously;
  • Demonstrably solid written and verbal communication skills to drive projects to successful conclusion.
  • Ability to troubleshoot and solve complex problems independently.
  • Ability to manage conflicting priorities effectively.