-
What you'll do:
- Keep DWH infrastructure healthy and evolve system architecture;
- Develop and refactor the ETL service (Airflow / Python);
- Write tests (pytest);
- Support legacy ETL pipelines (Logstash);
- Add/change data deliveries from Kafka and third-party service APIs;
- Create and change dbt models for aggregates and views;
- Take part in initiatives to reduce DWH cost (storage, compute, pipelines, and models);
- Participate in reviews and team meetings; collaborate with other engineering teams;
- Keep technical documentation and Wiki up to date.
-
What we expect:
- 3+ years of Data Engineer experience;
- Knowledge of Python 3.9+, asyncio, pytest;
- Experience writing SQL queries;
- Ability to work with Git version control;
- Knowledge of Docker, GitLab CI, Unix CLI;
- Knowledge of and hands-on experience with Kafka;
- Ability to find and solve problems;
- Experience with dbt and ClickHouse (or similar OLAP) is preferred;
- Experience optimizing DWH cost: reducing storage and compute via TTL/partitioning, incremental ETL, SQL/OLAP model optimization;
- Ability to measure and explain impact in terms of money and resources.
Nice to have: - Data lifecycle: hot/cold storage, archival, retention policies;
- Experience migrating and simplifying legacy ETL into a cheaper, more manageable setup;
- DBT: incremental models, volume-growth tests, control of expensive aggregates.
-
We offer:
- Work in the international company;
- Hybrid working format (office/home office);
- Corporate education – courses and trainings;
- Voluntary health insurance after probation period;
- Effective onboarding program for a better start;
- Corporate events and team buildings.