The work
- Develop and optimise batch and incremental pipelines that collect, process and load data into warehouses and marts.
- Design and support relational storage and, where needed, NoSQL storage.
- Work with architects, analysts and business teams to agree requirements, models, service levels and quality criteria.
- Check data quality and consistency, including reconciliation, completeness, duplicates and anomalies.
- Diagnose data incidents, optimise performance and automate routine work.
- Document pipelines, tables and interfaces and take part in code review.
Role-specific skills
- SQL experience with databases such as PostgreSQL, MS SQL Server, MySQL or Oracle.
- Understanding of data warehouse and data mart design and operation.
- Basic understanding and practical familiarity with NoSQL databases.
- Dimensional modelling and star/snowflake schemas; SCD and historisation are a plus.
- Basic understanding of data architecture and performance techniques, including indexes, partitioning and incremental loads.
- Confident use of Python, preferably, or Scala/Java.
- Git experience.
- Experience with Jenkins, GitLab CI or similar tools.
- Familiarity with infrastructure as code using Terraform or Ansible.
- Experience with automated workflows, including schedules, monitoring, retries, idempotency and logging.
Commercial experience
We consider Junior+, Middle and Senior specialists. Each field requires at least six months of relevant commercial experience. Courses and personal projects can add to it.
Courses and personal projects do not replace commercial experience.
Experience using AI tools in commercial work is a plus, with results checked through tests and review and confidentiality respected.
Tell us about your experience
Describe a pipeline you developed: its sources, storage, retries and quality checks. Include your programming language and orchestration tools.
Share your experienceWe connect with specialists in this field for future projects. This page is a role profile, not an open vacancy.