2 weeks ago

Data Processing Pipeline Engineer - COVID Infection

Newport 180 - 185 Contract

Job Reference: BBBH36721_1619621139

The COVID19 Infection Survey (CIS) provides statistical analysis on the COVID19 pandemic for Government and Research purposes. The Data Processing Pipeline (DPP) receives data from multiple sources, processes this data in variety of ways, then provides the cleaned / linked data to several Analysis teams for their investigation.

My client are seeking a developer who will be responsible for the technical development and implementation of the processing pipelines specified and designed in conjunction with the COVID analysis teams. You will be working with a small coding team and a wider analysis team to identify the best ways of ingesting and processing the data received.

Relevant Skills and Experience

  • Data engineering - functional programming in Python (including PySpark), multicore/distributed processing (hadoop), relational and non-relational (Hive/Impala) SQL database systems
  • Data engineering - development of ETL routines for slowly changing dimensions
  • Testing - writing unit tests for PySpark routines using pytest
  • Agile - applying Agile software development practices
  • DevOps - applying DevOps practices, including continuous integration and continuous deployment

This is a full time contract position that will initially run until the end of August. The role will begin as a remote position, but some onsite presence in Newport, South Wales is expected as restrictions lift.

The role is paying £24.30 per hour - which is equivalent to a salary of £36,027.

Share this job:

Apply now

 



Similar Jobs

2 days ago

Strategy Development Lead

Fareham 450 - 550 Contract

My client are a large government organisation based in Titchfield, Hampshire. They are seeking a Str...

1 week ago

Statistical Production Analyst

Newport 15 - 15 Contract

My client are a prestigious government organisation based in Newport, South Wales. They're seeking a...