Data Engineer (m/w/d) | 06602
Data Engineer (m/w/d)
For one of our clients in the area of Berlin we are looking for a Data Engineer (m/w/d) on a freelance basis.
- Design, build and continuously improve the data platform to enable other teams to make data-driven decisions, automate their data workflows and increase reliability of their data.
- Work closely with your team, the data chapter leads and the CTO to plan and build sustainable technical solutions that address business data needs.
- Build and maintain our event and real time data pipelines and overall data architecture.
- Some examples of current challenges are to revamp our data warehouse schema versioning and deployment, integrating Segment schema validation to our real time pipeline, provide a data quality stack for all other chapters that covers everything from raw data to anomaly detection
- Must-have Technology Stack:
BigQuery, Airflow, Python, Kafka, Scala (in this order, BQ, Airflow, Python is must have, Kafka, Scala not necessarily, but then it would be a shorter engagement, because we only have a few acute tasks, so think 3-4 weeks)
- must have already solved all or some of the projects we are working on in data engineering at least once in your career: versioning and deployment of (BigQuery) data warehouse schema (e.g. using dataform or custom stack), real time replication from MySQL via Kafka to BigQuery (debezium or custom build), unit testing of SQL deployments/views
- Additional info: Ideally someone that would be open for a permanent position, but not a must
- Languages: German/English
Duration: 2-4 months
If you are interested, we would be pleased to receive your skill profile with details of hourly rates and availability.
If it does not suit you at the moment, we will be happy to make a recommendation.
Contact person: Marvin Wiegert