symbol

PySpark Developer

APLIKUJ

Tasks

  • Developing Big Data and non-Big Data cloud-based enterprise solutions in Spark, Python and related frameworks/libraries
  • Developing scalable and re-usable, self-service frameworks for data ingestion and processing
  • Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of data
  • Autonomously working cross-projects bringing seniority in whole organisation
  • Processing performance analysis and optimization
  • Bringing best practices in following areas:
    – Design & Analysis
    – Automation (Pipelining, IaC)
    – Testing
    – Monitoring
    – Documentation

Requirements

  • Experience in building data pipelines and analysis tools using Python, PySpark, Scala
  • Knowledge of Big Data, data engineering, databases, data wrangling, relational databases
  • Proficiency with Big Data processing technologies (Hadoop, Spark, Azure, AWS)
  • Good knowledge about Artificial Intelligence and Machine Learning
  • Experience in building ETL/ Data Warehouse transformation process
  • Very good knowledge of SQL
  • Working with structured and unstructured data
  • Experience in working with Agile (preferably Scrum)
  • Hands-on experience in cloud-based solutions
  • Knowledge of data management principles
  • Strong communication skills and self-management
  • Very good knowledge of English
  • Nice to have experience of building data pipelines using Azure Data Factory, Azure Databricks or Azure Synapse

Offer

  • Remote work possibility
  • Innovative and challenging projects
  • Freedom of action in the field entrusted
  • Openness to your technical and managerial ideas
  • Leading a team of involved IT professionals
  • Experience in international projects in the scrum methodology
  • Contributing to the rapidly expanding company
  • Selection of forms of cooperation and benefits
  • Friendly atmosphere
  • International work environment standards

Benefits

  • Private health care (LuxMed)
  • Flexible working hours
  • Non-corporate atmosphere

na skróty

kontakt: wspolpraca@pytechsummit.pl