Senior Data Engineer (f/m/d)

Axpo

  • Comunidad de Madrid
  • Permanente
  • Tiempo completo
  • Hace 1 mes
Workload: 100%Join us as a Senior Data Engineer! Lead resilient event-driven processing with cutting-edge event queues. Manage data operations with IaC-based cloud resource management. Optimize "Lakehouse" for structured and unstructured data. Develop efficient ETL/ELT pipelines in SQL and OOP. Implement optimal data models, handle big data, and generate insightful reports. Dive into machine learning, IoT integration, and seamless release processes. Analyze end-to-end solutions for efficiency and excellence. Join our innovative team today!What you will do:
  • Ensure resilient event-driven processing with event queues.
  • Lead IaC-based cloud resource management for data operations.
  • Manage structured and unstructured data with "Data Warehouse" and "Data Lake”: Lakehouse
  • Develop efficient ETL/ELT pipelines in SQL and OOP languages.
  • Implement optimal data models for performance and maintenance.
  • Efficiently handle big data processing within systems.
  • Generate insightful reports for focused data analysis.
  • Conduct automated testing and data quality validation
  • Engage in machine learning for deeper insights.
  • Establish seamless release processes, and build DevOps Pipelines
  • Explore IoT integration into our data ecosystem.
  • Participate in systems integration: consumption and development.
  • Work in agile-methodology team setting
  • Analyze holistic end-to-end solutions for efficiency.
What you bring & who you are:
  • 8+ years of expert data solution crafting experience.
  • Proficiency in Python, pySpark/pandas, and Azure Eventhubs (Kafka).
  • Skilled in SQL, T-SQL, or Databricks (Better Spark SQL), SQL.
  • Experience with APIs, and precise timing functions (Azure Functions preferable, but not mandatory).
  • Azure coding and resource management via Portal/Cli via (Azure CLI & REST API) and Terraform.
  • Knowledge in Kimball and Inmon; Data Vault 2.0 is an advantage
  • Proficient in Lakehouse & Databricks, ELT/ETL techniques
  • Previous experience with Power BI
  • Data Governance and Datamesh
  • Familiarity with Azure Dev-Ops, as well as Hadoop, ML, IoT, and Agile methodologies.
  • Knowledge of Automated Functional testing and data quality testing technical frameworks
  • Fluency in English, German is considered an advantage
About the team:Our team consists of passionate professionals committed to innovation and excellence in the data realm. We value collaboration, continuous learning, and leveraging our diverse skills to meet the challenges of an ever-evolving data landscape. The goal of our team is to build a platform for all data-related roles in the company (data scientist, managers, data analysts, data engineers) under the same simple rules.If you're an enthusiastic individual looking to make an impact, we're excited to have you on board.Axpo has the ambition to enable society to have a sustainable future with innovative energy solutions. Axpo is Switzerland's largest producer of renewable energy and an international pioneer in energy trading and in the marketing of solar and wind power. More than 6,000 employees combine experience and know-how with a passion for innovation and the joint search for ever better solutions. Axpo relies on innovative technologies to meet the ever-changing needs of its customers in over 30 countries in Europe, North America and Asia.

Axpo