Senior Data Engineer (f/m/d)
Axpo
- Comunidad de Madrid
- Permanente
- Tiempo completo
- Ensure resilient event-driven processing with event queues.
- Lead IaC-based cloud resource management for data operations.
- Manage structured and unstructured data with "Data Warehouse" and "Data Lake”: Lakehouse
- Develop efficient ETL/ELT pipelines in SQL and OOP languages.
- Implement optimal data models for performance and maintenance.
- Efficiently handle big data processing within systems.
- Generate insightful reports for focused data analysis.
- Conduct automated testing and data quality validation
- Engage in machine learning for deeper insights.
- Establish seamless release processes, and build DevOps Pipelines
- Explore IoT integration into our data ecosystem.
- Participate in systems integration: consumption and development.
- Work in agile-methodology team setting
- Analyze holistic end-to-end solutions for efficiency.
- 8+ years of expert data solution crafting experience.
- Proficiency in Python, pySpark/pandas, and Azure Eventhubs (Kafka).
- Skilled in SQL, T-SQL, or Databricks (Better Spark SQL), SQL.
- Experience with APIs, and precise timing functions (Azure Functions preferable, but not mandatory).
- Azure coding and resource management via Portal/Cli via (Azure CLI & REST API) and Terraform.
- Knowledge in Kimball and Inmon; Data Vault 2.0 is an advantage
- Proficient in Lakehouse & Databricks, ELT/ETL techniques
- Previous experience with Power BI
- Data Governance and Datamesh
- Familiarity with Azure Dev-Ops, as well as Hadoop, ML, IoT, and Agile methodologies.
- Knowledge of Automated Functional testing and data quality testing technical frameworks
- Fluency in English, German is considered an advantage