Data Engineer (Modern BI & Big Data)

Descrição do trabalho

We are currently looking for a Data Engineer (Modern BI & Big Data) to join one of our leading client teams. If you thrive at building robust data pipelines and enjoy turning raw data into actionable insights, this is your stage.

  • Main Responsibilities:
  • Design, build, and maintain secure and efficient data pipelines using technologies like Azure Data Factory, Azure Synapse, Microsoft Fabric, Databricks, and Azure SQL.
  • Implement and optimize ETL and ELT processes for data transformation and processing.
  • Develop and refine complex PySpark and SQL scripts for data ingestion, transformation, and modeling.
  • Collaborate closely with data scientists and engineers to ensure data quality, integrity, and scalability.
  • Design and implement BI modeling solutions to support data-driven decision-making.
  • Monitor and enhance the performance of data pipelines, ensuring scalability and security.
  • Follow best practices for each technology stack while maintaining a critical eye for continuous improvement.
  • Contribute to internal knowledge sharing and documentation to support team development.
  • Key Requirements:
  • Bachelor’s degree in IT, Data Science, or equivalent practical experience.
  • 3+ years of hands-on experience as a Data Engineer working with modern BI and big data technologies.
  • Strong experience with Azure Data Factory, Azure Synapse, Microsoft Fabric, Databricks, and Azure SQL.
  • Advanced PySpark and SQL skills, including data modeling, functions, and stored procedures.
  • Familiarity with modern data architectures, including Lakehouses and cloud-based data systems.
  • Experience with performance optimization and data quality assurance for ETL and ELT processes.
  • Proficiency in English (written and spoken).
  • Nice to Have:
  • Knowledge of additional technologies such as Azure Logic Apps, Functions, Key Vault, and Log Analytics.
  • Experience with Power BI (semantic modeling and DAX).
  • Knowledge of Snowflake, DBT, Airflow, or other cloud providers.

Other Details:

This is a hybrid position. If you're based in Lisbon or Porto, you'll work from local office in your city. If you're based elsewhere, the role is remote with occasional visits to the office when necessary.