Descrição do trabalho
- Job Title: Data Engineer
- Design and develop scalable data pipelines to integrate, transform, and process large datasets.
- Build, maintain, and optimize robust data processing infrastructure to support analysts and data scientists with high-quality data.
- Develop analytics capabilities using cloud services like Azure Data Factory, Databricks, and Datalake Storage.
- Implement dimensional data models and maintain data warehousing solutions for reporting and analytical workloads.
- Write efficient, well-structured, and tested code in Python and SQL for large-scale data operations.
- Optimize Py Spark code and SQL queries based on Apache Spark internals for high performance.
- Collaborate with cross-functional teams to understand requirements and deliver data-driven solutions.
- Ensure data governance, monitor data quality, and implement best practices in data management and engineering.
- Support the deployment and monitoring of machine learning pipelines and data solutions in production environments.
- Requirements:
- Bachelor's degree in Computer Science or a related field.
- 4+ years of hands-on experience in software or data engineering.
- Solid grounding in software engineering best practices like OOP, TDD, CI/CD, and version control (Git).
- Proficiency in Python for data processing and automation tasks.
- Strong hands-on experience with Py Spark, including optimizing queries and understanding Spark internals.
- Expertise in Azure services like Databricks, Data Factory, Dev Ops, and Datalake Storage.
- Proven experience in dimensional data modeling and data warehousing principles.
- Familiarity with Kubernetes and Docker for containerization of data applications.
- Experience with Google Analytics data is a plus.
- What We Offer:
- Health Insurance.
- Feedback&Coffee system.
- Remote working model.
- A Training Academy for technical and behavioral training sessions.
- About Us:
- We value transparency, team spirit, regular feedback, and continuous learning.
- We offer reference prizes and birthday presents.
- We have no dress code.
- We know your name and use informal addresses.
