Jobs Data Engineer Spark

Role title: Data Scientist Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and build a more sustainable, more inclusive world. Location: Colombia RESPONSIBILITIES Develop and maintain efficient ETL pipelines using Azure Data Factory and Databricks to process and transform large datasets for analysis. Analyze large datasets and build machine learning models using Python and Spark to extract actionable insights and solve business problems. Utilize Apache Spark to process and analyze large volumes of data in a distributed environment, ensuring scalability and performance. Deploy machine learning models to production using Azure and Databricks, and monitor their performance for continuous improvement. Work closely with cross-functional teams (data engineers, business analysts, etc.) to understand requirements and communicate insights and results effectively to stakeholders. Requirements Advanced English skills (B2+) Experience in ETL and Azure tools: A minimum of 2-5 years of experience in developing ETL processes using Azure Data Factory, Azure Databricks, and other Azure cloud tools. Knowledge of Apache Spark and Databricks: Experience using Apache Spark for distributed data processing and knowledge of Databricks for data integration and transformation. Proficiency in Python and SQL: Advanced skills in Python for automating processes and manipulating data, along with a strong understanding of SQL for querying and transforming data. Handling large volumes of data: Ability to manage, transform, and optimize large volumes of data using cloud platforms and processing technologies like Spark. Teamwork and communication: Collaborative skills with multidisciplinary teams and effective communication ability to interact with both technical and non-technical stakeholders. WHAT YOU'LL LOVE ABOUT WORKING HERE We recognize the significance of flexible work arrangements to provide support. Be it remote work, or flexible work hours, you will get an environment to maintain healthy work life balance. At the heart of our mission is your career growth. Out array of career growth programs and diverse professions are crafted to support you in exploring a world of opportunities. Equip yourself with valuable certification in the latest technologies.