Lead

  • Pune
  • Birlasoft

Title: Azure Lead - Data Engineering

Responsibilities:

  • Lead a team of data engineers in designing, developing, and implementing end-to-end data solutions on the Azure platform, with a focus on Azure Data Factory, Azure Synapse, Azure Databricks, and related technologies.
  • Architect scalable and robust data pipelines and ETL workflows using Azure Data Factory, ensuring efficient data ingestion, transformation, and loading processes.
  • Design and implement data warehouse solutions in Azure Synapse Analytics, incorporating best practices for data modeling, partitioning, and indexing.
  • Develop and optimize Spark SQL scripts and queries in Azure Databricks for advanced data transformations, analytics, and machine learning workloads.
  • Establish and manage data lake solutions using Azure Blob Storage and Azure Delta Lake Storage, ensuring data governance, security, and compliance requirements are met.
  • Oversee the implementation of Azure SQL Server Database solutions for relational data storage, retrieval, and analysis, optimizing performance and scalability.
  • Provide technical leadership and mentorship to team members, fostering a culture of innovation, collaboration, and continuous learning.
  • Collaborate with cross-functional teams, including data architects, data scientists, and business stakeholders, to understand data requirements and deliver solutions that meet business objectives.
  • Define and enforce coding standards, development methodologies, and best practices for Azure data engineering projects.
  • Stay updated on emerging Azure data technologies, trends, and best practices, and provide recommendations for technology adoption and process improvements.

Qualifications:

  • Bachelor's degree in Computer Science, Information Technology, or related field; advanced degree preferred.
  • Proven experience as a technical lead or architect in Azure data engineering projects, with expertise in Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Spark SQL, Azure Blob Storage, Azure Delta Lake Storage, and Azure SQL Server Database.
  • Strong hands-on experience in designing and implementing scalable and performant data solutions on the Azure platform, with a focus on data integration, warehousing, and analytics.
  • Expertise in data modeling, schema design, and optimization techniques for both relational and non-relational databases.
  • Proficiency in SQL, T-SQL, and scripting languages for data manipulation, automation, and orchestration.
  • Excellent leadership, communication, and collaboration skills, with the ability to lead and mentor a team of data engineers and work effectively with cross-functional teams.
  • Experience with Agile methodologies and DevOps practices for software development and deployment.
  • Relevant certifications in Azure data services (e.g., Microsoft Certified: Azure Data Engineer Expert) and leadership (e.g., Azure Solutions Architect Expert) are highly desirable.

This job description outlines the leadership responsibilities and technical expertise required for an Azure Lead role focused on data engineering with hands-on experience in Azure Data Factory, Azure Synapse, Azure Databricks, and related Azure data services.