Sr. Data Engineer

  • Pune
  • Swiftwin |a Concord Company

Job Description:

As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines and ETL processes using Azure Data Factory and Databricks. You will work closely with data scientists, analysts, and other stakeholders to ensure data is accessible, reliable, and optimized for analysis.

Key Responsibilities:

  • Design and implement scalable data pipelines using Azure Data Factory.
  • Develop and maintain ETL processes to extract, transform, and load data from various sources.
  • Utilize Databricks for big data processing and analytics.
  • Collaborate with cross-functional teams to understand data requirements and deliver solutions.
  • Monitor and optimize data workflows for performance and reliability.
  • Ensure data quality and integrity throughout the data lifecycle.
  • Document processes, architectures, and workflows.

Required Skills:

  • Strong experience in data engineering concepts and practices.
  • Proficiency in Azure Data Factory for building ETL pipelines.
  • Experience with Databricks for big data processing.
  • Knowledge of SQL and experience with relational databases (e.G., SQL Server, SSIS, Azure SQL Database).
  • Familiarity with cloud platforms (Azure preferred).
  • Understanding of data modeling concepts and best practices.
  • Ability to work collaboratively in a team environment.