Data Engineer - Databricks

  • Pune
  • Newvision Software

Roles and Responsibilities mentioned below:

Job Description:

We are seeking a skilled Databricks Developer to join our dynamic team. The ideal candidate will have a strong background in Databricks programming with extensive experience on Databricks platform.

Responsibilities:

Collaborate with cross-functional teams to design, develop, and deploy Databricks solutions tailored to business requirements.

Implement and optimize ETL processes using Databricks to ensure efficient data processing and integration.

Troubleshoot and resolve issues related to Databricks workflows.

Design and implement data pipelines, ensuring scalability and reliability.

Collaborate with data scientists and analysts to understand data requirements and provide technical support.

Stay updated on industry best practices and emerging technologies to continuously enhance development processes.

Requirements:

Hands-on experience with Databricks, including building and optimizing Spark jobs.

Solid understanding of data processing concepts and ETL workflows.

Familiarity with cloud platforms like Azure and their integration with Databricks.

Strong problem-solving skills and ability to work in a fast-paced environment.

Excellent communication and collaboration skills to work effectively within a team.

Bachelor's degree in Computer Science, Information Technology, or related field.

Preferred Qualifications:

Experience with big data technologies such as Hadoop and Hive.

Knowledge of machine learning concepts and frameworks.