QA TA Lead

Total Experience-3-6Yrs

Location-Gurgaon

Role and Responsibilities Overview:

• Develop and implement a comprehensive quality assurance strategy for data engineering projects, ensuring that data quality standards are met throughout the data pipeline.
• Collaborate with cross-functional teams to define test plans and strategies for validating data pipelines, ETL processes, and transformations using PySpark, Databricks, and SQL.
• Design and execute tests using SQL queries and data profiling techniques to validate the accuracy, completeness, and integrity of data stored in various data repositories, including data lakes and databases.
• Adhere to formal QA processes, ensuring that the Systems Implementation (SI) team is using industry-accepted best Practices.
• Ensure data accuracy in both SQL and the Data Lakehouse platform. Use detailed and effective written communication skills to document the features tested and bugs found.
• Conduct performance testing of data processing jobs and workflows to identify bottlenecks, optimize code, and improve overall system performance using PySpark performance tuning techniques.
• Act as a key point of contact for all QA aspects of releases, providing test services and coordinating QA resources internally and externally in SQL, PySpark/Python, Databricks Delta, and Delta Live Tables.
• Identify the critical defects and work with BI and Data Engineering teams to mitigate them; define and enforce the acceptance criteria for testing, which inherently determines the approval for code promotions to higher environments.

Candidate Profile

• Bachelor’s/Master's degree in computer science/engineering, operations research, or related data and analytics areas; candidates with BA/BS degrees in the same fields from the top tier academic institutions are also welcome to apply

Proficiency in Data Engineering, Data Management (ETL) solutions with capabilities with HIVE (preferred)

• In-depth understanding of PySpark, Python, SQL, and Databricks Delta and Delta Live Tables with Unity Catalog (preferred)
• Insurance Industry knowledge preferred.
• QA best practices and methodologies to design, implement, and automate processes.
• Knowledge on SQL & PySpark (Mandatory)
• Experience extracting and manipulating data from relational databases with advanced SQL, HIVE, Python/PySpark.
• Proficiency in design and test bed creation to support various test cases and accounts for variety and volume of test data.
• Test strategy development experience and demonstrate knowledge in the key areas of managing timelines and dependencies across multiple tests in an agile environment.
• The entire software development life cycle and test cycles (unit, Regression, Functional, Systems, stress& scale, smoke and sanity)
• Strong knowledge of design principles, color proposition, and typography.
• 3-6yrs years experience in data engineering, preferably in data lake/data Lakehouse design-based platforms
• Outstanding written and verbal communication skills
• Able to work in a fast-paced, continuously evolving environment and ready to take uphill challenges.
• Able to understand cross-cultural differences and can work with clients across the globe.
• Self-motivated, works well independently and with others.
• Using metrics-driven approach and closed-loop feedback to improve software deliverables and improve the predictability and reliability of releases.