.. dynamic person with 2-3years years of experience in Customer Service / Inside Sales. The candidate would be expected to understand and assign the Salesforce cases to the Customer Service advocates and responding to the customer in needed. SFDC Case Handling in Categorization SAP SD - Creating/modifying .. read more
Primary Skill – Scala Spark
Total Exp – 5.2 Years to 13 Years
Notice Period – ONLY 0 to 60 Days Joiners
Job Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, Delhi
Job Description:
Job Description:
JD: Spark + scala Tech Lead
Job Description
We are looking for a Spark Scala Tech Lead with 5+ Years experience who knows how to fully exploit the potential of Spark cluster. You will be responsible to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data to feature developers and business analysts.
This involves both ad-hoc requests as well as data pipelines that are embedded in production environment.
Responsibilities
• Create Scala/Spark jobs for data transformation and aggregation
• Produce unit tests for Spark transformations and helper methods
• Write Scaladoc-style documentation with all code
• Design data processing pipelines
What you’ll be doing:
• Building distributed and highly parallelized Big data processing pipeline which process massive amount of data (both structured and unstructured data) in near real-time
• Leveraging Spark to enrich and transform corporate data to enable searching, data visualization, and advanced analytics
• Working closely with analysts and business stakeholders to develop analytics models
• Continuous delivering on Hadoop and other Big Data Platforms
• Automating? processes where possible and are repeatable and reliable
• Working closely with QA team
As an expert Data Engineer, you are expected to have:
• Excellent Programming language skills: Scala, Python, Bash, SQL, Bash
• Excellent understanding of Hadoop and HDFS.
• Excellent understanding of file types, their pros and cons
• Excellent understanding of Build Tools especially SBT/Gradle
• Very good understanding and good experience in implementing CICD e.g. Building and Maintaining Jenkins Pipelines
• Working experience in SAFe/Agile
• As an expert, you are also responsible for Juniors writing maintainable code
• Are collaborative and achieve your expectations through communication and teamwork
• Are curious, responsive and can understand the needs of others to ensure delivery of the desired results
• Work qualitatively and strive to always do things better
Skills
• Spark + Scala Scala (with a focus on the programming )
• Apache Spark 2.x
• Spark query tuning and performance optimization
• Understanding on Hadoop Architecture
• Experience working with Hive, SQL and HDFS
• Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)
• Good to have experience in Scala
• Experience working with tools like Jenkins, Jira, Bitbucket and Git
• Experience in writing shell scripts and working with Linux platforms.
As an advantage, familiarity with one or more of the below technologies:
• Experience? working in Agile/SAFe
• Experience/certification with AWS Cloud