Databricks Architect

  • Pune
  • Tredence Inc.

We are seeking an experienced professional who apart from the required mathematical and statistical expertise also possesses the natural curiosity and creative mind to ask questions, connect the dots, and uncover opportunities that lie hidden with the ultimate goal of realizing the data’s full potential.


Roles and Responsibilities:

  • Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack
  • Ability to provide solutions that are forward-thinking in data engineering and analytics space
  • Collaboration with DW/BI leads to understanding new ETL pipeline development requirements.
  • Triage issues to find gaps in existing pipelines and fix the issues.
  • Work with business to understand the need in reporting layer and develop data model to fulfill reporting needs.
  • Help joiner team members to resolve issues and technical challenges.
  • Drive technical discussion with client architect and team members.
  • Orchestrate the data pipelines in scheduler via Airflow.
  • Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack
  • Ability to provide solutions that are forward-thinking in data engineering and analytics space.
  • Collaboration with DW/BI leads to understanding new ETL pipeline development requirements.
  • Triage issues to find gaps in existing pipelines and fix the issues.
  • Work with business to understand the need in reporting layer and develop data model to fulfill reporting needs.
  • Help joiner team members to resolve issues and technical challenges.
  • Drive technical discussion with client architect and team members.
  • Orchestrate the data pipelines in scheduler via Airflow.


Skills and Qualifications:

  • Bachelor's and/or master’s degree in computer science or equivalent experience.
  • Must have total 6+ yrs. of IT experience and 3+ years' experience in Data warehouse/ETL projects.
  • Deep understanding of Star and Snowflake dimensional modelling.
  • Strong knowledge of Data Management principles
  • Good understanding of Databricks Data & AI platform and Databricks Delta Lake Architecture
  • Should have hands-on experience in SQL, Python and Spark (PySpark)
  • Candidate must have experience in AWS/ Azure stack
  • Desirable to have ETL with batch and streaming (Kinesis).
  • Experience in building ETL / data warehouse transformation processes
  • Experience with Apache Kafka for use with streaming data / event-based data
  • Experience with other Open-Source big data products Hadoop (incl. Hive, Pig, Impala)
  • Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J)
  • Experience working with structured and unstructured data including imaging & geospatial data.
  • Experience working in a Dev/Ops environment with tools such as Terraform, CircleCI, GIT.
  • Proficiency in RDBMS, complex SQL, PL/SQL, Unix Shell Scripting, performance tuning and troubleshoot
  • Databricks Certified Data Engineer Associate/Professional Certification (Desirable).
  • Should have experience working in Agile methodology.


Job Description:

  • 10+ Years of experience in Data & Analytics with Good Communication and presentations skills.
  • At least 2 years' experience in Databricks implementations, 2 large scale data warehouse end-to- end implementation experience.
  • Must have Databricks certified architect.
  • Proficiency in SQL and experience with scripting languages (e.g., Python, spark, Pyspark) for data manipulation and automation.
  • Solid understanding of cloud platforms (AWS, Azure, GCP) and their integration with Databricks.
  • Familiarity with data governance and data management practices. exposure to Data sharing, unity catalog, DBT, replication tools, performance tuning will be added advantage / must have skills.


About Tredence:


Tredence focuses on last-mile delivery of powerful insights into profitable actions by uniting its strengths in business analytics, data science and software engineering. The largest companies across industries are engaging with us and deploying their prediction and optimization solutions at scale. Head quartered in the San Francisco Bay Area, we serve clients in the US, Canada, Europe, and Southeast Asia. Tredence is an equal opportunity employer. We celebrate and support diversity and are committed to creating an inclusive environment for all employees.

Visit our website for more details: https://www.tredence.com/

Apply now

Other job offers that may interest you

Marketing specialist
PuneDatamatoYesterday
.. development, to produce engaging content and optimize user experience Lead generation responsibility using Digital Marketing strategies and actions Acquire insight in online marketing trends and keep strategies up to date Maintain partnerships with media agencies , vendors and OEM for co-marketing Implement .. read more
Manager cmcra (domestic ra)
PuneEnzene Biosciences LtdToday
.. maintenance documents, PSURs, etc.). · Analysis of regulatory procedures and special designations used during development, authorizations and extension of the product. · Uses and shares best practices, when handling various applications and procedures during interactions with health authorities and .. read more
Senior software engineer (java, spring boot, cloud)
PuneRocket SoftwareToday
.. debugs, and sets the direction for the design and implementation of software features/functionality/enhancements, correction of defects, and/or the creation of new products. Develops software tools including operating systems, compilers, routers, networks, utilities, databases, and Internet-related tools .. read more
Delmia apriso solution developer
PuneTata TechnologiesYesterday
.. Delmia Apriso Developer for Pune location. If below JD matches to your profile, kindly share your profile on Sayali.Y**@tatatechnologies.com Job Title: Delmia Apriso Solution Developer Total Exp:3-5 Years NP: Immediate-30 Days Location: Pune (WFO) Roles & Responsibilities: Solution Development, following .. read more
Mendix tech lead / architect / sr. developer
PuneIntelizignToday
.. architectural guidelines, coding standards, and performance requirements. Requirements Tech Lead / Architect Minimum of 6 years of overall experience is required Minimum 1 year of experience as a Mendix Solution Architect or completion of one end-to-end project in Mendix. Sr. Dev Minimum of 3 years of .. read more
Manager engineering – machine learning
PunePubmaticYesterday
.. organizations provide, such as stock options, paternity/maternity leave, Healthcare insurance, broadband reimbursement. As well, when we’re back in the .. statistics, modelling, and machine learning to improve the efficiency of systems and relevance algorithms across our business application products Conduct .. read more
Oracle ebs procurement techno-functional consultant
PuneJade GlobalToday
.. Change Requests and Root Cause Analysis / Problems Troubleshoot technical and functional issues involving SQL, PL/SQL, data fixes and provide solutions through configurations, setup changes Implement enhancements and problem management Undertake functional and integration testing during patch release .. read more
Sap finance control consultant
PuneNcs GroupToday
.. enhancement opportunities to resolve system gaps. Identifies systems deficiencies and performance gaps. Support the current FICO business processes and design. Conduct SAP FICO process configuration. Update and maintain documentation. Analyze and support end-user authorization requirements. Support users .. read more
Oracle procure to pay
PuneJade GlobalYesterday
.. Change Requests and Root Cause Analysis / Problems Troubleshoot technical and functional issues involving SQL, PL/SQL, data fixes and provide solutions through configurations, setup changes Implement enhancements and problem management Undertake functional and integration testing during patch release .. read more
Cloud security engineer (multi-cloud)
PuneDew SoftwareYesterday
.. SaaS based applications, AWS services Should be having strong understanding of cloud related technologies Worked on IAC scanning, Terraform etc Proficiency with at least one scripting language (e.g., Perl, Python and PowerShell) Strong understanding on Cloud architecture, data flows, encryption & security .. read more
Entrepreneur in residence
PuneFridoYesterday
.. and enhance profitability. Qualifications: Bachelor's degree in business Administration, management, or a related field; MBA or advanced degree preferred. Possess .. engage with key stakeholders effectively. Proficiently manage administrative tasks including calendar coordination, appointment scheduling, and travel .. read more
System test engineer
Pimpri-ChinchwadLear CorporationToday
.. Possess 4 to 7 years of experience in embedded software testing – unit / integration test (white box / black box)/System test Interest in Automotive or transportation technologies. Experience in setting up embedded system and software test setup and automation Strong in C and assembly language programming .. read more
Dentist (mobile polyclinic van)
PuneHealth Within Reach FoundationToday
.. Reach Foundation is a non-profit organization focused on bringing quality Healthcare services to powerless and resource-less individuals. The foundation .. mobile Healthcare infrastructure and technology to bridge access barriers and deliver .. read more
Senior software engineer
PuneYash TechnologiesYesterday
.. like ERWin / ERStudio Proficient Data Profiling and Source System Analysis Proficient Data Governance (working with data owners, data stewards and data custodians in a federated governance model) Proficient Metadata Management including using a data catalogue (Collibra or any other data catalogue tool) .. read more
Ebs project lead
PuneBirlasoftYesterday
.. project and with customer teams v. Team player - able to work with team (supporting / guiding other members), vi. Consulting skills - business user issue handling, requirement gathering and its documentation (aware of AIMs process), vii. Knowledge of delivery and quality controls processes viii. Well .. read more
Senior data engineer
PuneGeminus TechToday
.. seamless communication between various SDV components. Expertise in SDV Data Processing Tools (Preferred Skills): In-depth knowledge of data processing tools relevant to SDV, including but not limited to Apache Flink, Apache Pulsar, Apache Kafka, and Apache Spark. Proven experience in making informed .. read more
Assistant manager Administration
Pimpri-ChinchwadCatenon IndiaYesterday
.. ¡tenon, a Global Executive Search Firm is looking for an Assistant Manager - Administration to join our Office in Baner, Pune. Key Responsibilities: Manage employee .. of pocket, rent etc. Ensure timely payments. All activities under office Administration - systematic documentation, storage, cleanliness etc Manager all vendor / .. read more
State credit manager- pune (secured business loans)
PuneHiranandani Financial ServicesToday
.. Regularly review and monitor the existing loan portfolio, identifying early warning signals and recommending necessary actions to mitigate risks. Stakeholder Communication: Effectively communicate credit decisions and recommendations to the sales team and other relevant stakeholders. Process Improvement: .. read more
Secure by design - sap / erp
PuneVodafone2 days ago
.. have following: - Knowledge of Information Security Assurance (NIST SP 800-37, COBIT 5) Knowledge of Data Privacy / Information Protection/ GDPR Worked for a Telco either as directly employed or as industry experience. CISSP, CCSP, CRISC or any other SANS certifications. Working knowledge of Cyber Security .. read more
Finance controller
PuneAmberYesterday
.. its financial objectives. This is a finance role with high degree of visibility with senior stakeholders, including business and functional heads. Roles & responsibilities: Perform efficient monthly, quarterly, and annual closes and deliver reporting within timelines agreed to by leadership Oversees .. read more
Java golang developer
PuneZensar TechnologiesYesterday
.. language Good to have experience with Docker Containers, Microservices and AWS technologies like Lambda, Step Functions, DynamoDB, CloudWatch, Strong background with system design, documentation, implementation, and deployment on Hands on experience in Relational or NoSQL Databases" Good communication .. read more
System engineer
PuneSgsToday
.. technical societies and Contribute, write and review technical Engage with local universities, Customer institutes and research Assist with identifying areas where new and cost-effective techniques can be Academic Qualifications: Candidates must have a Bache/or's degree in Electrical Engineering (or .. read more
Jira admin
PuneDatamatoToday
.. key performance indicators (KPIs), and trends. Analyze data to provide insights that can help teams and stakeholders make informed decisions. 7. Continuous Improvement: Stay informed about the latest JIRA features, updates, and industry best practices. Propose and implement process improvements and optimizations .. read more
Manager | corporate due diligence | pune
PuneKpmg IndiaToday
.. Adaptability to a dynamic environment often accompanied by shifting priorities and stringent deadlines Have discretion and an ability to maintain confidentiality Drive and commitment to deliver high quality service Ability to work well in teams Advanced understanding of IT systems, Knowledge of MS office .. read more
Application engineer
PuneExpert Global GroupToday
.. all Instruments, Valves, Filters, Separators, Seal Gas Heaters, Seal Gas Boosters, Electrical and Instruments control modules etc. Selection and sizing of the above components. Instrument Hookups Electrical Wiring Diagram Logic Diagram Cause & Effect Control Narratives Documentation Index for MDR/SDRL/VDRL .. read more
Uk payroll specialist
PuneNeeyamoToday
.. updated as and when changes occur. · Expected to review business, processes and keep continuous improvement towards superior delivery and gain highest client satisfaction in the payrolls of the country he/she is responsible for. · Manage team of Country leads and Process experts and have proper governance .. read more
Quality assurance lead
PuneBenchpointToday
.. are seeking a talented Salesforce Quality Assurance Lead to join their growing Salesforce practice and teams. What You'll Do: Support Commercial and Public sector clients for a rapidly growing Salesforce team Support customers in QA of their SFDC application Establish and maintain a consistent test methodology .. read more
Director product
PuneData Axle IndiaToday
.. the product lifecycle including gathering and prioritizing product and customer requirements, and defining the product vision. This position will work closely with development, sales, marketing, customers, and support to ensure revenue and customer satisfaction goals are met. Roles & Responsibilities: .. read more