Big Data Engineer

  • Pune
  • Qualys

We are seeking a talented Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks.

Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security Risk Prioritization product.

This is a fantastic opportunity to be an integral part of a team building Qualys next generation platform using Big Data & Micro-Services based technology to process over billions of transactions data per day, leverage open-source technologies, and work on challenging and business-impacting initiatives.

Responsibilities:

  • Liason to product teams, professional services and sales engineers on solution and trade-off reviews and represent engineering in such conversations.
  • Support the team in technology explorations and roadmaps.
  • Functionally decompose complex problems into simple, straight-forward solutions.
  • Fully and completely understand system interdependencies and limitations.
  • Possess expert knowledge in performance, scalability, enterprise system architecture, and engineering best practices.
  • Leverage knowledge of internal and industry prior art in design decisions.
  • Effectively research and benchmark cloud technology against other competing systems in the industry.

Qualifications:

  • Bachelor’s degree in computer science or equivalent
  • 2+ years of total experience.
  • 2+ years of relevant experience in design and architecture Big Data solutions using Spark
  • 2+ years experience in working with engineering resources for innovation.
  • 2+ years experience in understanding Big Data events flow pipeline.
  • 2+ years experience in performance testing for large infrastructure.
  • Understanding various search solutions solr/elastic.
  • Understanding in Kafka
  • Knowledge in Presto technology.
  • Knowledge in Airflow.
  • Hands-on experience in scripting and automation
  • Understanding of RDBMS/NoSQL, Oracle , Cassandra , Kafka , Redis, Hadoop, lambda architecture, kappa , kappa ++ architectures with flink data streaming and rule engines
  • Experience in working with ML models engineering and related deployment.