Performance Test Engineer

As a Software engineer for performance, Resiliency and Scalability on this team, you will be working on complex systems running on-prem, relational databases and large and complex datasets. You will focus on optimizing overall product performance and reliability. You will focus on defining and enhancing an automate-able performance benchmarking/testing framework to understand and optimize application performance as data volume grows.

The key qualities we are looking for is a true self-started, problem solver, someone who enjoys digging deeper and has a need to understand systems and how they're wired. Someone who has strong SRE, Performance & Resiliency engineering background would be ideal for this role. This person will have experience with performance & reliability analysis and the ability to develop your own analysis tools (with open-source technology) and performance testing tools.

Job Responsibilities:

• Write code to build tools, frameworks & infrastructure with a focus on performance & resiliency testing.
• Implement API based Mock frameworks to generate production like load in lower tests environments.
• Implement shift left automated Performance and Resiliency testing as part of PR analysis to prevent defects from reaching production
• Work in initiatives and strategic project to ensure Amex products are highly scalable and performs well with sophisticated workload.
• Develop, execute and analyze performance tests with complex workload and large simulated test datasets for distributed system running in Kubernetes environment.
• Develop tools or use open-source frameworks to automate performance & Resiliency testing, application monitoring & analyzing results, and reporting issues
• Define performance & Resiliency test strategy, publish metrics and execute performance related requirements
• Setup test environment and test data required to execute performance & Resiliency tests
• Identify and analyze performance bottlenecks in the product and work with engineering to resolve the issue
• Own performance, Resiliency engineering Planning, design, implementation and execution
• Prepare and present performance benchmarks, best practices, test results, comparisons and analysis
• Provide day to day technical guidance and hands-on help to other engineers on the team
• Analyze performance results to identify performance bottlenecks and suggest optimizations for our systems

Basic Qualifications:

• Passion and drive for Performance, Scalability and Reliability engineering of all systems.
• 5+ years of experience in software development roles like developer, SRE or Performance Engineering
• Good understanding of object-oriented programming languages like Java
• Strong Analytical and debugging skills
• Hands-on experience testing distributed microservices and event driven architecture.
• Hands-on performance testing timeseries and SQL datastores
• Ability to implement Industry standard performance testing tools & mock frameworks (Jmeter, Gatlin, Jmock)
• Strong experience in bottleneck analysis, profiling, and distributed tracing tools
• Hands on experience on system monitoring & profiling tools and frameworks like Grafana, InfluxDB, Prometheus, Dynatrace, Splunk etc.)
• Experience developing & testing APIs and/or SDKs
• Comfortable using Linux, Docker, Kubernetes (any flavor of Kubernetes) & GIT

Preferred Qualifications:

• Experience in performance test engineering or SRE roles
• OOP Programming or scripting skills (Java preferred)
• Experience working in globally distributed teams
• Knowledge of how to communicate performance & reliability metrics during a release cycle
• Advanced interpersonal skills: demonstrated ability to explain complex technical issues to both technical and non-technical audiences.
• Bachelor’s in computer science or related field.