The Perception Evaluation team at Waymo is at the forefront of autonomous driving, ensuring the safety and reliability of our self-driving technology. We develop and utilize cutting-edge tools and methodologies to rigorously assess the performance of our Perception systems, a critical component for safe and effective autonomous operation. We are seeking a Staff Software Engineer play a pivotal role in shaping the future of transportation by directly impacting the quality and reliability of Waymo's autonomous vehicles.

In this hybrid role, you will report to a Staff Technical Lead Manager.

You will:

• Lead Advanced Evaluation Framework Design: You will be responsible for the architectural design and development of innovative evaluation frameworks to thoroughly assess perception system performance, particularly in new and diverse urban environments. This includes defining technical specifications and ensuring robustness.
• Innovate with AI in Evaluation Methodologies: You will develop and implement cutting-edge evaluation methodologies and scalable pipelines. This involves leveraging advancements in large-scale AI models (including language and multimodal models) to enhance analysis, generate deeper insights, and pioneer new ways to assess performance.
• Utilize Simulation and Reference Data Systems: You will drive the use of large-scale inference and simulation systems to establish performance benchmarks, validate perception capabilities, and proactively identify potential perception challenges in a variety of scenarios.
• Drive End-to-End Platform Delivery: You will exercise complete end-to-end ownership in delivering a robust and adaptable discovery evaluation platform. This platform must be capable of supporting rapid iteration and meeting the demanding timelines associated with Waymo's city expansions, enabling proactive problem-solving.
• Integrate Evaluation with ML Development Cycles: You will collaborate closely with Machine Learning and Data Infrastructure teams. Your responsibility is to ensure that evaluation findings are seamlessly integrated into the data-driven model development loop, creating a flywheel to accelerate training, hillclimbing, and overall system improvement.
• Resolve Complex Technical Challenges: You will proactively identify, troubleshoot, and resolve complex technical hurdles within the perception evaluation domain. This involves deep technical investigation and ensuring the delivery of high-quality, actionable feedback for system enhancements, often under tight deadlines.

You have:

• 8+ years of experience designing and building complex, scalable systems
• Demonstrated technical leadership in defining long-term strategy, developing roadmaps, and guiding teams to execute projects from ambiguous requirements to high-impact deliverables
• Demonstrated ability to work effectively across diverse teams and communicate technical concepts clearly
• Deep expertise in the evaluation of Machine Learning systems at scale, encompassing metrics design, rigorous experimentation, large-scale data analysis, and statistical methods
• Hands-on experience with ML infrastructure, modern AI concepts (including foundation models or similar), and their practical application to solve real-world problems
• Proficiency in C++ and Python in a production environment
• Excellent problem-solving skills, clear technical communication, and a strong sense of end-to-end ownership

We prefer: