Research Engineer, Frontier Safety Risk Assessment

About the Role

Artificial Intelligence could be one of humanity’s most useful inventions. At Google DeepMind, we’re a team of scientists, engineers, machine learning experts and more, working together to advance the state of the art in artificial intelligence. We use our technologies for widespread public benefit and scientific discovery, and collaborate with others on critical challenges, ensuring safety and ethics are the highest priority. Our team identifies, assesses, and mitigates potential catastrophic risks from current and future AI systems. As a member of technical staff, you will design, implement, and empirically validate approaches to assessing and managing catastrophic risk from current and future frontier AI systems. At the moment, these risks range from loss of control of advanced AI systems or automated ML R&D through misuse of AI for widespread CBRN or cyber harm. The Risk Assessment team measures and assesses the possible risks posed by frontier systems, making sure that GDM knows the capabilities and propensities of frontier models so that adequate mitigations are in place. We also make sure that the mitigations do enough to manage the risks. But the risks posed by frontier systems are, themselves, unclear. Forecasting the possible risk pathways is challenging, as is designing and implementing sensors that could reliably detect emerging risks before we actually have real-world examples. We focus on building decision-relevant and trustworthy evaluation systems that prioritise compute and effort on risk measurements with the highest value of information. We then need to be able to assess the extent to which proposed and implemented mitigations actually cover the identified risks, and to measure how successfully they generalize to novel settings. The Risk Assessment team is part of Frontier Safety which is responsible for measuring and managing severe potential risks from current and next-generation Frontier models. Our approach is one of adaptively scaling risk assessment and mitigation processes to handle the near-future. We are part of GDM’s AGI Safety and Alignment Team, whose other members focus on research aimed at enabling systems further in the future to be aligned and safe. These include interpretability, scalable oversight, control, and incentives. We are seeking 2 Research Engineers for the Frontier Safety Risk Assessment team within the AGI Safety and Alignment Team. In this role, you will contribute novel research towards our ability to measure and assess risk from frontier models.

Responsibilities

• Identifying new risk pathways within current areas (loss of control, ML R&D, cyber, CBRN, harmful manipulation) or in new ones;
• Conceiving of, designing, and developing new ways to measure pre-mitigation and post-mitigation risk.
• Forecasting and scenario planning for future risks which are not yet material.

Requirements

• You have extensive research experience with deep learning and/or foundation models (for example, but not necessarily, a PhD in machine learning).
• You are adept at generating ideas and designing experiments, and implementing these in Python with real AI systems.
• You are keen to address risks from foundation models, and have thought about how to do so. You plan for your research to impact production systems on a timescale between “immediately” and “a few years”.
• You are excited to work with strong contributors to make progress towards a shared ambitious goal.
• With strong, clear communication skills, you are confident engaging technical stakeholders to share research insights tailored to their background.

Qualifications

• Experience in areas such as frontier risk assessment and/or mitigations, safety, and alignment.
• Engineering experience with LLM training and inference.
• PhD in Computer Science or Machine Learning related field.
• A track record of publications at venues such as NeurIPS, ICLR, ICML, RL/DL, EMNLP, AAAI and UAI.
• Experience with collaborating or leading an applied research project.