About the Role
As a Software Engineer on the Ray Serve team at Anyscale, you will focus on building and enhancing Ray Serve, a scalable and flexible serving library for machine learning models. This role involves developing the infrastructure and APIs that enable developers to deploy and manage production-ready AI services with ease, from single models to complex multi-model pipelines. You will work on challenges related to high-throughput inference, auto-scaling, model lifecycle management, and integration with various ML frameworks, ensuring that Ray Serve remains a leading solution for scalable AI deployment. Your contributions will directly impact the ability to bring cutting-edge AI research into real-world applications, bridging the gap between development and production.