About the Role
- At Hugging Face, we're on a journey to democratize good AI. We are building the fastest growing platform for AI builders with over 11 million users who collectively shared over 4 million models, 1 million datasets & 1.5 million Gradio apps. Our open-source libraries have more than 700,000 stars on Github.
- This role sits on the Xet Storage team, the group responsible for the storage system behind all of Hugging Face. Today we store over 200PB (and growing rapidly!) of the world's most important ML & AI assets. Xet is the underlying storage architecture for the entire Hugging Face platform and community -- from the largest model repositories to the datasets and Spaces that millions of people build on every day.
- In this role, you'll work across two closely connected surfaces. You'll contribute to xet-core, our open-source project written in Rust that powers hf-xet -- the Python library underpinning the Hugging Face Hub client and the wider ecosystem of open-source tools the community relies on. And you'll design, build, and operate meaningful and challenging features in the Xet Storage backend, contributing to the broader Infrastructure organization at Hugging Face. We're a small team building and operating incredible things at enormous scale, in a high-trust, low-process, async, and remote environment -- and we lean heavily on the latest AI tools to move fast.
- If you love writing low-level, high-performance code and are energized by the challenge of large-scale, scalable services, this is the team for you. This usually means proven experience building and operating production systems software, but we consider every applicant on an individual basis.
Responsibilities
- Contribute to xet-core, an open-source project written in Rust that powers hf-xet.
- Design, build, and operate meaningful and challenging features in the Xet Storage backend.
Requirements
- 8+ years building and scaling distributed systems, storage, or networking infrastructure.
- Proficiency in a low-level systems language, with Rust strongly preferred (we also work in Python, Typescript, Go, and C++ across the stack).
- A track record of working independently in a high-trust, low-process environment -- you're comfortable with ambiguity and take ownership end to end.
- Comfort operating in a fast-moving, async, and fully remote environment.
- A passion for building simple, robust, and scalable systems relied on by engineering and science teams around the world.
Qualifications
- Experience designing efficient, high-performance, fault-tolerant data storage and retrieval systems.
- Experience operating production systems -- monitoring, alerting, and distributed debugging and recovery.
- Familiarity with git internals, cloud infrastructure (AWS, Azure, GCP, Kubernetes), databases (relational and non-relational), or networking.
Benefits
- Flexible working hours
- Remote options
- Health, dental, and vision benefits for employees and their dependents
- Parental leave
- Flexible paid time off
- Unlimited PTO
- Company equity as part of their compensation package
- Reimbursement for relevant conferences, training, and education
- Workstation outfitting