Open-sourcing Rebalancer for solving resource allocation problems

Meta has open-sourced Rebalancer, a high-performance library used internally for over nine years to solve complex resource allocation and assignment problems.

2 min read

Rebalancer open source announcement graphic showing the library title and balancing scale icon.
Rebalancer is a generic high-performance library for solving resource allocation and assignment problems, now available as open source.

Meta has open-sourced Rebalancer, a framework designed to solve complex resource allocation and assignment problems across large-scale infrastructure. The library has been used internally at Meta for over nine years to manage hardware placement, service allocation, task scheduling, and global traffic routing.

Key facts

Framework
Rebalancer
Organization
Meta Platforms
Conference
OSDI '24
Publication Year
2024

Solving resource allocation challenges

Assignment problems involve matching a set of objects to a set of bins while optimizing specific objectives and respecting constraints. At Meta, this includes positioning server racks in datacenters to handle power and cooling limitations, assigning servers to services to maximize fault tolerance, and routing user traffic to geographically distributed datacenters.

Developing a reusable framework for these tasks presents major challenges in usability and scalability. Practitioners often struggle to translate real-life policies into formal mathematical formulas, while complex models can be difficult for standard solvers to handle efficiently. Rebalancer addresses this by cleanly separating how a problem is specified from how it is solved.

Specification and solvers

Rebalancer provides a high-level specification language that uses modeling constructs such as dimensions, partitions, scopes, and utilization. Users describe problems using predefined recipes known as specs, which automatically generate the underlying mathematical formulas. Once specified, the framework translates the problem into an expression graph.

Diagram showing the architecture flow from specification through expression graph to optimal and local search solvers.
Architecture diagram illustrating how Rebalancer separates problem specification from the expression graph representation and solvers., Source ↗

Users can choose between two main solving techniques depending on their scale and requirements:

  • Optimal Solver: Translates the expression graph into a mixed integer program, which can be processed by commercial solvers like FICO Xpress or Gurobi, or open-source solvers like HiGHS.
  • Local Search Solver: Operates directly on the expression graph by exploring the local neighborhood around a current assignment, allowing it to handle massive problems without memory limitations.

At Meta, the local search solver handles the vast majority of large-scale problems. For debugging, the open-source release also includes Rebalancer Explorer, a Dockerized web user interface designed to help developers inspect constraints and analyze placement decisions.

Availability

The library is now available as open-source software under the Apache 2.0 license, alongside its documentation, Python package on PyPI, and accompanying research papers.

Source: Original announcement

Worth a read?

Report

Comments