Back to home

Ponyhof

Dysfunctional Programming

Fair Resource Distribution Algorithm v1

Imagine a finite resource that you want to distribute amongst peers in a fair manner. If you know the number of peers to be n, the problem becomes trivial and you can assign every peer 1/n-th of the total. This way every peer gets the same amount, while no part of the resource stays unused. But what if the number of peers is only known retrospectively? That is, how many resources do you grant a peer if you do not know whether there are more peers or not? How do you define “fairness”? And how do you make sure as little of the resource as possible stays unused?

The fairdist algorithm provides one possible solution to this problem. It defines how many resources a new peer is assigned, considering the following propertis:

  1. The total amount of resources already distributed to other peers. This is also referred to by the term consumption.
  2. The number of peers that already got resources assigned.
  3. The amount of resources remaining. That is, the resources that are remaining to be distributed. This is also referred to by the term reserve.

The following is a mathematical proof of the properties of the fairdist algorithm. For the reference implementation of the algorithm and information on the different applications of it, see the r-fairdist project.

Prerequisites

We define a set of symbols up front, to keep the proofs shorter. Whenever these symbols are mentioned, the following definition applies:

  • Let be a total amount of consumed resources.
  • Let be a total amount of reserved resources.
  • Let be a number of peers that consumed resources.
  • Let be a function that computes the proportion of a peer can consume, based on the number of peers that currently have resources consumed.
  • Let be a function that computes the proportion of a peer is guaranteed, based on the number of peers that currently have resources consumed.

The algorithm considers a total amount of resources, but splits it into two separate parts, the remaining reserve and the consumed part . Their sum represents the total amount that was initially available. It then declares a function , which is the resource allocator. It will later on be used to calculate how many resources of the reserve a peer can allocate: . That is, defines the proportion of the reserve a new peer gets access to. The smaller it is, the more a peer gets.

Similarly, the guarantee is used to declare a lower bound of the total resources the allocator grants a new peer. That is, while is a function applied to allocations, is a property the allocator will guarantee you. Unlike an allocation, will later on be calculated based on the total amount of resources: . Again, the function defines the proportion that is guaranteed. So the smaller is, the stronger the guarantee becomes.

Definition

The allocator is said to guarantee the limit , if there exists a function so that for all , , and :

implies both:

The idea here is to define a function which calculates how many resources a new peer can allocate. That is, considering a new peer requests resources, it will get of the reserve. The first property of this implication guarantees that this allocation is bigger than, or equal to, the guaranteed total for each peer. The guaranteed total is calculated through based on the total amount of resources (which is the consumption plus the reserve).

If you now pick an allocator and a guarantee that fulfil this definition, the idea is that this ensures you that the allocator can be used to serve resource requests from new peers, and it ensures that regardless of how many peers will request resources, each one will be guaranteed an amount equal to, or bigger than, the guarantee .

This definition requires the existance of a reserve watermark . It uses this watermark as a selector for an inductive step. That is, if the requirements of this reserve selector are true, the second implication guarantees that they are true for an infinite number of following allocations. That is, the right hand side of the second implication matches exactly the requirement of the implication, once a single allocation was performed (i.e., a resource chunk was subtracted from the reserve and added to the total consumption, while the number of peers increased by one).

Note that if is , then the requirement of the implication is true for all and . This guarantees that there is always a situation where allocator can actually be applied.

Lemma 1

To prove an allocator guarantees , it is sufficient to show that fulfils:

and

This lemma is used to make it easier to prove a specific allocator guarantees a specific limit. Without it, each proof of the different allocators would have to replicate it.

However, this lemma also gives a better feeling of what the different functions actually mean. For instance, it clearly shows must always be smaller than , and that by a considerable amount. If , then no would ever fulfil this requirement (remember: ). At the same time, you can see the closer and are together, the smaller gets, and as such the requirements on the reserve get harder to fulfil.

The second requirement gives you a recursive equation to find an for any allocator you pick. Hence, in combination both these requirements show you an iterative process to find and , for any guarantee you pick. However, the closer and get, the harder it becomes to solve the recursive equation.

Proof

To show this lemma is true, we must show both implications of the definition are true. As first step, we show the first implication is true, which is:

We show this b starting with the left-hand side and showing it implies the right hand side, using the requisite of this lemma.

As second step, we need to show the second implication of the definition is true, which is:

To prove this, we start with the second requisite of this lemma and then show it implies the right-hand side of the implication, using the requisite of the implication.

Hint: The following introduction of is correct, since is per definition greater than , so neither side can be 0.

Theorem

The following allocators each guarantee the specified limit:

This theorem defines three different allocators for different guarantees. The last one provides the strongest guarantee. Both the allocation and the guarantee are quasilinear. It is thus a good fit for fair allocation schemes, while still being reasonably fast to compute.

The other two provide quadratic and exponential guarantees and are mostly listed for documentational purposes. With the quasilinear guarantees at hand, there is little reason to use the other two.

As you might notice, this theorem does not provide a solution where and become infinitesimally close. It remains open whether what this solution would look like. However, the listed quasilinear solution is good enough, that it is unlikely that better options exist, which can still be calculated in reasonable amounts of time.

Proof

We provide a function for each pair. We then substitute them in Lemma 1 and show through equivalence transformations that the assertions are true.

Proof 1: Exponential Guarantee

  • Allocator:
  • Guarantee:

Let .

Part 1:

Part 2:

Proof 2: Polynomial Guarantee

  • Allocator:
  • Guarantee:

Let .

Part 1:

Part 2:

Proof 3: Quasilinear Guarantee

  • Allocator:
  • Guarantee:

Let .

Part 1:

Part 2:

For this part, we rely on the following property:

This is true for all logarithms for all .

We now show the second requirement of the Lemma is true. However, we cannot use equivalence transformations as in the other proofs. Hence, we show it by implication.

Written by David Rheinsberg, on March 13, 2019.