Evaluator and Synchronizer

Details of Open Match Synchronizer and Evaluator and how to configure them.

Overlapping Match Profiles

In a basic flow to generate Matches, the Director calls FetchMatches on Open Match backend for one or more MatchProfiles. Open Match triggers execution of MatchFunctions for each of these requested MatchProfiles concurrently. Each MatchFunction queries Open Match for the Tickets that Match the filtering criteria specified for the Pools in their MatchProfile. The MatchFunction then generates Matches using the Tickets it fetched and returns them as proposals.

If the MatchProfiles perfectly partition the concurrent Ticket Pool (their Pool filters are such that each MatchProfile will only Match a non overlapping set of Tickets), then the Matches generated by the MatchFunction execution do not have any overlap.

However, this may not always be the case. There are scenarios in which, the Pools specified in concurrent MatchProfiles may have an overlap and thus concurrently executing MatchFunctions could use overlapping Ticket in its proposals.

Here is an example scenario for clarification: Lets assume a simple case where a Ticket has an attribute ‘mmr’ that can be a value between 0 and 10.

Case1 - Non-overlapping MatchProfiles

The Director could request for the following Profiles:

  • Profile1 has a Pool that filters all Tickets with mmr 0 to 5.
  • Profile2 has a Pool that filters all Tickets with mmr 6 to 10.

Given the mmr ranges, there is no overlap between the Tickets fetched by the MatchFunction execution for each of these profiles. Hence there is no overlap in the generated result matches.

Case2 - Overlapping MatchProfiles

The Director could request for the following Profiles:

  • Profile1 has a Pool that filters all Tickets with mmr 0 to 5.
  • Profile2 has a Pool that filters all Tickets with mmr 2 to 8.
  • Profile3 has a Pool that filters all Tickets with mmr 6 to 10.

In this case, the MatchFunction executing Profile1 and Profile2 will have some common players they consider for generating Matches (Similarly with Profile2 and Profile3). Thus, the generated Matches may have overlapping Tickets.

This overlap is desired in some real matchmaking scenarios, . Inorder to generate high quality Matches, you may want your Ticket to be considered concurrently for multiple possible MatchProfiles and then be able to compare the quality of these overlapping Matches, picking the highest quality Match and discarding the rest. For instance in the above example, a Ticket with mmr 3 could be a part of a Matches generated by Profile1 and Profile2. Based on the game design and the mmr of other Tickets in these overlapping Matches, one of these two Matches may actually be a better fit for this Ticket - and thus, searching for Matches with overlapping mmr ranges resulted in considering 2 options for this Ticket and picking the better fit.

The ability to provide this model of executing concurrent MatchFunctions on overlapping Ticket Pool is a core value proposition of Open Match.

Evaluator

Why do we need an Evaluator

To allow generation of proposals using overlapping Tickets but still provide unique Matches to the Director, we need to decollide the proposals. To achieve this, Open Match has an Evaluator component that accepts all concurrent Match proposals, evaluates them for collision and returns non-overlapping results back to Open Match. If there are multiple overlapping Matches, the Evaluator evaluates their quality, promotes the highest quality Match as a result and discards the others. The key responsibility of the Evaluator is to ensure that there are no overlapping Tickets in the result Matches returned.

How to use an Evaluator

Open Match provides a default score based Evaluator that can be used in most common scenarios but it also provides the user the ability to build your own Evaluator and to customize Open Match to use the user’s Evaluator. At a high level, there are three evaluation scenarios for you to consider when using Open Match:

  1. Your MatchProfiles completely partition the Ticket Pool and so you will never have collisions. The default Evaluator handles this scenario and you do not have to make any changes.
  2. Your MatchProfiles may use overlapping Ticket Pools - but each MatchFunction can simply generate a quality score for that Match based on certain game specific criteria (latency, mmr etc.). The default Evaluator can be used for this scenario. See Default Evaluator Tutorial
  3. You have complex evaluation logic that cannot be simplified to a score for a Match but rather involves actually comparing some details of the overlapping Matches. Open Match provides you with the ability to plug in your custom Evaluator to handle this case. See the Custom Evaluator Tutorial for details.

How to configure an Evaluator

Inorder to invoke the Evaluator at runtime, Open Match needs to know the location of the Evaluator endpoint. This is provided to Open Match as an override ConfigMap. Here are the field in that config map that specifically configure an Evaluator in Open Match:

  api:
    evaluator:
      hostname: <Fully qualified host name>
      grpcport: <service port for the Evaluator>

Synchronizer

Why do we need a Synchronizer

Concurrently executing MatchFunctions fetch Tickets for their MatchProfiles from Open Match at the same time. Thus, the same active Ticket Pool is visible to multiple MatchFunctions (resulting in collision). Although Evaluation handles decolliding the overlapping results, there is a race condition. While the Evaluator is evaluating Match proposals, if MatchFunction execution is permitted, then the MatchFunctions may possible fetch Tickets that were already promoted as results. There is thus a need to stop execution of MatchFunctions (thereby halting new observations from being made) while existing results are being evaluated. This is called Synchronization.

Overview of Synchronizer States

Synchronizer’s key role is to make sure that all the proposals generated by concurrent MatchFunctions get evaluated together. Further, the Synchronizer needs to halt function execution while the results are evaluated. Given that the MatchFunction is authored by the user, Open Match has no control over its execution time. However, Open Match has to make sure its results are evaluated with the other concurrent Match Functions - or are discarded if they missed the Evaluation time. To achieve this, the singleton Synchronizer component in Open Match has the following states:

Idle State

The synchronizer is in Idle state when there is no ongoing MatchFunction execution.

Registration

Open Match backend registers each FetchMatch request with the Sychronizer before it calls into the MatchFunction. The Synchronizer enters the Registration window (configurable) upon first FetchMatches registration . All the MatchFunctions that get registered in this registration window will be evaluated together.

ProposalCollection

After the Registration Window is over, The synchronizer can be configured to give some time for the registered functions to return. This is called the ProposalCollection window. This is only so that even the last registered FetchMatches request has a chance to generate proposals before triggering Evaluation.

Evaluation

This is where all the generated proposals from all the registered MatchFunctions are sent to the Evaluator for evaluation. Synchronization is achieved by primarily two behaviors:

  • Any new Match Function that attempts to Register in the ProposalCollection or Evaluation window is blocked till the evaluation is complete and registration opens again.
  • Proposals from a MatchFunction execution that return after the ProposalCollection window are discarded (they missed their evaluation window)

How to configure a Synchronizer

Open Match enables the following two window durations to be configurable by the user. These are specified in the override ConfigMap.

  registrationIntervalMs: <millisecond value>
  proposalCollectionIntervalMs: <millisecond value>

Here is what you should consider when configuring these:

  • registrationIntervalMs: This is the amount of time within which we expect the Director to send all the FetchMatch requests for new matches and backfills.
  • proposalCollectionIntervalMs: This is the maximum amount of time it would take to execute a MatchFunction after which, the results from the MatchFunction will be ignored.


Last modified January 18, 2024