✅Reviewer System

Safeguarding data quality

Reviewers are algorithmically selected users who are asked to review contributions made by other users. A contribution consists of dApp specific data provided by a user. Before being accepted, the data needs to be undergo a quality check by other users.

Reviewing Contributions

Every time a contribution is made (= data is added to a dApp by a user), the contribution gets a status of contribution in review. dApp builders can decide on the number of reviewers they want for each contribution and also what the minimum reputation score of a reviewer should be.

Curator then pushes review requests to a number of users, asking them to review a specific contribution. These users then temporarily receive a reviewer status for the assigned contribution. The number of assigned users will always be higher than the number of users needed to reach consensus for a review.

The Prisoner's Dilemma​

Game theory is used to safeguard the quality of the review process. This process can be compared to The Prisoner's Dilemma, where each person is isolated from other people being questioned and doesn't know what the other person will answer.

To keep everyone honest, rewards and penalties are linked to one's answer. Game theory has learned us that people will then act out of self-interest and feel pressure to speak the truth. The Curator reviewer process follows a similar logic.

The Review Process

At any time, a reviewer can start the review process in which they can pick the contributions they want to review. To ensure the integrity of the review process, the subset of contributions to which the reviewer has access is assigned on a random basis. This prevents malicious collaboration between reviewers and contributors.

Reviewer buckets are used to guarantee this random assignment. At a specific point in time, a reviewer will only have access to 1 reviewer bucket, determined by the following formula where hash() is a secure hash function and block_hash is the hash of the latest block on the blockchain:

$$b_{reviewer} = {hash}(reviewer\_address\ |\ block\_hash )\ mod\ n$$

Note: The reviewer bucket changes at specific intervals, which may not coincide with every new block.

Reviewer Consensus

To guarantee the integrity of the review process, it is important to make sure that the impact of a malicious reviewer is limited. Curator uses a reviewer consensus mechanism where multiple reviewers evaluate the validity of a contribution and the majority vote of the reviewers will determine the acceptance or rejection of the contribution.