Evaluation service v0.1

[James-Mart]

General flow

Actions in each phase can only be called during that phase.

Registration phase

• Evaluation registration (Called by everyone, for every eval): Register for the evaluation. This ensures they pay for the storage costs of the record. Also, submit entropy. Also, optionally, provide context to help others with their evaluation

Deliberation phase

• Generate groups (Called once by one member, for every eval):
  • Calculates all group sizes based on the total number of registered participants
  • Then calculates a random ordering of the registered members based on some entropy source (v0.1 can just be last block ID or something)
  • 1:1 assignment between the random ordering of members and the calculated groups.
• Propose (Called by every member of each group, for every eval)
  • Group initialization: The first member to attempt to propose a rank-ordering will also generate and publish a shared group secret. To do this, generate a symmetric key (which is the secret), then encrypt it with each group member's encryption key (using ECIES), then call an action to publish the encrypted secrets.
  • Propose: Submit a proposed rank ordering of the other members of the group, encrypted with the shared symmetric key.

Submission phase

• Attest (Called by every member of each group, for every eval): Calculate and submit the unencrypted final alignment-weighted rank-ordering.
  • The vote that solidifies finality for the rank-ordering is the first vote that meets the threshold where a simple majority of group members have submitted an aligned vote. On finality, the final group ranking is saved.

Service interface

Something like...

////////////////////////////////////// TYPES

interface hook-service-api {
   eval-register: func(eval-id, user, description) -> bool
   eval-propose: func(proposer, eval-id, group-nr, ordering: list<u8>)
   eval-attest: func(attester, eval-id, group-nr, ordering: list<account>)
   eval-group-fin: func(eval-id, group-nr, hash)
}

record Timings { // seconds
   registration: u32,
   deliberation: u32, 
   submission: u32
}

record EvalConfig {
   owner: account,
   use-hooks: bool,
   group-sizes: option<list<u8>>, // None means only one group, otherwise list entries are allowed group sizes. Minimum = 3.
   timings: Timings
}

record Eval {
   id: u32,
   creation-date,
   config: EvalConfig,   
}

record score {
   eval-id: u32, 
   user: account,
   entropy: u64,
   rank: u8,
   group-id: option<u16>,
   group-key: option<list<u8>>,
}
score-table: table<score>

record attestation: {
	eval-id: u32,   
	hash: list<u8>, // primary key: { evail-id, hash }
	group-nr: u16,
	ordering: list<account>,
	nr-attested: u8,
}

////////////////////////////////////// ACTIONS

/// Creates a new evaluation
/// - Temporarily: Anyone can open an eval on behalf of anyone. 
///   Future: Validate that the sender == owner
open(eval: Eval)

/// Register for an evaluation
/// - Requires specified eval is open
/// - Requires being in the registration phase
/// - Calls `hook-service:is-valid-reg` to allow eval owner to enforce constraints
register(id: u32, entropy: u64, description: option<string>)

/// Generate groups
/// - Requires being in the deliberation phase
/// - Calculates the groups on the server side
generate-groups(id: u32)

/// Publish group-key.
/// The group key is an encrypted symmetric key used for sharing group-private messages
/// The number of keys should match the number of group members
/// - Requires being in the deliberation phase
/// - Requires no group-key exists
group-key(id: u32, group-nr: u16, keys: list<list<u8>>)

/// Propose a rank-ordering (encrypted with group key)
/// - Requires being in the deliberation phase
propose(id: u32, group-nr: u16, ordering: list<u8>)

/// Attest to a final rank-ordering
/// - Requires being in the submission phase
/// - Requires that sender is in the specified group
/// - Requires that the sender has not attested prior
attest(id: u32, group-nr: u16, ordering: list<account>)

/// Closes an evaluation, deleting all associated data
/// - Requires that the submission phase has ended
/// - Temporarily: anyone can close an eval
///   Future: Validate that the sender == owner
close(id: u32)

[Velua]

/// Attest to a final rank-ordering
/// - Requires being in the submission phase
/// - Requires that sender is in the specified group
/// - Requires that the sender has not attested prior
attest(id: u32, group-nr: u16, ordering: list)

@James-Mart
Is it required to be in the submission phase? I think we found in previous instances of fractal meetings some users were able to decide really quick and afterwards ended up awkwardly waiting around for the submission phase.

[Velua]

In this system, evaluations are more async than sync, members of a group can figure out when they want to deliberate together, therefore the process can be around a week long.

Room communication

Users need their clients to talk in order to determine their final attetations based on all their rank orders being combined privately.

WebRTC

Use the same service that does the video calls and audio calls to also allow syncing.

Service itself

Dump the data on the service encrypting it with a shared key.
First user to join publishes a shared group secret, encrypted with the public keys of all the group members for them to decrypt.