Evaluation Metrics

Purpose

These evaluation metrics provide a shared vocabulary for observing and comparing how institutional systems behave in real-world experimental contexts.

They are qualitative by design and focus on structural friction, not performance optimization.

Metric Categories

1. Identity Friction

Situations where participants experience difficulty establishing, proving, or understanding identity.

Examples:

• unclear identity requirements,
• repeated verification,
• mismatch between social identity and formal representation.

2. Credential Portability

The ease with which credentials or attestations move across contexts.

Observed issues may include:

• credentials valid in one context but unusable elsewhere,
• lack of clarity around issuer authority,
• breakdowns in cross-system recognition.

3. Authorization Clarity

How clearly participation rights and permissions are defined and enforced.

Indicators:

• ambiguity around eligibility,
• inconsistent access decisions,
• ad-hoc overrides.

4. Exception Handling

How systems respond to cases outside predefined norms.

This includes:

• manual workarounds,
• informal trust substitution,
• silent failures.

5. Comprehension Cost

The cognitive effort required for participants to understand what is happening.

High comprehension cost is indicated by:

• repeated explanations,
• reliance on intermediaries,
• disengagement or avoidance.

6. Trust Formation and Breakdown

How trust is established, maintained, or lost.

Signals include:

• peer verification replacing formal checks,
• social signaling compensating for system gaps,
• breakdowns leading to exclusion or conflict.

Usage Guidance

Not all metrics will appear in every case.

Cases should document:

• which metrics were relevant,
• where observations were inconclusive,
• and which areas require further investigation.

Metrics are intended to support comparison, not scoring.