Validation Transparency Policy
Preflight is a commercial product, but validation trust depends on users understanding how outcomes are produced.
Our policy is:
- Public behavior contract: We document rules that affect user-visible outcomes.
- Private implementation details: We keep tuning internals proprietary where needed.
What We Publish
Section titled “What We Publish”These behaviors are part of the public contract:
- Canonical adjudication behavior:
- One issue => one adjudicated outcome.
- Deterministic precedence rules (severity, fix availability, source tie-break).
- Health score coupling:
- The same canonical diagnostics stream powers issue counts and score.
- UI, MCP, and CLI consume the same adjudicated results model.
- Health score production guardrails:
- Catastrophic load/import failures force score to 0.0.
- Any unresolved error caps score to Blocked (
<= 4.5). - Errors are not demoted to warnings in render scoring.
- Non-applicable categories are excluded from scoring.
- Validator/fix semantics:
- Why specific diagnostics matter in production pipelines.
- What a fix changes at a schema/authoring level.
This gives teams predictability and auditability for release decisions.
What We Keep Proprietary
Section titled “What We Keep Proprietary”These are intentionally not fully disclosed:
- Exact per-issue scoring weights and fine-grained thresholds.
- Internal prioritization heuristics and ranking formulas.
- Defensive handling and anti-gaming logic for pathological assets.
- Operational details that could be exploited to inflate score without improving runtime quality.
Why This Balance
Section titled “Why This Balance”Publishing the behavior contract builds confidence and enables pipeline integration. Keeping detailed tuning private protects product differentiation and prevents score gaming.
In short:
- Transparent on outcomes
- Selective on internals
That is the recommended posture for Preflight as a commercial validation platform.