CID-0012: USCIRF 2002 Annual Report

Document classification

Document type: TYPE 4, Composite Index Rationale: The annual report’s primary output is CPC (Countries of Particular Concern) designations. That is a categorical ranking system. Multiple indicators feed a single classificatory output. The Conditional Module for Index Construction activates.

Rubric version: v0.3.2 All 8 dimensions apply in full for TYPE 4, plus the Conditional Module (weighted at 10%, other dimensions reduced proportionally).

---

Dimension scores

D1: Definitional precision (10.8% adjusted) · Score: 4

USCIRF operates under the International Religious Freedom Act. IRFA defines “particularly severe violations of religious freedom.” The structure audit confirms a Definitions/Glossary section exists. This gives the report a legal-definitional floor that most advocacy documents lack.

But statutory language is not a coding framework. “Particularly severe” is a threshold judgment, not a decision rule. No published codebook specifies how conditions in a given country map to CPC designation versus Watch List versus no listing. Five trained analysts applying IRFA’s text to the same country conditions could reach five different classifications. The distance between statutory definition and operational decision rule keeps this in the 4–6 band.

D2: Classification rigor (16.2% adjusted) · Score: 2

CPC designations are deliberative judgments by political appointees. No structured coding protocol. No published inter-coder reliability. No documented adjudication process. No evidence of systematic training. The structure audit confirms it: inter-coder reliability is missing.

What distinguishes a CPC country from a Watch List country from an unlisted one? USCIRF does not publish the evidentiary threshold. Commissioners deliberate and vote. The reasoning behind classification outcomes stays inside the room. An outside analyst cannot replicate the process because the process is not described. Score of 2: opaque classification without formal reliability testing.

D3: Case capture and sampling (13.5% adjusted) · Score: 2

This score triggers the non-compensatory cap at 5.9.

How does USCIRF select which countries to scrutinize? The 2002 report covers many countries. The selection methodology is absent. No published search strategy. No framework for determining which countries enter the assessment universe. No documented process for checking whether the country set reflects global religious freedom conditions rather than Commissioner interest, staff capacity, or geopolitical salience.

The pre-analysis flags zero denominator references. USCIRF does not report its assessments relative to any external baseline. The closed-universe problem applies: the assessed countries are a pre-filtered dataset, but USCIRF never publishes how the filtering works.

D4: Coverage symmetry (13.5% adjusted) · Score: 5

The title is universalist (“International Religious Freedom”). The content matches, at least in part. Identity term analysis shows coverage across Muslim (46 mentions), Christian (25), Hindu (6), Buddhist (6), and Sikh communities. USCIRF in 2002 was not monitoring persecution in only one direction. That counts for something.

Where the score drops: country selection is not benchmarked against base-rate data on religious freedom globally. Content directionality shows 100% of directional terms framed as anti-Christian, suggesting that while multiple groups appear, the victimization framing skews in one direction for this year. The statutory criteria are formally neutral, so the report partially passes the Swap Test. But opacity around country selection prevents a higher score. We cannot confirm the coverage is proportional because we cannot see how it was built.

D5: Source independence (9.0% adjusted) · Score: 3

One URL in 31,464 words. One. The Herfindahl Index hits 1.0, maximum concentration. Organization mentions: Congress (44), USCIRF itself (11). Zero academic sources. Zero media sources cited with verifiable links.

USCIRF does take testimony, conduct hearings, and make site visits. These are real information inputs from outside the organization. The problem is the report does not cite them in a way that permits provenance tracing. Run the Provenance Trace on any country characterization: USCIRF says conditions exist, the report presents USCIRF’s assessment, and no external source appears that would allow independent confirmation. Nearly every claim dead-ends at USCIRF’s own judgment.

Score of 3 reflects that USCIRF is not a single-analyst shop. It has institutional structure and external inputs. But the citation infrastructure makes independent verification of specific claims impossible.

D6: Verification standards (16.2% adjusted) · Score: 2

This score (below 7) prevents Research-Grade classification.

The structure audit confirms: Data Availability missing. No dataset for download. No machine-readable format. No documented formal request process for underlying country assessments. This is Tier 3 data access, the lowest tier, which hard-caps D6 at 5 under the rubric. The actual score falls below that cap.

Country conditions appear as aggregate narrative without individual event sourcing. The 5% Replication Standard cannot apply because no dataset exists to replicate against. Try selecting 10 claims at random from the report and locating an independently verifiable primary source within 30 minutes for each. With one URL in the entire document, the success rate would approach zero. The report asks you to trust USCIRF’s authority. It gives you no other option.

D7: Transparency and governance (4.5% adjusted) · Score: 6

Congressional appropriation funds USCIRF. That is fully disclosed by statute. Commissioners are presidential appointees confirmed by Congress. Their identities, affiliations, and appointment authority are public. The governance structure is statutory and clear. The funding disclosure section appears in the structure audit.

The score stops at 6 because D7 measures the transparency of the decision-making process, not just institutional structure. Who decides how country conditions are characterized? The governance of the designation process itself stays opaque even within a transparent institutional shell. A politically appointed commission with clear institutional transparency but hidden methodological governance lands in the mid-upper range.

D8: Counter-evidence (6.3% adjusted) · Score: 1

Limitations section: missing. Counter-evidence section: missing. Corrections/Errata Policy: missing. The orientation assessment flags it directly: “Recommendations present but no limitations = ADVOCACY orientation.”

No evidence that USCIRF engaged with methodological criticism in 2002. No published corrections policy. No limitations acknowledgment. No documented methodology updates. The report presents its assessments as authoritative, full stop. Score of 1.

Conditional Module: Index construction (10%) · Score: 1

For an organization whose primary product is a country classification system, this is the score that matters most. The CPC designation system publishes no mathematical specification. No indicator selection justification beyond statutory language. No sensitivity testing. No confidence intervals. No robustness checks. No documentation of how multiple indicators get aggregated into CPC versus Watch List versus unlisted.

The “index” is an expert panel judgment with no published aggregation method. Weights are implicit in Commissioner deliberation. They are not formally specified or tested. This falls in the 0–3 band: opaque formula, arbitrary-looking weights.

---

Score computation

Adjusted weights (Conditional Module active at 10%, all others × 0.9):

Dimension	Score	Adjusted weight	Weighted
D1	4	10.8%	0.432
D2	2	16.2%	0.324
D3	2	13.5%	0.270
D4	5	13.5%	0.675
D5	3	9.0%	0.270
D6	2	16.2%	0.324
D7	6	4.5%	0.270
D8	1	6.3%	0.063
CM	1	10.0%	0.100
Total		100%	2.73

Non-compensatory caps applied:

• D3 = 2 (below 3): Overall score capped at 5.9. Raw score already falls below; cap is active but not binding.
• D6 = 2 (below 7): Cannot reach Research-Grade. Active but irrelevant at this score level.

Raw score: 2.73 Final score: 2.73 Grade: Advocacy-Grade (2.0–3.9)

---

Sensitivity analysis

Weighting scheme	Score	Grade
Standard (v0.3.2)	2.73	Advocacy-Grade
Equal weights (all 9 equally)	2.44	Advocacy-Grade
Verification-heavy (D6 at 25%)	2.46	Advocacy-Grade

Grade is stable across all three schemes. No instability to report.

---

Key findings

The methodology gap is structural, not evolutionary. This 2002 score fits the longitudinal pattern across previously scored USCIRF annual reports (1999, 2013, 2016). Zero verifiable citations. Opaque classification methodology. Tier 3 data access. No counter-evidence engagement. These are not problems of a particular year or a particular set of Commissioners. They are architectural features.

D4 is the differentiator. USCIRF’s multi-directional coverage across religious communities produces a D4 score higher than single-direction monitoring organizations earn. The statutory mandate creates this. It requires attention to persecution regardless of which group is targeted.

D7 benefits from government structure. Congressional funding and public appointments give USCIRF transparency that nonprofits must build from scratch. That institutional transparency does not compensate for methodological opacity. It never has, across any scored year.

The Conditional Module exposes the core problem. CPC designations are USCIRF’s primary product. The complete absence of a published aggregation methodology, sensitivity testing, or robustness checks is not a minor gap. It is the gap. An organization that ranks countries on religious freedom without publishing the ranking formula has a credibility problem that no amount of institutional prestige can paper over.