Why narrow scales still need reliability evidence

The substantive question

Revelle and Condon review different ways of evaluating reliability and why the choice depends on the measurement question. For MPF, the practical implication is that internal consistency, stability, and agreement address different concerns. A reliable software calculation cannot substitute for any of them.

A narrow scale may have strongly related items because it covers a coherent tendency or because it repeats the same situation. Reserve, for example, includes online disclosure and interpersonal disclosure; their relationship may depend on age, access, and norms. Pressure activation may vary with current demands more than task order does. These possibilities should shape the study rather than be treated as defects to remove automatically. Each facet needs uncertainty estimates appropriate to its structure and intended interpretation. Retest work should also record whether relevant contexts changed. Averaging all sixty items to obtain an impressive consistency statistic would be inappropriate because the items are not intended to measure a single common quantity.

A worked interpretation

A pilot could find that task-order responses are consistent within an administration but shift after a person changes working arrangements. That change may involve a new environment, altered self-understanding, or measurement variation. The study needs enough design information to distinguish these explanations as far as possible.

For the current reader, the safest interpretation is modest: the result summarizes answers given under one set of conditions. A difference between two facets should not be treated as a significant psychological contrast, and a retest change should not be labeled progress until the relevant measurement evidence exists.

A question worth testing

Correlated errors are another concern when several items share nearly identical situations. Three questions about public meetings could move together because of meeting experience rather than independent voice generally. MPF’s analysis should inspect such clusters and compare them with the content map. A statistical model should not conceal a narrow situation behind a broad facet label simply because the resulting fit indices appear acceptable.

How we would evaluate MPF

Reliability questions for MPF are inseparable from facet breadth. Repeating closely related questions about one behavior could make a score highly consistent while excluding other behaviors the facet name suggests. A broader independent-voice scale might include written disagreement, private conversation, and public discussion, each with different practical demands. Researchers would examine whether the common tendency is strong enough to summarize those contexts or whether the domain should be narrowed. Maximizing an internal-consistency coefficient is not an adequate rule for choosing between those interpretations.

Retest work should also examine changes in the demands available to the participant. Pressure activation cannot be interpreted identically during a quiet week and a period of accumulated obligations unless the instructions establish a common reference. Personal reserve may differ when respondents think about close relationships versus unfamiliar groups. The study should record these reference choices and report their implications for score stability. Until such evidence exists, MPF cannot determine that a small movement represents increasing confidence, improving adaptability, or worsening stress. Its software can reproduce the scoring formula exactly while the psychological meaning remains uncertain. The report should maintain that distinction and avoid precise claims of change from repeated self-description alone.