Table 2 Methodological framework for establishing feasibility, validity and reliability
Stage | Process |
|---|---|
Proof of concept–feasibility | 1. Field testing and pilot testing of measure in controlled and free-living settings |
Content and Face validity | 2. Examination of relevant literature 3. Consultation with relevant experts 4. Theoretical examination of measure and domain/dimension 5. Examination of proposed data processing and decision algorithms including sensitivity analysis |
Convergent validity | 6. Assessment of the agreement between your measure and an existing (non-criterion) measure |
Criterion validity | 7. Assessment of the agreement between your measure and a criterion measure |
Internal validity | 8. Examination of bias such as reactivity and missing data |
External validity | 9. Examination of sample bias (age, sex, ethnic origin, socio-economic status) |
Inter-rater reliability | 10. Assessment of stability of tests administered by different researchers |
Inter-instrument reliability | 11. Assessment of stability of tests administered using multiple versions of the same instrument |
Test-retest reliability | 12. Assessment of stability of consecutive tests |
Behavioural reliability | 13. Assessment of stability accounting for behavioural changes |
Context validity | 14. Based on all assessments, will measure give useful information in the proposed context? |
Purpose validity | 15. Based on all assessments and considering study design, are the validity and reliability results suitable for the proposed use and likely to allow the research question to be answered? |