Skip to main content

Validity and reliability of the semi-quantitative self-report Home Food Availability Inventory Checklist (HFAI-C) in White and South Asian populations

Abstract

Background

Despite interest in the importance of the home food environment and its potential influence on children’s diets and social norms, there remain few self-report checklist methods that have been validated against the gold standard of researcher-conducted inventories. This study aimed to assess the criterion validity and reliability of the ‘Home Food Availability Inventory Checklist’ (HFAI-C), a 39-item checklist including categories of fruit, vegetables, snacks and drinks.

Methods

The HFAI-C was completed by 97 participants of White and Pakistani origin in the UK. Validity was determined by comparing participant-reported HFAI-C responses to data from researcher observations of home food availability using PABAK and weighted kappa statistics. The validity of measuring the amount of items (in addition to presence/absence) available was also determined. Test-retest reliability compared repeated administrations of the HFAI-C using intra-class correlation coefficients.

Results

Validity and reliability was fair to moderate overall. For validity, the average category-level PABAK ranged from 0.31 (95 % CI: 0.25, 0.37) for vegetables to 0.44 (95 % CI: 0.40, 0.49) for fruits. Assessment of the presence/absence of items demonstrated higher validity compared to quantity measurements. Reliability was increased when the HFAI-C was repeated close to the time of the first administration. For example, ICCs for reliability of the measurement of fruits were 0.52 (95 %CI: 0.47, 0.56) if re-administered within 5 months, 0.58 (95 % CI: 0.51, 0.64) within 30 days and 0.97 (95 %CI: 0.94, 1.00) if re-administered on the same day.

Conclusions

Overall, the HFAI-C demonstrated fair to moderate validity and reliability in a population of White and South Asian participants. This evaluation is consistent with previous work on other checklists in less diverse, more affluent populations. Our research supports the use of the HFAI-C as a useful, albeit imperfect, representation of researcher-conducted inventories. The feasibility of collecting information using the HFAI-C in large, multi-ethnic samples can facilitate examination of home food availability in relation to exposures such as ethnicity and outcomes including behavioural, social and health outcomes. Future work using the HFAI-C could provide important insights into a modifiable influence with potential to impact health.

Background

Measuring the home food environment and understanding the potential differences in the availability of foods and drinks in homes of different cultures and populations permits a greater knowledge of the causes of energy over-consumption and may support the development of effective obesity prevention and treatment interventions. The majority of work in the area of home food availability has used self-report checklists in which parents/caregivers are asked to report whether foods are present or absent. Despite the increased interest in the importance of the home food environment in recent years [1–5], few researchers have validated self-report checklist methods against the gold standard of researcher-conducted exhaustive home food availability inventories and none within South Asian populations [3, 6–10]. Exhaustive researcher-conducted inventories of all foods in the home provide data that describe the total amounts of foods and nutrients available. However, this approach is rarely used due to the burden of collecting and analysing exhaustive data from within family homes [5].

As food selection is highly linked to culture and ethnicity, it is likely that the foods in the home differ in ethnically and socially distinct households. It is also possible that reporting of foods in the home differs by culture and ethnicity. As such, consideration is also required of the population characteristics when conducting a validation of home food availability checklists. Work investigating home food availability in minority populations has thus far included validity examination in African American, Somali and Hispanic populations [11, 12]. There remains a lack of evaluation of the validity and reliability of home food availability checklists in other populations, including those of South Asian origin.

The current study aimed to assess the validity and reliability of a brief 39-item home food availability checklist; the ‘Home Food Availability Inventory Checklist (HFAI-C)’ in a sample of White British and South Asian populations in the UK. As the precision to which a checklist can assess the amount of foods in the home (in contrast to assessment of their presence only) is not known, we also aimed to determine the degree to which a self-report checklist can accurately estimate the quantity of foods available in homes by comparing validity and reliability from dichotomous outcome data (i.e. food/drink items present or absent) to data from varying levels of quantity (i.e. creating quartiles and tertiles of the quantity of foods available).

Methods

Sample

Participants completing the checklist were recruited from ‘Born in Bradford 1000’ (BiB1000) [13]; a nested cohort study within the ‘Born in Bradford’ (BiB) study [14]. In brief, BiB is a longitudinal multi-ethnic birth cohort of 12,453 women who were recruited during pregnancy if they were registered to give birth in the Bradford Royal Infirmary (resulting in 13,776 babies). Participants were almost exclusively of White or South Asian ethnicity. This cohort aims to examine environmental, psychological and genetic factors that impact health and development perinatally and during childhood [14]. The BiB1000 nested population was not a random sample. Instead, all mothers recruited to the full BiB study between August 2008 and March 2009 who had completed the baseline questionnaire were approached to take part in BiB1000 during their routine 26–28 week glucose tolerance test (GTT) [13]. This nested cohort of 1736 mothers aimed to understand the predictors and influences of pregnancy and early life health-related behaviours specifically to inform the development of a culturally specific childhood obesity prevention intervention. BiB1000 characteristics are similar to that of the full BiB cohort [14]; with a similar distribution of age, ethnicity, and marital status (See Table 1). All BiB1000 participants were asked to complete the HFAI-C and a sub-sample of 100 mothers were opportunistically recruited to have researcher-conducted home food availability inventories conducted around the time of their 18 month BiB1000 assessment (i.e. all participants completing their 18 month assessment were invited to take part, with the first 100 consenting included in the study). Inventories were then scheduled to occur as soon as possible following the 18 month assessment.

Table 1 Demographic characteristics for analytic sample (n = 97)

There were no language restrictions for study eligibility and multi-lingual staff were trained to collect data in homes in which the parents were unable to speak English. All questionnaires were transliterated into Urdu and Mirpuri language, as the majority of Pakistani populations residing in Bradford are of Mirpuri origin and one of the official languages of Pakistan is Urdu.

This study was conducted according to the guidelines of the Declaration of Helsinki and all procedures involving human subjects/patients were approved by the Bradford Research Ethics Committee (07/H1302/112). Written or verbal (for mothers unable to read and/or speak English) informed consent was obtained from all participants. Verbal consent was witnessed and formally recorded.

Measures

Home food availability inventory checklist

The Home Food Availability Inventory Checklist (HFAI-C) was modelled after the Healthy Home Survey [10] and informed by data from a study of home food availability that used an exhaustive barcode scanning method [15]. These studies provided information on which foods were commonly available in the home in addition to providing a range of available sizes for each item. The checklist was intended for use in large cohort studies and therefore was designed so that it did not incur a high level of burden on participants. Foods and drinks were restricted to a list of 39-items that were available within the categories of 1. Fruits (16-items including fresh, dried and canned); 2. Vegetables (12-items including fresh and canned; 3. Snacks (7-items including savoury (e.g. salted nuts) and sweet (e.g. cake)); and 4. Drinks (4-items including regular and diet fizzy drinks, sports drinks and fruit drinks). These were chosen for study because; 1. they are often the target of obesity interventions [16], 2. there is some evidence that their intake is related to obesity in children [17–19] and/or, 3. early literature indicates a relationship between availability in the home and either diet [20, 21] or obesity [22].

The HFAI-C instructs participants to report the maximum availability of each food and drink item in their homes over the previous 7 days. Rather than using a dichotomous response option of ‘present’ or ‘absent’, four categories of response options were generated to enable an estimation of the quantity of foods within the home (absent, small amount, medium amount, large amount). These response options were not intended to precisely quantify the exact amount of foods, but to rank availability. A range of the quantities within each response option was provided based on the distribution of sizes of foods/packages that were previously available from our Universal Product Code (UPC) scanning study [15] and on the usual packaging for purchase.

The HFAI-C was administered as part of a larger questionnaire, which was completed by all BiB1000 participants during cohort assessments when infants were 18 and 36 months of age. For the purposes of the test re-test reliability analyses, checklist data from the 18-month assessment have been used (planned as close to the the time of the researcher-conducted inventorie as possible, but with any permitted). A full questionnaire pack was completed for each infant, so that mothers with multiple births were asked to complete the questionnaire pack multiple times (e.g. two times for twins). Data from three parents were incomplete and as such, the validation analysis sample consisted of 97 parents. Two of these 97 parents included twins but only one child was randomly allocated for comparisons between researcher completed and parent self-report checklists. Three samples were used for the reliability testing including: (1) Full sample, independent of timing between repeated administrations, excluding twins (n = 95); (2) Reduced sample with repeated measures conducted within 30 days, excluding twins (n = 30); and (3) Twin data taken from an additional BiB1000 sample in which parents were asked to complete separate questionnaires for both infants on the same visit (n = 15). This latter sample was included to explore reliability estimates that were independent of shopping and consumption behaviours. Since the HFAI-C was administered as part of a larger questionnaire pack, the timing between repeated administrations of the HFAI-C for parents with twins was approximately 1 h. This reduces the likelihood of parents simply copying responses from one administration to the other. Further, questionnaires were researcher administered and data were entered directly on to an electronic table by the researcher.

Researcher-conducted food availability inventories

Inventories were conducted when infants were approximately 18 months old using a standardised protocol based on well-established methodologies from previous research [10, 15, 23] in which researchers measured all foods from all food storage areas in participants’ homes within the higher categories of fruits (with sub-categories: fresh, tinned, dried and frozen), vegetables (with sub-categories fresh, tinned and frozen), snacks (with sub-categories: crisps/tortillas, biscuits, salted nuts, chocolate, sweets, cakes and ice-cream) and drinks (with sub-categories sugar-sweetened and sugar-free). Within each sub-category, open ‘exhaustive’ data were collected rather than using a pre-defined checklist of items. This method requires researchers to remove all items from one storage location at a time and only replace them after they have been recorded; ensuring that all relevant items are included. Exhaustive data from 836 food and drink items that were identified were grouped into 215 individual food and drink types. For example, a ‘packet of chocolate digestive biscuits’ was grouped as ‘biscuits with chocolate topping’ within the sub-category of ‘Biscuits/Sweet snacks’ (under the higher category of snacks). Similarly, all crisps that were made with corn were assigned to the group of ‘tortillas’ within ‘Crisps/Savoury snacks’ and ‘red grapes’ and ‘green grapes’ were grouped as ‘grapes’ within the sub-category of fresh fruits (under the higher category of fruits). For the purpose of these analyses, 1 handful of fresh produce represented 1 serving/cup. Other fresh produce that were recorded as whole units (e.g. melons) were converted to the number of adult size servings by a nutritionist (MB) using standards provided by Self Nutrition Data (http://nutritiondata.self.com/facts) and USDA National Nutrient Database for Standard Reference Release 27 (http://ndb.nal.usda.gov/ndb/foods). A repeated administration of the HFAI-C was also completed by participants in the nested sample of 100 homes on the same day as the researcher-conducted inventories.

Other measures

Demographic data were obtained from the full BiB cohort during recruitment by self-report (26–28 weeks of pregnancy) [13, 14]. Measures relevant to these analyses include: household structure, marital status, educational status (as a proxy for socio-economic status), and ethnicity.

Analysis

Criterion validity of the HFAI-C was assessed using the researcher-conducted inventory as the gold standard, with the individual checklist questions matched to food/drink item(s) from the researcher-conducted inventories. Agreement was assessed using kappa statistics, sensitivity (proportion of homes with a given food/drink identified as present by the researcher that were correctly identified as present by the participant) [24] and specificity (proportion of homes with a given food/drink identified as absent by the researcher that were correctly identified as absent by the participant) [8]. Kappa statistics were calculated for responses grouped into two (items present/absent), three (items absent or available in small and medium/large amounts), and four (items absent or available in small, medium and large amounts) categories. The medium and large response categories were combined for the tertile categorization given the small number of responses in the large category.

To adjust for differences in response prevalence and for bias between the researcher and participant, prevalence- and bias- adjusted kappa (PABAK) [24] was calculated in addition to Cohen’s kappa [25] for the dichotomized responses. Cicchetti-Allison linear-weighted kappa [26] was used to examine concordance between the reported and observed responses with the data divided into tertiles and quartiles Weights (w) were thus assigned according to the participant category (i) and the researcher category (j), such that: w ij  = 1 – [|C i – Cj|/(C C – C 1)], where C i is the score for i, C j is the score for j, Cc is the score for category C (the number of categories), and C 1 is the score for the first category. For these analyses, the number of categories equalled either four (quartiles) or three (tertiles). Scores were assigned to each closed-ended category using the median of the range in each respective category. The response category indicating the largest amount was open-ended. For this category, we assigned a score that was 1.5 times the total range of the other responses. As opposed to PABAK, the weighted kappa approach allows for differential weighting according to the level of disagreement between reported and observed responses.

Single-measures intra-class correlation coefficients (ICC) [27] were used to examine test-retest reliability across parent completion of the HFAI-C within 5 months, 30 days and repeated on the same day. Two-way random effects models with measures of absolute agreement [28] were used to assess the dichotomized responses using the INTRACC macro created by Hamer for SAS [29]. For a more detailed account of psychometric methods, see Nunnally [30].

Category-level kappa statistics and ICCs were assessed by collapsing all checklist and researcher responses for all items in a given food/drink category into two variables: an HFAI-C response and a researcher response. Corresponding confidence intervals were obtained using bootstrap resampling method by sampling with replacement from the sub-cohort of 100 mothers 1000 times and then calculating the appropriate statistics on each resulting dataset. The 2.5 and 97.5 percentiles of the distributions of kappa statistics were reported as the bounds of the confidence intervals.

For Cohen’s kappa and weighted kappa interpretation purposes, we used the guidelines proposed by Landis and Koch: < 0.00 = poor, 0.00 – 0.20 = slight, 0.21 – 0.40 = fair, 0.41 – 0.60 = moderate, 0.61 – 0.80 = substantial, and 0.81 – 1.0 = almost perfect [31]. PABAK statistics have been similarly interpreted in the context of existing literature (akin to kappa statistics), although it is recognised that descriptive terms like ‘moderate’ and ‘fair’ are not always universally accepted. Because Cohen’s kappa is mathematically equivalent to the weighted kappa in the case of only two categories, Cohen’s kappa as opposed to PABAK was used for comparison purposes. ICCs were interpreted according to guidelines proposed by Shrout: <0.10 = virtually none, 0.11 – 0.40 = slight, 0.41 – 0.60 = fair, 0.61 – 0.80 = moderate, and 0.81 – 1.0 = substantial [32]. All above analyses were repeated for the White British and Pakistini origin subgroups to explore potential differences by ethnicity. All statistical analyses were performed using SAS version 9.4 (SAS Institute, Inc., Cary, North Carolina).

Results

Sample characteristics

Table 1 presents demographic data collected at baseline from the 97 participants, who had complete HFAI-C data. The analytic sample was predominately of White British (47 %) or Pakistani origin (42 %). A high proportion of mothers were between the ages of 25 and 29 years (38 %) and were married (76 %). The level of educational attainment was diverse, with 14 % having not completed any education and 32 % having completed some university education. At least 2 persons (of all ages) lived in nearly all households (98 %). These characteristics are similar to those of the full BiB cohort (e.g. which has 45 % participants of Pakistani origin) [14].

Criterion validity

The criterion validation results are shown for individual food/drink items and by the four food/drink categories (fruits, vegetables, snacks, and drinks) for the dichotomized responses (presence/absence) (Table 2). Items within each category were ordered in descending order according to participant-reported prevalence. Among fruits and vegetables, participants most commonly reported apples, bananas, and carrots as being present over the previous week. For snacks and drinks, participants most commonly reported that biscuits and fizzy drinks had been available respectively. Across all food/drink items, sensitivity (range: 0.49–1.00) tended to be higher than specificity (range: 0.14–0.88).

Table 2 Criterion validity comparing researcher administered and participant reported data for analytic sample using dichotomized responses (presence/absence) (n = 97)

PABAK statistics were relatively similar across the four food/drink categories and indicated fair to moderate performance of the HFAI-C at the category level, with PABAKs ranging from 0.31 (95 % CI: 0.25, 0.37) for vegetables to 0.44 (95 % CI: 0.40, 0.49) for fruits. PABAK values were lowest for greens (PABAK -0.09, 95 % CI: -0.19, 0.02) and highest for grapefruit (PABAK 0.77, 95 % CI: 0.71, 0.83).

Table 3 shows the distribution of mean participant and researcher responses as well as the weighted kappa statistics for the four food/drink categories using the two, three, and four quantities of availability. Prevalence as reported by participants tended to be higher than that reported by researchers, except for the large category and for medium drinks. Kappa statistics for the dichotomized responses were generally greater than those found when items were recorded in four quantities of availability (except for vegetables). Additionally, the use of three quantities of availability for the fruit and vegetable categories was more valid than the use of dichotomized responses. The validity of the HFAI-C was fair to moderate across the two, three, and four quantities of availability. However, the validity of the drink category was only fair when the three and four quantities of availability were used, and the validity of the snack category was only fair when the four quantities of availability were used.

Table 3 Criterion validity comparing researcher administered and participant reported data for analytic sample using 2, 3, and 4 response categorizationsa (n = 97)

The potential impact of ethnicity was examined by comparing validity data between homes of White British and Pakistani origin. Results were similar between groups with a few exceptions: Agreement was higher for fruits in the White British (PABAK 0.58, 95 % CI 0.53, 0.64) compared to the Pakistani (PABAK 0.32, 95 % CI 0.24, 0.39). Conversely, homes with participants of Pakistani origin had higher validity in the reporting of the quantity of vegetables (PABAK 0.34, 95 % CI: 0.28, 0.42) compared to home with White British participants (PABAK 0.31, 95 % CI: 0.22, 0.39) (See Additional file 1).

Test-retest reliability

Results from the reliability analyses are presented in Table 4. ICCs are presented only for the dichotomized responses. When all participants with HFAI-C data available at both time points (with repeated completion within 5 months) were included in the analysis, ICCs at the category level indicate that agreement between the two time points was slight to fair, with ICCs ranging from 0.38 (95 % CI 0.29, 0.48) for drinks to 0.52 (95 % CI 0.47, 0.56) for fruits. For individual items, ICCs ranged from 0.14 (for ‘other fresh vegetables’) to 0.61 (for grapes and canned tomatoes). Restricting the sample to those who completed both assessments within a 30-day time period improved the reliability such that agreement at the category level was fair to moderate, with ICCs ranging from 0.48 (95 % CI 0.34, 0.60) for drinks to 0.58 (95 % CI 0.51, 0.64) for fruits. The ICCs for all sets of twins in BiB1000 (where both questionnaires were completed on the same day) had the smallest range of ICCs at the item level, with only four items, (tinned fruits (water/juice and syrup), grapes, and celery resulting) in ICCs less than 1.00. Agreement was substantial for all categories among the sample of twins.

Table 4 Test re-test reliability for full (n = 95), 30-day restricted (n = 43), and twin (n = 15) samples using dichotomized responses (presence/absence)

Discussion

On the whole, the validity and reliability of the 39-item HFAI-C was moderate, with high levels of sensitivity and reliability and fair to moderate agreement between participant- and researcher- collected data. The instrument was more accurate in the assessment of the presence of foods in the home compared to determining the amount available, although this had little impact on the overall validity results here. Our evaluation also indicated that the HFAI-C may be appropriate for use in households of both Whites and South Asians living in Britain.

The level of agreement we found in item-level validation is consistent with previous literature in which researcher conducted exhaustive inventories were used as a gold standard comparator [6–9, 33]; however, comparison of mean category level validation values with other studies is difficult, as others have not conducted a bootstrapping method to enable category level means with confidence intervals. Furthermore, not all previous studies have adjusted analyses and determined PABAK statistics. Boles and colleagues validated a 126-item checklist of foods and drinks in 25 homes and reported kappa values ranging from 0.03 to 1.00 for fruits and vegetables [33]. Mean category level kappas were not provided. Similarly, a validation study by Marsh et al. [9] presented item level Cohen’s kappa statistics for 31-items within the categories of fruit juices, fruits and vegetables. Ranges within categories were similar to PABAK results of the current study; with kappa values for fruit juices ranging from 0.24–0.53; kappa values for fruits at 0.12–0.76 (compared to our PABAK values ranging from 0.17–0.77); and kappa values for vegetables between 0.19–0.66 (compared to our PABAK values ranging from 0.01–0.66). These ranges fit into similar boundaries of acceptability (e.g. the majority of fruit items from both studies were described as having at least moderate validity (0.41–0.60). However, caution should be applied when comparing adjusted PABAK findings with kappa statistics as adjusted analyses using PABAK resulted in higher levels of agreement in all studies, including the present study.

In a further validation study of the Healthy Home Survey by Bryant et al. [10] mean PABAK values for categories were provided, although confidence intervals were not. This study differed in the type of data collected as, participants reported all items within 6 food and drink categories (i.e. using an exhaustive ‘within category’ approach rather than a checklist). Mean PABAK values were higher than the current study (0.85 and 0.85 for fresh fruits and vegetables respectively) although these were simple means and did not include category level variables. Confidence intervals were generated for the current study to provide a description of the distribution of PABAK data. These results indicated a substantial variation between item validation results and mean confidence interval values ranged from -0.02 to 0.87. This suggests that the ability of the checklist to accurately reflect what was found in homes by researchers varied between households. Other validation studies including those by Crockett et al. [6], Miller and Edwards [7] and Fulkerson et al. [8] report similar sensitivity and specificity results to those of the HFAI; however Kappa values were higher. This might be explained by the differences in study design as, instead of comparing checklist data to exhaustive researcher conducted inventories, these other studies validated their participant completed checklists by comparing data to repeated administration of the checklists by researchers.

Test-re-test data suggest that the HFAI-C demonstrated fair reliability, particularly if the two repeated administrations were within 30 days of each other. Given the high probability that there were actual changes to foods in the home over this period, the relatively lower reliability with up to 5 months between measurements may actually indicate the transient nature of the environment and acts as a measure of stability rather than consistency. Conversely, when administrations occurred in the same day (i.e. by parents with twins), the higher reliability estimates suggested that the tool has substantial consistency.

It is a limitation that our validity data may have been impacted by differences in the time referent of the HFAI-C compared to the researcher-conducted inventory. Whereas the HFAI-C required participants to report the maximum amount of each item over the past 7 days, researchers conducted inventories on a single day. Given this difference, it is not surprising that, for the most part, prevalence as reported by participants was higher than that reported by researchers. This issue has been reported previously by Marsh et al., whose checklist required participants to report over the previous 7 days, whereas researcher conducted inventories were on a single day [9]. These authors theorised that the variability in observed agreement statistics may therefore be related to the perishable nature of some foods. However, kappa and PABAK statistics in the current study were similar between perishable and non-perishable items. Evidence from a small study with 9 households, each measured five times, indicates high intra-monthly changes in availability due to purchasing and consumptions behaviours [34]. However, in larger studies home food availability has been shown to be relatively stable, as the within household variation of home food availability is relatively low compared to between household variability [35]. Intuitively, this is logical, as households tend to purchase similar items over time. In fact, research in this area suggests that, only one observation per household may be required to obtain a correlation between r = 0.7–0.9 with the true within household mean when assessing the number of items available [35]. Nevertheless, it is likely that the absolute maximum amount of foods reported amount over 7 days will differ from that in a single day within a household, especially with fresh produce. This may be particularly important in low-income households who are less likely to have car access enabling them to shop at larger retailers [36].

It is a strength of the current study that we assessed the quantity of foods and drinks that were available. We found that quantitative results were generally less valid than the identification of the presence or absence of items. However, these differences had little impact on the overall validity results. Measurement of the presence/absence of food items in the home is the most common approach used in other checklists of home food availability, perhaps because of feasibility. Data from this study may indicate that such an approach is most appropriate for the assessment of home food availability when using a self-report checklist. It has been argued that the quantity of foods available provides a richer picture of the environment and would better discriminate between households providing a ‘healthy’ compared to a ‘non-healthy’ environment [10]. Further research is required in this area as few checklists assess any level of quantity other than presence or absence.

It is a strength that this study included participants with South Asian cultural identities. Given the large role of culture in food selection types of foods and drinks available in the home are likely to be impacted and differences were anticipated between White and South Asians in our sample. In the checklist development stage, data from 24-h dietary recalls taken from South Asian populations in Bradford were examined to identify foods or drinks that were consumed regularly that were missing from the checklist. Instead of identifying completely new items, this process identified prompts and examples that needed to be added. For example, under ‘other vegetables’, ‘okra’ was included as one of the examples. Providing a range of ethnically appropriate examples or prompts for measures is therefore one way that a tool may be developed or modified for use in mixed population groups. Other approaches to make tools more culturally have been suggested by others including working with the target populations to modify existing tools (i.e. delete items, add items, or altering categories) or by applying modifications (i.e. additions) to produce a tool appropriate for use in a mixed population [12, 37, 38]. The latter approach ensures appropriate items are included, but increases the length of checklists, which may be less feasible to administer. Administration of the HFAI-C in other populations should involve working with members of the intended population to modify the prompt and examples provided without increasing the number of items.

It is possible that the validity of such measures may be impacted by cultural or social identity (for example, participants of one ethnicity may be more accurate at reporting availability than others using the checklist). Our investigation of this identified similarities in validity and reliability by ethnicity, except that agreement between participant and researcher reported fruit availability was greater for White British participants compared to those of Pakistani origin, but that participants of Pakistani origin had better validity for reporting the quantity of vegetables in their homes. This study was not statistically powered to identify differences by ethnicity, but analyses were conducted as a means to explore potential disparities in the validity and reliability. Homes with participants of Pakistani origin from this cohort have a greater amount of fruits available compared to those of White British origin [23], which may be one possible explanation for differences in the ability to accurately measure them; as the measurement of fresh produce is more likely to be impacted by shopping and consumption behaviours [36] Further research may be warranted to explore potential cultural or ethnic difference in reporting accuracy, in addition to understanding the impact of statistical adjustment of other potential covariates [35].

It has been argued that the use of checklists compared to researcher-conducted exhaustive inventories is somewhat analogous to differences between assessment of diet using food frequency questionnaires (FFQs) and food diaries or histories [35]. The former are restricted to the items that are to be relevant to the investigators’ hypotheses, with the latter able to collect an exhaustive or complete account. FFQs are useful in large epidemiological studies in which detailed dietary measurement is not feasible or required, and the data ranks individuals moderately well along a distribution of intake, but absolute amounts are not well-quantitated [39]. Similarly, a home food availability checklist is a less time intensive method that is likely to be more feasible in large epidemiological studies, and it can be useful if ranking individuals on pre-determined types of foods is adequate to answer the study questions. Research has attempted to quantify the absolute amount of nutrients available in the home using barcode scanners [15, 22]. This method requires a researcher or the participant to scan all foods and drinks in their homes and is labour intensive. More recently, the ability to quantify nutrient availability in the home has been tested by comparing detailed home food inventory data with data gathered from existing Food Frequency Questionnaires (Block Dietary Fat Screener and Block Fruit–Vegetable–Fiber Screener) [40]; with observed high correlations between the two (e.g. r = 0.76–0.78, p < 0.01 for the Fruit-Vegetable-Fiber Screener). Annotated grocery receipts may also be used to quantify home food availability [41] although this method relies on detailed receipts (or participant reporting on ‘vague’ receipts, ignores foods such as those gathered from gardens and its analysis and interpretation can be labour intensive). Methods that do not rely on self-reported data, including researcher conducted inventories, are most likely to provide the most valid assessment; however these are often less feasible especially in larger studies.

This study adds to the literature on the measurement of home food availability by providing an evaluation of a brief checklist in relatively large a bi-ethnic sample, that can be used to rank the availability of foods and drinks (including an estimation of quantity) that are hypothesised to relate to childhood obesity. An important strength of the current study is its inclusion of a diverse ethnic population with high levels of deprivation. The HFAI-C has also been administered to a wider sample of 1700 families within the BiB1000 cohort, which benefits from the inclusion of a large amount of other assessments [42]. This will allow further examination of home food availability in relation to exposures such as ethnicity and outcomes including behavioural, social and health outcomes; thus providing further insight into how the home food environment may relate to obesity and health related behaviours. One limitation of the validity of the tool might relate to confusion in the interpretation on some of the items. Items with the lowest validation results were ‘greens’ and ‘other fresh vegetables’. It is likely that lower values for these items related to uncertainties in their definitions as we have since learnt that this differs between individuals. Cognitive testing of the HFAI-C was not conducted due to its checklist nature, although issues related to interpretation may have been alleviated if cognitive testing was conducted. Further implementation of the HFAI-C should therefore provide a clearer definition of these items. Like other checklists, the HFAI-C is limited by the foods that are include in the list and therefore cannot assess the overall availability of foods in the home. Lastly, although the variance in the timing between administrations for the reliability study permitted a greater understanding of the impact of time on measurement, estimates may have been improved if data were gathered from a larger sample of participants in which data were collected within 30 days or even 7 weeks of one another.

Conclusion

The HFAI-C is a simple checklist that can be used to assess the availability of foods in the home, in addition to estimating the amounts available. Similar to other home food availability checklists, the HFAI-C demonstrated fair to moderate validity and reliability, with high variability between items. More research is recommended in this area to confirm this hypothesis or to improve on the validity of the measurement of quantity. Future work should also focus on the ability of the checklist to discriminate between participant characteristics such as obesity and to evaluate the impact of home food availability on behaviours such as dietary intake.

References

  1. Cullen KW, Baranowski T, Owens E, Marsh T, Rittenberry L, de Moor C. Availability, accessibility, and preferences for fruit, 100 % fruit juice, and vegetables influence children’s dietary behavior. Health Educ Behav. 2003;30(5):615–26.

    Article  PubMed  Google Scholar 

  2. Arcan C, Hannan PJ, Fulkerson JA, Himes JH, Rock BH, Smyth M, et al. Associations of home food availability, dietary intake, screen time and physical activity with BMI in young American-Indian children. Public Health Nutr. 2013;16(01):146–55.

    Article  PubMed  Google Scholar 

  3. Boles RE, Scharf C, Filigno SS, Saelens BE, Stark LJ. Differences in Home Food and Activity Environments between Obese and Healthy Weight Families of Preschool Children. J Nutr Educ Behav. 2013;45(3):222–31.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Ding D, Sallis JF, Norman GJ, Saelens BE, Harris SK, Kerr J, et al. Community Food Environment, Home Food Environment, and Fruit and Vegetable Intake of Children and Adolescents. J Nutr Educ Behav. 2012;44(6):634–8.

    Article  PubMed  Google Scholar 

  5. Bryant M, Stevens J. Measurement of food availability in the home. Nutr Rev. 2006;64(2):67–76.

    Article  PubMed  Google Scholar 

  6. Crockett SJ, Potter JD, Wright MS, Bacheller A. Validation of a self-reported shelf inventory to measure food purchase behavior. J Am Diet Assoc. 1992;92(6):694.

    CAS  PubMed  Google Scholar 

  7. Miller C, Edwards L. Development and validation of a shelf inventory to evaluate household food purchases among older adults with diabetes mellitus. J Nutr Educ Behav. 2002;34(5):261–7.

    Article  PubMed  Google Scholar 

  8. Fulkerson J, Nelson M, Lytle L, Moe S, Heitzler C, Pasch K. The validation of a home food inventory. Int J Behav Nutr Phys Act. 2008;5(1):55.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Marsh T, Cullen KW, Baranowski T. Validation of a fruit, juice and vegetable availability questionnaire. J Nutr Educ Behav. 2003;35:100–4.

    Article  Google Scholar 

  10. Bryant M, Ward D, Hales D, Vaughn A, Tabak R, Stevens J. Reliability and validity of the Healthy Home Survey: A tool to measure factors within homes hypothesized to relate to overweight in children. Int J Behav Nutr Phys Act. 2008;5(23):5868–23.

    Google Scholar 

  11. Santiago-Torres M, Adams AK, Carrel AL, LaRowe TL, Schoeller DA. Home food availability, parental dietary intake, and familial eating habits influence the diet quality of urban Hispanic children. Child Obes. 2014;10(5):408–15.

    PubMed  PubMed Central  Google Scholar 

  12. Hearst MO, Fulkerson JA, Parke M, Martin L. Validation of a home food inventory among low-income Spanish- and Somali-speaking families. Public Health Nutr. 2013;16(07):1151–8.

    Article  PubMed  Google Scholar 

  13. Bryant M, Santorelli G, Fairley L, West J, Lawlor DA, Bhopal R, et al. Design and characteristics of a new birth cohort to study the early origins of childhood obesity: the BiB1000 study. Longitudinal Life Course Stud. 2013;4(3):119–35.

    Google Scholar 

  14. Wright J, Small N, Raynor P, Tuffnell D, Bhopal R, Cameron N, et al. Cohort profile: The Born in Bradford multi-ethnic family cohort study. Int J Epidemiol. 2012;42(4):978–91.

    Article  PubMed  Google Scholar 

  15. Stevens J, Bryant M, Wang L, Borja J, Bentley ME. Exhaustive measurement of food items in the home using a universal product code scanner. Public Health Nutr. 2010;DOI: 10.1017/S1368980010001837

  16. Ogen J. Health Psychology: A textbook. 3rd ed. Maidenhead: McGraw-Hill; 2004.

    Google Scholar 

  17. Resnicow K, Hearn MD, Smith M, Baranowski T, Lin L, Baranowski J, et al. Social-Cognitive predictors of fruit and vegetable intake in children. Health Psychol. 1997;16:272–6.

    Article  CAS  PubMed  Google Scholar 

  18. Malik VS, Schulze MB, Hu FB. Intake of sugar-sweetened beverages and weight gain: a systematic review. Am J Clin Nutr. 2006;84(2):274–88.

    CAS  PubMed  PubMed Central  Google Scholar 

  19. Nicklas TA, Yang S-J, Baranowski T, Zakeri I, Berenson G. Eating patterns and obesity in children: The Bogalusa Heart Study. Am J Prev Med. 2003;25(1):9–16.

    Article  PubMed  Google Scholar 

  20. Jago R, Page A, Froberg K, Sardinha LB, Klasson-Heggebø L, Andersen LB. Screen-viewing and the home TV environment: The European Youth Heart Study. Prev Med. 2008;47(5):525–9.

    Article  PubMed  Google Scholar 

  21. van der Horst K, Oenema A, Ferreira I, Wendel-Vos W, Giskes K. A systematic review of environmental correlates of obesity-related dietary behaviors in youth. Health Educ Res. 2006;22:203.

    Article  PubMed  Google Scholar 

  22. Byrd-Bredbenner C, Maurer Abbot J. Differences in Food Supplies of U.S. Households with and without Overweight Individuals. Appetite. 2009;52:479–84.

    Article  PubMed  Google Scholar 

  23. Bryant M, Sahota P, Santorelli G, Hill A. An exploration and comparison of food and drink availability in homes in a sample of families of White and Pakistani origin within the UK. Public Health Nutr. 2015;18(7):1197–205. doi: 10.1017/S1368980014000147

  24. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423–9.

    Article  CAS  PubMed  Google Scholar 

  25. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46.

    Article  Google Scholar 

  26. Cicchetti D, Allison T. A new procedure for assessing reliability of scoring eeg sleep recordings. Am J EEG Technol. 1971;11:101–9.

    Google Scholar 

  27. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–8.

    Article  CAS  PubMed  Google Scholar 

  28. McGraw KO, Wong SP. Forming Inferences About Some Intraclass Correlation Coefficients. Psychol Methods. 1996;1(1):30–46.

    Article  Google Scholar 

  29. Hamer RM. A SAS macro for computing intraclass correlation coefficients. In: Robert M. Hamer: Associate Professor of Psychiatry and Biostatistics VCU, editor.: Copyright (C) 1990, all rights reserved.; 1991.

  30. Nunnally JC. Psychometric Theory. 25 Years Ago and Now. Educ Res. 1975;4(10):7–21.

    Google Scholar 

  31. Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.

    Article  CAS  PubMed  Google Scholar 

  32. Shrout PE. Measurement reliability and agreement in psychiatry. Stat Methods Med Res. 1998;7(3):301–17.

    Article  CAS  PubMed  Google Scholar 

  33. Boles RE, Burdell A, Johnson SL, Gavin WJ, Davies PL, Bellows LL. Home food and activity assessment. Development and validation of an instrument for diverse families of young children. Appetite. 2014;80:23–7.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Sisk C, Sharkey J, McIntosh W, Anding J. Using multiple household food inventories to measure food availability in the home over 30 days: a pilot study. Nutr J. 2010;9(1):19.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Stevens J, Bryant M, Wang C-H, Cai J, Bentley ME. Sample Size and Repeated Measures Required in Studies of Foods in the Homes of African-American Families. J Nutr. 2012;142(6):1123–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Gustat J, O’Malley K, Luckett BG, Johnson CC. Fresh produce consumption and the association between frequency of food shopping, car access, and distance to supermarkets. Prev Med Reports. 2015;2:47–52.

    Article  Google Scholar 

  37. Bauer KW, Widome R, Himes JH, Smyth M, Rock BH, Hannan PJ, et al. High food insecurity and its correlates among families living on a rural American Indian Reservation. Am J Public Health. 2012;102(7):1346–52.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Patterson R, Kristal A, Shannon J, Hunt J, White E. Using a brief household food inventory as an environmental indicator of individual dietary practices. Am J Public Health. 1997;87:272–5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr. 2002;5(4):567–87.

    Article  PubMed  Google Scholar 

  40. Martin-Biggers J, Koenings M, Quick V, Abbot JM, Byrd-Bredbenner C. Appraising nutrient availability of household food supplies using Block Dietary Screeners for individuals. Eur J Clin Nutr. 2015;69(9):1028–34.

    Article  CAS  PubMed  Google Scholar 

  41. French SA, Shimotsu S, Wall M, Gerlach AF. Capturing the Spectrum of Household Food and Beverage Purchasing Behavior: A Review. J Am Diet Assoc. 2008;108:2051–8.

    Article  PubMed  Google Scholar 

  42. Bryant M, Stevens J, Wang L, Tabak R, Borja J, Bentley ME. Relationship between home fruit and vegetable availability and infant and maternal dietary intake in African-American families: evidence from the exhaustive home food inventory. J Am Diet Assoc. 2011;111(10):1491–7.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

We are grateful to all the families who took part in this study, to the midwives for their help in recruiting them, the paediatricians and health visitors and to the Born in Bradford team which included interviewers, data managers, laboratory staff, clerical workers, research scientists, volunteers and managers.

This article presents independent research commissioned by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research funding scheme (RP-PG-0407-10044). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Maria Bryant.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

MB designed the research, trained the data collection staff, provided oversight in data collection, interpreted results and led the manuscript preparation. ML conducted the analysis, interpreted the data and helped draft the manuscript. PS contributed to the study design, data interpretation and reviewed the manuscript. JC supported the data analysis methodology, interpreted the data and reviewed the manuscript. JS contributed to the design, analysis plan, interpretation and manuscript preparation. All authors read and approved the final manuscript.

Additional file

Additional file 1: Table S1.

White British (WB, n = 47) and Pakistani (P, n = 40) criterion validity for analytic sample using 2, 3, and 4 response categorizations* (n = 97). Table S2. White British (WB) and Pakistani (P) reliability for full (n = 95), 30-day restricted (n = 43), and twin (n = 15) samples using dichotomized responses (presence/absence). Table S3. Comparison of weighted kappa statistics for 2, 3, and 4 response categorizations using simple averaging vs. category-level variables. Table S4. Comparison of ICC for simple averages vs. category-level variables. (DOCX 18 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bryant, M., LeCroy, M., Sahota, P. et al. Validity and reliability of the semi-quantitative self-report Home Food Availability Inventory Checklist (HFAI-C) in White and South Asian populations. Int J Behav Nutr Phys Act 13, 56 (2016). https://0-doi-org.brum.beds.ac.uk/10.1186/s12966-016-0381-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12966-016-0381-y

Keywords