- Open Access
Adding team-based financial incentives to the Carrot Rewards physical activity app increases daily step count on a population scale: a 24-week matched case control study
International Journal of Behavioral Nutrition and Physical Activity volume 17, Article number: 139 (2020)
Mobile health applications (mHealth apps) targeting physical inactivity have increased in popularity yet are usually limited by low engagement. This study examined the impact of adding team-based incentives (Step Together Challenges, STCs) to an existing mHealth app (Carrot Rewards) that rewarded individual physical activity achievements.
A 24-week quasi-experimental study (retrospective matched pairs design) was conducted in three Canadian provinces (pre-intervention: weeks 1–12; intervention: weeks 13–24). Participants who used Carrot Rewards and STCs (experimental group) were matched with those who used Carrot Rewards only (controls) on age, gender, province and baseline mean daily step count (±500 steps/d). Carrot Rewards users earned individual-level incentives (worth $0.04 CAD) each day they reached a personalized daily step goal. With a single partner, STC users could earn team incentives ($0.40 CAD) for collaboratively reaching individual daily step goals 10 times in seven days (e.g., Partner A completes four goals and Partner B completes six goals in a week).
The main analysis included 61,170 users (mean age = 32 yrs.; % female = 64). Controlling for pre-intervention mean daily step count, a significant difference in intervention mean daily step count favoured the experimental group (p < 0.0001; ηp2 = 0.024). The estimated marginal mean group difference was 537 steps per day, or 3759 steps per week (about 40 walking min/wk). Linear regression suggested a dose-response relationship between the number of STCs completed (app engagement) and intervention mean daily step count (adjusted R2 = 0.699) with each new STC corresponding to approximately 200 more steps per day.
Despite an explosion of physical activity app interest, low engagement leading to small or no effects remains an industry hallmark. In this paper, we found that adding modest team-based incentives to the Carrot Rewards app increased mean daily step count, and importantly, app engagement moderated this effect. Others should consider novel small-teams based approaches to boost engagement and effects.
The health benefits of physical activity (PA) are irrefutable and yet, widespread inactivity persists [1, 2]. Interventions that increase PA on population-levels are needed to help tackle this important public health issue . As smartphone ownership increases (approaching 90% in the US) , so has the number of mobile health applications (mHealth apps) in the major app stores (over 325,000; 30% are PA apps) . Part of the appeal of mHealth apps is their potential to reach large populations at relatively low cost . Their effectiveness, however, is often limited by low user engagement with 90% of mHealth apps being deleted within 30 days [7,8,9]. A still limited number of RCT studies suggest a dose-response relationship exists between engagement and effectiveness, with greater app use associated with larger PA improvements [10,11,12]. Conversely, waning engagement has been linked with declining PA [10, 11, 13,14,15]. Recent advances in behavioural science provide a new framework from which to develop practical solutions to address this notorious mHealth app engagement problem.
Behavioural economics, a branch of economics shaped by insights from psychology [16,17,18], has stimulated renewed interest in financial health incentive interventions, such as rewarding people to walk more . This increasingly common intervention is grounded in a behavioural economics concept called “present bias” which describes how individuals tend to place disproportionate emphasis on the present “cost” of a health behaviour (e.g., time) while discounting the future “benefits” of that behaviour (e.g., increased health) . Behavioural economics suggests that providing timely financial incentives for behaviours with benefits that are largely delayed (e.g., PA) may encourage individuals to choose to engage in those behaviours rather than put them off [16, 18, 20]. Individual-level financial incentives for PA (e.g., incentives for personal PA achievements) have shown promising results. A recent meta-analysis of RCTs concluded that financial incentives increased PA in the short-term by up to 4000 steps per day, with some evidence of long-term (six or more months) and sustained (after incentives withdrawn) effects . A number of studies also suggest team-based incentives (e.g., incentives for group achievements) may be efficacious as well . Compared to individual incentives, team incentives have yielded better gym attendance, more PA, and greater weight loss in RCT settings [23,24,25]. Interestingly, Patel et al. (2018) found that combining individual and team (i.e. combined) incentives was more efficacious than individual or team incentives alone .
The notion of “aligning the thoughts or behaviours of individuals in a group” is another pertinent behavioural economics concept called “herd behaviour” . Herd behaviour describes how individuals are more likely to follow others in decision making instead of making independent decisions (e.g., “My friend is going for a walk, so I probably should too.”) . The tendency for humans to want to behave in ways that are consistent with the people in their social networks may be leveraged in an mHealth context, for instance: (a) by providing feedback on peers’ progress, and/or (b) with team-based incentives. Recent evidence, though, suggests that adding a social component to mHealth interventions does not necessarily translate into positive effects [7, 11, 28, 29]. It appears that mHealth features designed to increase social connectivity among participants with no prior relationship do not work as well as those delivered among people with existing relationships (e.g., work colleagues challenge each other in an online walking challenge) [7, 26, 29, 30]. Babcock et al. (2015) compared anonymous partners to partners with an existing social connection and found PA incentives were not as effective in the anonymous group, highlighting the importance of leveraging pre-existing social connections in mHealth interventions [24, 26, 30].
Despite their popularity, very little is known about the effectiveness of commercial PA apps (or their design features) since few have undergone rigorous peer-reviewed evaluation [10, 31]. Among the 15 studies included in the recent Petersen et al. (2019) review of PA apps, for example, only five examined commercially available ones (e.g., Fitbit, ‘Zombie, Run!’) despite there being over roughly 100,000 in the major app stores [31, 32]. Among these five, little consideration was given to the role of engagement as an effect moderator despite suggestions that intervention exposure is imperative and that greater engagement usually yields larger effects . The Carrot Rewards app was a top tier Canadian app (i.e. 1.3+ million downloads, 500,000+ monthly active users (MAUs) as of May 2019) that rewarded users with loyalty points redeemable for consumer goods (e.g., gas, movies) for walking more. It was developed in partnership with the Public Health Agency of Canada as part of its Multi-Sectoral Partnership Approach to Healthy Living and Chronic Disease Prevention . One of the stated objectives of the initiative was to conduct rigorous evaluations of the app intervention, including the impact of new features [12, 35]. In March 2018, Carrot Rewards launched a new social feature called ‘Step Together Challenges’ (STCs) to complement their existing walking program (called ‘Steps’) where individualized daily step goal achievements were rewarded with very small incentives ($0.04 CAD per day). STCs allowed users to invite a friend from their existing social network to participate in a collaborative walking challenge for bonus incentives ($0.40 CAD per week).
To enhance our understanding of mHealth interventions, and acknowledging how difficult it can be to conduct RCTs in fast paced commercial digital environments, non-RCT alternatives (e.g., quasi-experimental designs) have been recommended [31, 36, 37]. Quasi-experimental evaluations of “top tier” commercial apps (i.e. the top 2% of apps reporting more than 500,000 MAUs)  may provide particularly valuable insight into mHealth app engagement, it’s role in promoting health behaviours, and how it can be improved on a population scale. The primary objective of this study, then, was to examine the impact of adding team incentives to the Carrot Rewards app on mean daily step count. An important secondary objective was to determine whether a dose-response relationship existed between app engagement (i.e. STCs completed) and mean daily step count.
Study design and sample
A 24-week retrospective pre-post matched pairs design was used to examine the effect of adding STCs to the Carrot Rewards ‘Steps’ walking program on mean daily step count. Participants were drawn from the existing Carrot Rewards user base which included Canadians 13 years of age or older living in the three provinces the app was launched (i.e. British Columbia (BC), Newfoundland and Labrador (NL), Ontario (ON)). All participants had to have opted-into the ‘Steps’ walking program to be included in the study. The experimental group included participants using the STC feature for the first time between March 19 and April 16, 2018 (the first month STC was available). Control participants were drawn from the cohort of current Carrot Rewards users who had enabled the ‘Steps’ walking program but had not engaged in a STC during the study period. Control participants were matched with existing experimental participants on age (±1 yr), gender, province and baseline step count (±500 steps/d, so individuals with similar PA levels would be compared). Only one control user was selected to match to experimental users if they met each of the four criteria; therefore, one control user could be matched with multiple experimental users who shared the same age, gender, province and baseline daily step count. Notably, 10% of the study population with the highest matching ratios (more than 1:18 and up to 1:250) were excluded to minimize the experimental-control imbalance (for more details see Additional file 1). Sensitivity analyses were conducted with experimental-control participants matched 1:1 only as well to check if the imbalance influenced results.
The pre-intervention period was defined as the 12 weeks preceding experimental users’ first STC (Study Weeks 1–12). The intervention period included the 12 weeks following the initiation of the first STC (Study Weeks 13–24). Participants were required to have valid pre-intervention and intervention study periods, consisting of a minimum of four weeks of daily step count data in each period—a valid week was operationally defined as a minimum of four days with step counts between 1000 and 40,000 inclusive, as previously done . A study flow chart is provided (Additional file 2). Ethical approval for this study was provided by Western University’s Human Research Ethics Board (#111252).
Individual and team incentives
Upon downloading the free, commercial Carrot Rewards app, and following a two-week baseline period, Carrot Rewards users earned individual-level incentives in the form of loyalty points (redeemable for consumer goods like movies or groceries) each day they reached a personalized daily step goal (worth $0.04 CAD/day). Given finite reward budgets and a large user base, and to maximize program scalability and sustainability, the smallest possible loyalty point increment was selected (i.e. the app could not offer less than 1 point/d = $0.04 CAD/d). Previous research has suggested that as part of a multicomponent intervention this incentive magnitude could stimulate PA . In addition, several RCTs have demonstrated positive effects with PA incentives worth $0.09 to $0.75 USD per day [23, 39,40,41]. Goals were initially set using the two-week baseline median (e.g., if a user’s baseline daily step count median was 5441 steps, their first goal would be rounded to 5400). See Mitchell et al. (2018) for a more full description of the goal setting approach, including how goals were progressed . While small, incentives were tied to objectively measured PA and were given nearly instantaneously with a push notification using smartphone technology (e.g., linking data from native smartphone accelerometer with loyalty program application programming interfaces (APIs)). Manual entry of daily step count was not possible (e.g., from a pedometer) nor were participants able to set their own step goal, in order to ensure incentives were earned for meaningful PA efforts. To boost app engagement and PA, the ‘Steps’ walking program evolved with refinement of older features, as well as the introduction of new ones. For example, the algorithm used to calculate each user’s daily step goal was updated to be more personalized and adaptive .
STCs were introduced in March 2018 to allow users to collaboratively pursue team-based goals with a peer of their choosing for additional rewards (i.e. a pre-existing friend they had already connected with on the app). Users participating in a STC could each earn a bonus incentive worth $0.40 CAD for together reaching 10 individual daily step goals in a seven-day period (e.g., Partner A completes four goals and Partner B completes six goals in a week; Fig. 1). Users could only participate in one STC at a time. The app allowed users to see their partner’s daily step progress in real time, as well as their own, though users could not communicate about their shared progress in-app (this needed to be done through other means e.g., text messages, in-person, etc.). Over the course of 12-weeks, STC participants could earn a maximum of $9.76 CAD in points. In addition to promoting social support, the STC feature integrated other behaviour change techniques as well including goal setting/review, self-monitoring and demonstration. For more app design detail, completed Mobile App Rating Scale (MARS self-score 4.23/5; for understanding app quality, aesthetics and functional appeal)  and App Behavior Change Scale (ABACUS self-score 4.5/5; for measuring potential to change behaviour)  are provided (Additional files 3 and 4).
The primary outcome was mean daily step count as measured by built-in smartphone accelerometers. In recent validation studies, the iPhone step counting feature, as well as those for Android smartphones were accurate in laboratory and field conditions [45,46,47,48]. Duncan et al. (2018) did determine, however, that steps were under-estimated by the iPhone step counting feature in their free-living condition by approximately 20%, or 1340 steps/day. According to the study authors this likely reflects not carrying the iPhone continually throughout the day rather than inaccuracy in the step counting feature; if adherence can be optimized, they suggest, then smartphones may be suitable for PA evaluations. Self-reported demographics (i.e. age, gender, province) and number of STCs completed were also collected. Number of STCs completed was defined as any STC that was started and finished within the seven-day window, irrespective of whether the Challenge was completed successfully or not. To finish the Challenge, a user simply needed to open the app to facilitate app vs. smartphone data synchronization.
Chi-square and independent t-tests were conducted to examine group equivalency on demographic measures. Controlling for pre-intervention mean daily step count, ANCOVA was performed to examine group differences in intervention period mean daily step count. Data were expressed in estimated marginal means (95% CI). To complement the ANCOVA and increase internal validity (i.e. the extent to which causality can be established) in this quasi-experimental study a number of analysis phase strategies recommended by Handley et al. (2018) were deployed . First, a pairwise t-test examined the mean daily step count change over time (pre-intervention vs. intervention) for each group. Second, ANCOVA and pairwise t-test sensitivity analyses were performed with (a) users with complete data sets only (highly-engaged users with valid step count data for all 24 study weeks), as well as (b) participants with a 1:1 control to experimental matching ratio only (vs. in the overall sample where controls were matched with up to 18 experimental users). Finally, linear regression was performed to determine whether a relationship existed between the number of STCs completed and intervention period mean daily step count. Statistical significance were two-sided and set at 0.05 . Reported effect sizes followed Cohen’s (1988, 1992) criteria; Cohen’s d: small = 0.20, medium = 0.50, large = 0.80, Cramer’s V for chi squared: small = 0.10, medium = 0.30, large = 0.50, partial eta squared: small = 0.01, medium = 0.06, large = 0.14 [51, 52]. Statistical analyses were performed using IBM SPSS Statistics Version 25.
Study sample characteristics, mean baseline daily step count, and mean number of valid weeks in the pre-intervention and intervention period are in Table 1. Group differences in age, gender and province were detected likely due to the large sample size, though effect sizes were very small. As well, the mean baseline daily step count for our study sample was higher than for Carrot Rewards users in general possibly because they were a more engaged sub-group. Lastly, experimental users were different than controls by virtue of the fact that they were early STC adopters (thus arguably ‘more engaged’). The comparable number of valid weeks in the pre-intervention period (11.16 and 10.81 for experimental and controls, respectively), on the other hand, suggest they might have been similarly engaged.
Controlling for pre-intervention mean daily step counts, ANCOVA showed a significant difference in intervention mean daily step count (F(1, 61,167) = 1515.97, p < 0.0001), favouring the experimental group with a small effect (ηp2 = 0.024; Table 2). An estimated marginal means difference of 537 steps per day favoured the experimental group.
A pairwise t-test was also performed on the total sample (n = 20,530 matched pairs of experimental and control users) to compare change in mean daily step count from pre-intervention to intervention for each group. Mean daily step count increased from pre-intervention to intervention for both experimental (1133.92 steps, 95% CI (1110.34 - 1157.50); p < 0.0001, Cohen’s d = 0.658) and control (629.49 steps, 95% CI (609.29–649.68); p < 0.0001, Cohen’s d = 0.426) groups (Table 3). The experimental group showed an increase of 504 mean steps per day more than the control group from the pre-intervention to intervention periods.
Sensitivity analyses examining users with complete data sets only (those highly-engaged users with data for all 24 weeks) and users who were matched on a 1:1 experimental to control user ratio showed no difference compared to the main findings (Additional files 5 and 6).
Linear regression revealed a significant dose-response relationship between the number of STCs completed and mean steps per day [F (1, 14) = 35.834, p < 0.0001], with an adjusted R2 of 0.699). On average, participants’ intervention mean daily step count increased 196.80 (unstandardized beta coefficient) for each new STC completed. Descriptive data presented below illustrates this dose-response relationship (Fig. 2 and Additional file 7). Although the increase in intervention mean steps per day appears somewhat exponential when inspecting increases in step count for users completing 15 and 16 STCs in particular, these means actually represent a very small proportion of users (n = 270 vs. n = 39,355 experimental users in total; see Additional file 7).
In this large quasi-experimental evaluation of the Carrot Rewards app we found that adding team-based incentives increased mean daily step count. Specifically, our experimental group had an increase of 1143 steps whereas the control group saw an increase of 606 steps during the study period – a 537 adjusted mean daily step count difference, or 3759 additional steps per week (roughly equivalent to 40 min of walking). This may be attributed to the application of “herd behaviour”, the behavioural economics principle describing the tendency for individuals to follow others’ behaviours instead of making independent decisions. Carrot Rewards exploited this predictable human tendency by providing real-time feedback on peer progress, as well as by rewarding users only if both achieved at least a few daily goals throughout the Challenge. In doing so, we speculate that STCs may have also served to increase feelings of social connectedness within the small teams, which according to self-determination theory (a global theory of human motivation) promotes quality health behaviour change . A dose-response relationship was also observed with app engagement positively associated with mean daily step count. Each new STC corresponded to about 200 more steps per day.
The clinical implications of daily step count increases of these magnitudes are important. For instance, an increase of 1000 steps per day has been associated with significant weight loss in adults and better glycemic control (i.e. lower A1C) in individuals living with type 2 diabetes [15, 54]. Higher step counts in general are also associated with improved mood and overall health ratings, and are inversely related to systolic blood pressure [15, 55]. From a public health perspective, a 1% reduction in the number of Canadians classified as “physically inactive” (fewer than 5000 daily steps) would yield annual healthcare savings of $2.1 billion CAD . With nearly half of the Carrot Rewards users in general accumulating less than 5000 steps per day, a mere 500 to 1000 step increase from baseline values could have broad implications . In fact, a recent 12 month analysis of the Carrot Rewards app suggests approximately 100,000 Canadians moved up from the “physically inactive” category to the “moderately active” (more than 5000 steps/d) .
The present study provides further evidence that even very small incentives (STC users earned on average $3.60 CAD over 12 weeks, for a total of about $142,000 CAD) can be implemented as part of a multicomponent intervention to increase PA. Recent evidence suggests that reward size may be less important than other program design features (e.g., incentive timing or form) . It has been suggested that manipulating these other features (nine are outlined by Adams et al.  and updated by Mitchell et al. ) may help reduce the cost of incentives while maintaining or even boosting effects . In addition to leveraging “herd behaviour”, the very small incentives in this study increased PA because they were offered immediately and in the form of loyalty points, exploiting two other behavioural economics concepts as well: (a) the human tendency to prefer payoffs close to the present time (“present bias”)  and (b) the tendency for people to equate numbers of unclear significance (i.e. the loyalty points used in this case) with greater value (“numerosity effect”) . These and other theoretically-informed manipulations may appeal to governments and insurers looking to deploy PA incentives as efficiently as possible .
Findings from our quasi-experimental examination in a real-world commercial context compliment those from traditional RCT studies examining the impact of team-based incentives on PA or weight loss, in digital and non-digital settings [15, 24,25,26]. Babcock et al. (2015) found that the number of gym visits was 9–17% higher in the team incentive compared to the individual incentive conditions. In particular, team incentives for teams where members knew each other were more effective than those for anonymous teams . Patel et al. (2016) examined a PA intervention delivered through a research-based smartphone app, combining the social aspects of team incentives with individual-level rewards, much like the incentive scheme used in the present study . In comparing a control condition to individual, team and combined (individual plus team) incentives groups, Patel et al. (2016) found participants in the combined incentives group had a significantly higher mean daily step counts compared to controls (1446 daily step count group difference), whereas the team and individual incentive groups did not outperform controls . Smith-McLallen et al., (2017) also compared a digitally delivered standard walking program to an enhanced program including incentives, feedback and competitive challenges and found that the enhanced group improved their mean daily step count by 726 more than the standard program group over nine months . Finally, a year-long evaluation of the standard Carrot Rewards ‘Steps’ walking program (i.e. before STCs were introduced) found an average increase of 448 and 884 steps per day from baseline for ‘regular’ and ‘committed’ users (i.e. users engaging with the app on 26 to 51 out of 52 weeks, or all 52 weeks, respectively) . It is interesting to note that these pre-post daily step count differences are similar in magnitude to the ones reported here, and that in both studies greater engagement yielded larger effects. This aligns with a growing but still limited (to a small number of RCTs) evidence base suggesting that app exposure is paramount and that greater engagement may produce greater effects [10,11,12, 33]. Regarding PA incentives in general, caution is warranted given that positive effects are not automatic as seen in RCTs demonstrating the benefits of one incentive design/arm (e.g., chance-based incentives) but not others (e.g., guaranteed incentives) .
Limitations and future directions
This study was not without limitations. First, randomization of participants into experimental and control arms was not logistically feasible within this quasi-experimental design making it difficult to conclude with certainty that the Carrot Rewards STC feature caused an increase in PA. For this reason, internal validity may be limited. To increase internal validity in this real-world public health intervention context, we matched experimental participants with similar controls at the study design stage, as well as used three main analytic approaches (i.e. ANCOVA, pairwise t-test, linear regression) to address our primary objective and conducted separate sensitivity analyses (i.e. subgroup ANCOVA and pairwise t-tests) at the analysis stage. Self-selection bias, especially with regard to engagement (i.e. since controls did not engage in a STC during the study period, they may have been less engaged a priori) may have confounded our results despite our best efforts to match experimental users with similar controls. Although we were limited in our ability to match users to a few demographic variables, we were able to match based on baseline daily step count and control for baseline daily step count discrepancies in our analyses. In the future, we suggest other mHealth researchers attempt to maximize group equivalency in quasi-experimental contexts by matching participants using an engagement variable as well (e.g., number of weeks with at least one app open in the past 6 months, number of app features engaged). Unlike what was done here, mHealth researchers should also ensure a 1:1 matching ratio to prevent case-control imbalances. Other design phase strategies to strengthen quasi-experiments, and address selection bias in particular, include interrupted time series designs where multiple observations are evaluated before, during and after intervention within the same group . While traditional RCTs strongly prioritize internal validity, this quasi-experimental design seeks to achieve a greater balance between internal and external validity within real-world conditions to facilitate real-world implementation.
Changing seasons (the study started in the typically cold Canadian Winter and ended in the warmer Spring/Summer months) may also have impacted our results. It is reasonable to assume though that this potentially confounding seasonality effect influenced PA step count similarly in both study groups. In addition, smartphone wear time may have differed between study groups. It is possible that PA increases may have been due to the fact that experimental users simply started carrying their smartphones more to get credit for the steps they were taking. As previously cited, it is hard to dis-entangle ‘wear time’ from actual daily step count increases . Device adherence was uniformly high before and during the intervention, however, as seen with similar numbers of valid weeks in the pre-intervention and intervention periods. This was likely optimized with the presence of the individual-level PA incentives, possibly shrinking the gap between measured and actual steps (i.e. assessment error) for both groups . The fact that budget restrictions dictated reward size in this study could also be construed as a limitation, as larger incentives have generally produced larger effects . Lastly, our study did not evaluate the long-term (six or more months ⎯the theoretical threshold of behaviour maintenance) effects of adding team incentives to an existing walking program rewarding users with individual incentives . Future research should evaluate whether combined incentives drive PA improvements over the long-term, and whether adding unrewarded team-goals produces similar effects.
Given the persisting physical inactivity pandemic, there is an urgent need for scalable and effective digital PA interventions . Examinations of real-world effectiveness have been repeatedly called for in the literature highlighting the important contributions made here. We have shown that incorporating concepts from behavioural economics (e.g., herd behaviour) in the design of mHealth app features in a way that leverages pre-existing social networks has the potential to improve user engagement and PA behaviours. Specifically, participants using the Carrot Rewards app with team-based incentives accumulated more steps per day during a 12-week intervention period compared to matched controls. The more the STC feature was used, the more participants walked. While the effect of STCs on long-term behaviour change is not clear, their role in increasing feelings of social relatedness (even in a digital environment), and thus the potential for sustained change, would be an interesting line of future inquiry. Taken together, much can be learned from this large-scale evaluation of a top-tier commercial PA app. In particular, more high quality quasi-experimental designs are needed to examine real-world effectiveness in a fast paced mHealth context that does not necessarily lend itself to more carefully controlled RCTs that prioritize internal over external validity.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Mok A, Khaw KT, Luben R, Wareham N, Brage S. Physical activity trajectories and mortality: population based cohort study. BMJ. 2019;365:l2323.
American College of Sports Medicine. ACSM’s guidelines for exercise testing and prescription. 10th ed. Philadelphia: Lippincott Williams & Wilkins; 2017.
Reis RS, Salvo D, Ogilvie D, Lambert EV, Goenka S, Brownson RC. Scaling up physical activity interventions worldwide: stepping up to larger and smarter approaches to get people moving. Lancet. 2016;388(10051):1337–48.
Meeker M. Internet trends report; 2019. https://www.cmswire.com/digital-marketing/mary-meekers-2019-internet-trends-report-8-takeaways-for-marketers/.
Research2Guidance. mHealth App Economics 2017: Current status and future trends in mobile health 2017. https://research2guidance.com/product/mhealth-economics-2017-current-status-and-future-trends-in-mobile-health/.
Middelweerd A, Mollee JS, van der Wal CN, Brug J, Te Velde SJ. Apps to promote physical activity among adults: a review and content analysis. Int J Behav Nutr Phys Act. 2014;11:97.
Maher CA, Lewis LK, Ferrar K, Marshall S, De Bourdeaudhuij I, Vandelanotte C. Are health behavior change interventions that use online social networks effective? A systematic review. J Med Internet Res. 2014;16(2):e40.
Laranjo L, Arguel A, Neves AL, et al. The influence of social networking sites on health behavior change: a systematic review and meta-analysis. J Am Med Inform Assoc. 2015;22(1):243–56.
Looyestyn J, Kernot J, Boshoff K, Ryan J, Edney S, Maher C. Does gamification increase engagement with online programs? A systematic review. PLoS One. 2017;12(3):e0173403.
Schoeppe S, Alley S, Van Lippevelde W, et al. Efficacy of interventions that use apps to improve diet, physical activity and sedentary behaviour: a systematic review. Int J Behav Nutr Phys Act. 2016;13(1):127.
Maher C, Ferguson M, Vandelanotte C, et al. A Web-Based, Social Networking Physical Activity Intervention for Insufficiently Active Adults Delivered via Facebook App: Randomized Controlled Trial. J Med Internet Res. 2015;17(7):e174.
Mitchell M, White L, Lau E, Leahey T, Adams MA, Faulkner G. Evaluating the Carrot Rewards App, a Population-Level Incentive-Based Intervention Promoting Step Counts Across Two Canadian Provinces: Quasi-Experimental Study. JMIR Mhealth Uhealth. 2018;6(9):e178.
Ryan J, Edney S, Maher C. Engagement, compliance and retention with a gamified online social networking physical activity intervention. Transl Behav Med. 2017;7(4):702–8.
Edney S, Plotnikoff R, Vandelanotte C, et al. "Active Team" a social and gamified app-based physical activity intervention: randomised controlled trial study protocol. BMC Public Health. 2017;17(1):859.
Smith-McLallen A, Heller D, Vernisi K, Gulick D, Cruz S, Snyder RL. Comparative effectiveness of two walking interventions on participation, step counts, and health. Am J Health Promot. 2017;31(2):119–27.
Camerer CF, Loewenstein G. Behavioral economics: past, present, future. Advances in Behavioural economics. Princeton: Princeton University Press; 2003.
Ariely D. Predictably irrational: the hidden forces that shape our decisions. New York: Harper Perennial; 2010.
Loewenstein G, Asch DA, Volpp KG. Behavioral economics holds potential to deliver better results for patients, insurers, and employers. Health Aff (Millwood). 2013;32(7):1244–50.
Adams J, Giles EL, McColl E, Sniehotta FF. Carrots, sticks and health behaviours: a framework for documenting the complexity of financial incentive interventions to change health behaviours. Health Psychol Rev. 2014;8(3):286–95.
Adams MA, Hurley JC, Todd M, et al. Adaptive goal setting and financial incentives: a 2 x 2 factorial randomized controlled trial to increase adults' physical activity. BMC Public Health. 2017;17(1):286.
Mitchell MS, Orstad SL, Biswas A, et al. Financial incentives for physical activity in adults: systematic review and meta-analysis. Br J Sports Med. 2020;54:1259–68.
Patel MS, Volpp KG, Rosin R, et al. A randomized trial of social comparison feedback and financial incentives to increase physical activity. Am J Health Promot. 2016;30(6):416–24.
Condliffe S, Işgın E, Fitzgerald B. Get thee to the gym! A field experiment on improving exercise habits. J Behav Exp Econ. 2017;70:23–32.
Babcock P, Bedard K, Charness G, Hartman J, Royer H. Letting down the team? Social effects of team incentives. J Eur Econ Assoc. 2015;13(5):841–70.
Kullgren JT, Troxel AB, Loewenstein G, et al. Individual- versus group-based financial incentives for weight loss: a randomized, controlled trial. Ann Intern Med. 2013;158(7):505–14.
Patel MS, Asch DA, Rosin R, et al. Individual versus team-based financial incentives to increase physical activity: a randomized, Controlled Trial. J Gen Internal Med. 2016;31(7):746–54.
Raafat RM, Chater N, Frith C. Herding in humans. Trends Cogn Sci. 2009;13(10):420–8.
Kullgren JT, Harkins KA, Bellamy SL, et al. A mixed-methods randomized controlled trial of financial incentives and peer networks to promote walking among older adults. Health Educ Behav. 2014;41(1 Suppl):43s–50s.
Cavallo DN, Tate DF, Ries AV, Brown JD, DeVellis RF, Ammerman AS. A social media-based physical activity intervention: a randomized controlled trial. Am J Prev Med. 2012;43(5):527–32.
Kurtzman GW, Day SC, Small DS, et al. Social incentives and Gamification to promote weight loss: the LOSE IT randomized, Controlled Trial. J Gen Intern Med. 2018;33(10):1669–75.
Petersen JM, Prichard I, Kemps E. A Comparison of Physical Activity Mobile Apps With and Without Existing Web-Based Social Networking Platforms: Systematic Review. J Med Internet Res. 2019;21(8):e12687.
Research GV. mHealth Apps Market Size, Share & Trends Analysis Report by Type, and Segment Forecasts, 2019–2026; 2019.
Romeo A, Edney S, Plotnikoff R, et al. Can Smartphone Apps Increase Physical Activity? Systematic Review and Meta-Analysis. J Med Internet Res. 2019;21(3):e12053.
Government of Canada. National Healthy Living Platform: “Carrot Rewards” targets lifestyle improvements. Available at: https://www.canada.ca/en/news/archive/2015/07/national-healthy-living-platform-carrot-rewards-targets-lifestyle-improvements.html. Accessed 24 Oct 2020.
Mitchell M, Lau E, White L, et al. Commercial app use linked with sustained physical activity in two Canadian provinces: a 12-month quasi-experimental study. Int J Behav Nutr Phys Act. 2020;17(24). https://0-doi-org.brum.beds.ac.uk/10.1186/s12966-020-00926-7.
Victora CG, Habicht JP, Bryce J. Evidence-based public health: moving beyond randomized trials. Am J Public Health. 2004;94(3):400–5.
West SG, Duan N, Pequegnat W, et al. Alternatives to the randomized controlled trial. Am J Public Health. 2008;98(8):1359–66.
Colley RC, Garriguet D, Janssen I, Craig CL, Clarke J, Tremblay MS. Physical activity of Canadian adults: accelerometer results from the 2007 to 2009 Canadian health measures survey. Health Rep. 2011;22(1):7–14.
Carrera M, Royer H, Stehr M, Sydnor J. Can financial incentives help people trying to establish new habits? Experimental evidence with new gym members. J Health Econ. 2018;58:202–14.
Rohde K, Verbeke W. We like to see you in the gym—a field experiment on financial incentives for short and long term gym attendance. J Econ Behav Organ. 2017;134:388–407.
Leahey TM, Subak LL, Fava J, et al. Benefits of adding small financial incentives or optional group meetings to a web-based statewide obesity initiative. Obesity. 2015;23(1):70–6.
Mitchell M, Lau E, White L, Faulkner G. Commercial app use linked with sustained physical activity in two Canadian provinces: a 12-month quasi-experimental study. Int J Behav Nutr Phys Activity. 2020;17(1):24.
Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR MHealth UHealth. 2015;3(1):e27.
McKay FH, Slykerman S, Dunn M. The App Behavior Change Scale: Creation of a Scale to Assess the Potential of Apps to Promote Behavior Change. JMIR MHealth UHealth. 2019;7(1):e11130.
Duncan MJ, Wunderlich K, Zhao Y, Faulkner G. Walk this way: validity evidence of iphone health application step count in laboratory and free-living conditions. J Sports Sci. 2017;28:1–10.
Hekler EB, Buman MP, Grieco L, et al. Validation of Physical Activity Tracking via Android Smartphones Compared to ActiGraph Accelerometer: Laboratory-Based and Free-Living Validation Studies. JMIR Mhealth Uhealth. 2015;3(2):e36.
Evenson KR, Goto MM, Furberg RD. Systematic review of the validity and reliability of consumer-wearable activity trackers. Int J Behav Nutri Phys Act. 2015;12:159.
Case MA, Burwick HA, Volpp KG, Patel MS. Accuracy of smartphone applications and wearable devices for tracking physical activity data. JAMA. 2015;313(6):625–6.
Handley MA, Lyles CR, McCulloch C, Cattamanchi A. Selecting and improving quasi-experimental designs in effectiveness and implementation Research. Annu Rev Public Health. 2018;39:5–25.
Tabachnick BG, Fidell LS. SPSS for Windows workbook to accompany large sample examples of using multivariate statistics. New York: HarperCollins College Publishers; 1996.
Cohen J. A power primer. Psychol Bull. 1992;112(1):155–9.
Cohen J. Statistical power analysis for the behavioral sciences. Hillside: Lawrence Erlbaum Associates; 1988.
Deci EL, Ryan RM. Intrinsic motivation and self-determination in human behavior. New York, NY: Plenum; 1985.
Dasgupta K, Rosenberg E, Joseph L, et al. Physician step prescription and monitoring to improve ARTERial health (SMARTER): a randomized controlled trial in patients with type 2 diabetes and hypertension. Diabetes Obes Metab. 2017;19(5):695–704.
Pillay JD, Kolbe-Alexander TL, van Mechelen W, Lambert EV. Steps that count: the association between the number and intensity of steps accumulated and fitness and health measures. J Phys Act Health. 2014;11(1):10–7.
Krueger H, Turner D, Krueger J, Ready AE. The economic benefits of risk factor reduction in Canada: tobacco smoking, excess weight and physical inactivity. Can J Public Health. 2014;105(1):e69–78.
Brower J, LaBarge MC, White L, Mitchell MS. Examining Responsiveness to an Incentive-Based Mobile Health App: Longitudinal Observational Study. J Med Internet Res. 2020;22(8):e16797.
Mitchell MS, Goodman JM, Alter DA, Oh PI, Faulkner GE. Development of the Health Incentive Program Questionnaire (HIP-Q) in a cardiac rehabilitation population. Transl Behav Med. 2015;5(4):443–59.
Wertenbrock K, Soman D, Chattopadhyay A. On the perceived value of money: The reference dependence of currency numerosity effects. J Consum Res. 2007;34(1):1–10.
Finkelstein EA, Haaland BA, Bilger M, et al. Effectiveness of activity trackers with and without incentives to increase physical activity (TRIPPA): a randomised controlled trial. Lancet Diabetes Endocrinol. 2016;4(12):983–95.
Prochaska JO, Velicer WF. The transtheoretical model of health behavior change. Am J Health Promot. 1997;12(1):38–48.
Kohl HW 3rd, Craig CL, Lambert EV, et al. The pandemic of physical inactivity: global action for public health. Lancet. 2012;380(9838):294–305.
All who have contributed significantly to the work have been acknowledged.
Public Health Agency of Canada (PHAC) Multi-Sectoral Partnership Approach to Healthy Living and Chronic Disease Prevention, as well as the Government of Ontario. The funding bodies did not contribute to any aspect of the study.
Ethics approval and consent to participate
Consent for publication
MM received consulting fees from Carrot Insights Inc. from 2015 to 2018 as well as travel re-imbursement in January and March 2019. MM had stock options in the company as well but these are now void since Carrot Insights Inc. went out of business in June 2019. LW was employed by Carrot Insights Inc. from March 2016 to June 2019 and also had stock options which are now void. EP, HP, CH and RP declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Detailed description of the matching process.
Completed Mobile Application Rating Scale (MARS).
Completed App Behavior Change Scale (scale to assess the potential of apps to promote behavior change).
ANCOVA results adjusting for pre-intervention mean daily step count for total sample, users with complete data sets only (sensitivity), and 1:1 matched users only (sensitivity).
Pairwise t-test results for total sample, users with complete data sets only (sensitivity), and 1:1 matched users only (sensitivity).
Mean steps per day in the pre-intervention and intervention periods by number of Step Together Challenges completed.
Completed TIDieR (Template for Intervention Description and Replication) Checklist.
Completed STROBE Statement—checklist of items to be included in reports of observational studies.
About this article
Cite this article
Pearson, E., Prapavessis, H., Higgins, C. et al. Adding team-based financial incentives to the Carrot Rewards physical activity app increases daily step count on a population scale: a 24-week matched case control study. Int J Behav Nutr Phys Act 17, 139 (2020). https://0-doi-org.brum.beds.ac.uk/10.1186/s12966-020-01043-1
- Social relatedness
- Physical activity
- Public health
- Behavioural economics