SUM-UP: Detecting Receptivity for mHealth Interventions
Detecting Receptivity for mHealth Interventions
Machine-learning models deployed in a real-world chatbot app improved user receptivity to health intervention messages by up to 36% — simply by timing when messages were sent.
— Core finding, Mishra et al., 202101 Background & Motivation
The paper emerges from a well-established body of mHealth research showing smartphones and wearables can detect stress, mood, physical activity, and addiction. The authors argue that sensing alone isn’t enough — the real challenge is delivering interventions at the right moment. Just-In-Time Adaptive Interventions (JITAIs) are the framework that attempts to solve this. Prior work mostly built receptivity models post-hoc; this study deploys them in real time.
02 The Prior Ally Study (Foundation)
The team had already built the Ally app (189 participants, Switzerland, 6 weeks) to test physical-activity coaching via a German-speaking chatbot. That study explored which contextual signals (time of day, motion, phone usage) correlate with receptivity and showed a 77% F1-score improvement over a random baseline. This new paper asks: do these models actually work when deployed live?
03 Operationalizing Receptivity
The authors define receptivity using four concrete metrics: (1) Just-in-time response — user replies within 10 minutes; (2) Response — user replies at any time; (3) Response delay — seconds between message receipt and first reply; (4) Conversation engagement — user sends more than one reply within 10 minutes.
04 The Ally+ App & Three Models
Ally+ (iOS) added a real-time receptivity module on top of the original app. Each day, the server sent three silent push notifications (one per time block). Upon receiving each, the app chose which model to use for timing delivery: Control (sends immediately, randomly), Static (pre-built SVM trained on prior Ally data), or Adaptive (personalized logistic regression + static model blended, updated with each interaction). The adaptive model required 7 days of warm-up data before activating.
05 Study Design & Participants
83 participants (64 female, avg. age 30±10.8) recruited via Facebook ads. The study used deception — participants were told the study was about how context affects physical activity, not about receptivity detection. This prevented artificial engagement. Compensation was ~USD 25 for ≥14 days of use. IRB-approved; participants were debriefed afterward. Total: 2,023 messages delivered across three conditions.
06 Results
Using generalized linear mixed effects models: the static model showed statistically significant improvements — +36.6% just-in-time response rate (p=0.002), +9.75% overall response (p=0.015), +32.18% conversation engagement (p=0.007). The adaptive model improvements were not significant overall, but a day-by-day analysis revealed a clear positive trend — by Day 21 the adaptive model had 51% higher just-in-time response than Day 8, and conversation engagement showed a significant upward slope (p=0.045).
07 Implications for JITAI Design
The authors argue receptivity should be treated as a spectrum, not binary. A person may be receptive to one intervention type but not another at the same moment. Future JITAIs should consider three dimensions together: degree of vulnerability, level of receptivity, and expected intervention effectiveness — then choose which intervention to deliver, not just whether to deliver one.
-
Timing matters as much as content
The same intervention message delivered at a detected “receptive” moment dramatically outperformed messages delivered randomly — without changing a single word.
+36% just-in-time response (static) +32% conversation engagement -
Pre-built static models can outperform random delivery immediately
A model trained before the study — on different participants from a prior study — still significantly improved receptivity from Day 1. Contextual patterns generalize across people well enough for a static model to be useful without personalization.
p=0.002 (just-in-time) p=0.015 (response rate) p=0.007 (engagement) -
Adaptive models need time — but show clear learning trajectories
The adaptive model’s overall numbers weren’t statistically significant, partly because it required 7+ days of warm-up. But its day-by-day trend was clear: by Day 21 it showed a 51% improvement over its own Day 8 baseline.
Slope: +0.0092/day Engagement slope p=0.045 +51% by Day 21 -
The control condition worsened over time — engagement fatigue is real
Random delivery didn’t stay neutral — it actively declined. The control model’s just-in-time response rate dropped significantly over 3 weeks (p=0.011), making smart timing even more important for longer programs.
Control slope: −0.0069/day (p=0.011) -
Binary receptivity is a first step — a spectrum model is the goal
This study treated receptivity as yes/no. But the authors argue it’s dimensional: a person might be receptive to a motivational tip but not a lengthy reflection exercise at the same moment. Future JITAIs should match intervention type to both vulnerability level and receptivity level simultaneously.
-
Post-study model evaluation ≠ real-world performance
A key methodological contribution: most prior mHealth ML research validates models after data collection ends, then assumes deployment will look similar. This study shows that’s not guaranteed — and that deploying models in-the-moment requires thinking about warm-up periods, data sparsity, and feedback loops.
-
Limitations to keep in mind
Only 3 weeks. 83 participants is modest. The adaptive model had fewer data points than other conditions. Results are from a physical-activity context; generalizability to other health behaviors is unclear.
Comparing model performance improvements over the control baseline
Bars are scaled for visual comparison. Grey = not statistically significant. Teal = within-model improvement over time.
Why this reference matters
This is the foundational prior work the entire Ally+ study builds on. The original Ally study (189 participants, Switzerland, 6 weeks) was where the team first developed and validated the concept of detecting receptivity through passively-collected contextual phone signals. It achieved a 77% F1-score improvement over a biased random model.
Key contribution to this paper
Provided the training data (141 iOS users) used to build the static model. Defined the 10-minute window for “just-in-time response” that is carried forward as the primary metric. Established the contextual features used in the classifiers.
Why this reference matters
This is the canonical JITAI framework paper — the theoretical backbone of the entire study. It formally defines JITAIs and their six key components: distal outcome, proximal outcomes, decision points, intervention options, tailoring variables, and decision rules.
Key contribution to this paper
Provides the vocabulary and conceptual structure that allows the authors to position receptivity as a tailoring variable within a JITAI system. The “implications” section of Ally+ maps directly back to this framework.
Why this reference matters
One of the earliest papers to study receptivity/availability for just-in-time interventions in naturalistic settings. Cited to establish that other researchers have independently explored ML models for detecting when users are available to engage with health interventions.
Key contribution to this paper
Validates the problem statement and highlights the limitation shared by most prior work: models were built post-hoc rather than deployed live — which is exactly the gap Ally+ addresses.
This paper has a tight citation network: [4] is the direct precursor providing data and metrics; [6] provides the JITAI theoretical framework; [7] and [2] establish the broader ML-for-receptivity literature. Most authors appear across multiple papers — this is a focused, multi-year collaborative research program rather than an isolated study.