You don’t have to tell me twice: 

Additional text message reminders for confirming volunteer attendance for Mobilize event

Gabrielle Goldstein, Director of Research

Mallory Roman, Associate Director of Research

Abstract: This study tested the efficacy of sending an additional personal text message reminder to volunteers who have RSVP’d for an event in boosting event turnout. Using an event on April 29, 2021, SDAN randomized the RSVP list to either receive a personal text message in addition to the automated text message sent by Mobilize (treatment) or not (control). Volunteers assigned to receive personal messages did not attend at higher rates than volunteers who did not receive personal messages. Further, people who had already confirmed and people who had not already confirmed behaved similarly in their attendance regardless of treatment. Results indicate that the auto-generated Mobilize text message reminder is sufficient and an additional personal text message reminding RSVP’s of the event does not result in additional attendance.

Takeaways:

• People who RSVP’d attended the event at almost the same rate, regardless of condition.

• Event attendance was only 0.38% more in the treatment condition than in the control condition, and this difference was not statistically significant (p = 0.947)

• Effects did not vary based on confirmation status.

• People who already confirmed attendance and people who had not confirmed their plan to attend the event ended up attending at about the same rate regardless of condition, and any difference was not statistically significant (p = 0.253).

• Though unrelated to the intervention, people who confirmed their RSVP for the event attended the event at a significantly higher rate than people who did not confirm their attendance (p<0.001). This is potentially useful, as confirmation is a strong indicator of attendance.
• Even though this study was intended as a pilot, it is almost adequately powered at its current n. The results suggest that there is no effect here to explore further.

1. Background and Research Questions:

         Organizer wisdom and previous work by SDAN and others has determined that volunteers are more likely to attend events or activities they have RSVP’d to attend when their RSVP is confirmed. This is accepted as fact, and services like Mobilize and EventBrite have built in functions to automatically remind RSVPs of the upcoming event. But are these reminders enough to increase attendance? Are additional, personal reminders from a fellow volunteer still necessary?

In order to test this, SDAN investigated the use of an additional confirmation text message, sent by a real person the morning of an event to which targets had RSVP’d, to determine if this additional reminder was useful or necessary. Further, we wanted to determine if people who had already confirmed their attendance via Mobilize were affected by additional text message confirmation reminders.

H1: RSVPs who receive an additional personal text message reminding them to attend the event will attend at higher rates than people who do not receive an additional reminder.

H2: But people who have already confirmed their attendance via the Mobilize reminder will be unaffected by additional reminders.

2.  Methods:

1. Study Design.

This study was a randomized controlled trial (RCT) designed by SDAN. SDAN chose a fundraising event with a large RSVP list for the pilot. The inclusion criteria for the study were RSVPing for the event and listing a phone number with which one could be sent a text message. All RSVPs that met inclusion criteria were enrolled in the study. All of the RSVPs were sent email reminders and a text message reminder automatically by Mobilize. A randomly chosen half of the sample received an additional text message from a real person the morning of the event on April 29, 2021. Confirmations and event attendance were tracked via Mobilize.

2. Statistical Analysis

Main model. The central question posed in this analysis is whether there is an association between receiving an additional personal text message reminder, after receiving email and one automated text message reminder from Mobilize, and increased event attendance. To test the model, a chi square test was used to assess differences between conditions (personal text message or no personal text message) with respect to attendance (yes, no).

Moderation model. The moderation model focused on whether the effect of condition was moderated by confirmation status. Logistic regression was used to test the moderation model by regressing attendance onto condition, confirmation status, and the interaction between condition and confirmation status.

3.  Results:

Main Model.  The sample attended at essentially the same rate in both conditions, with about 56% of RSVPs actually attending the event. This indicates that there is no significant difference between groups and that a second text message had no effect on event attendance.

Table 1: Condition x Attendance Chi Square Test

𝝌2 (1) = 0.0043, p = 0.947 (Logistic regression estimates: Odds Ratio - 1.015625; std. Err. 0.2390494; z = 0.07; 95% CI = 0.6403016 - 1.61095)

Moderation Model. The moderation model indicates that RSVPs who had not confirmed their attendance benefitted slightly from receiving an additional personal text message reminder and people who had previously confirmed actually saw a slight backlash, but this effect was not statistically significant (p = 0.253). The text message also had no main effect in the moderated model (p = 0.367), but confirming one’s attendance is a statistically significant predictor of attendance (p <0.001).

Table 2: Moderation Model with voting outcome and interactions- Estimated odds ratios (robust standard errors), 95% confidence intervals, and p-values for the relationship between positive response, randomized condition and other predictor variables.

N = 293, 𝝌2 (3) = 42.61, p <0.001, pseudo R2 = 0.1060

The graph below illustrates the relationship between confirmation status and condition found in the data. While reminders didn’t increase attendance significantly for either group, they seem particularly unhelpful in motivating RSVPs who have already confirmed their attendance and may actually be detrimental.

Costs. The cost of sending a text message via Hustle is $0.045 and we sent 146 text messages for this study (146 texts at $0.045 each = $6.57). The text message condition increased event attendance by 0.56 of a person, so rounded up to 1 the cost of an additional attendee using this tactic was $6.57. Based on the non-significant p value is it unclear if this would be scalable.

4.  Conclusions, Limitations, and Future Research

The hypothesis that the people who received personal text reminders on the day of an event, in addition to the email reminders and the automated text message reminder from Mobilize prior to the event, would attend the April 29th event at higher rates than people who did not was not borne out at a statistically significant level in this study. In fact, there is no suggestive evidence in support of sending additional personal text messages and we believe this pilot is conclusive enough to not require a larger scale test. This suggests that additional text message reminders beyond the automated message from Mobilize are unnecessary.

It also appears that reminders did not affect people based on confirmation status. Neither people who confirmed their attendance nor people who did not were affected by the additional personal text message reminder. This suggests that there are no populations of event RSVPs, confirmed or unconfirmed, that would benefit significantly from an additional personal text message reminder.

This study was meant to be a pilot, but was almost adequately powered according to a priori sample size analyses. The results suggest that this tactic is not fruitful for Sister District volunteer events. This is only one test and is not generalizable to all volunteer populations, as it only included Sister District volunteers and those who RSVP’d to this Sister District event. Further, only people who included their phone numbers could be included, and those who omitted their phone numbers could be a categorically different group of RSVPs.

Future research may explore attendance to voter outreach events or in person events as opposed to virtual event attendance. These contexts may lead to different effects from the personal text message reminder.

Appendix

Text message:

Hi [RSVP first name]! This is Mallory from Sister District. I just wanted to remind you about the State Bridges event for the Texas Organizing Project tonight, April 29th, at 8 pm ET. Thank you for joining us - see you there!