Background
Despite the growing use of technology in the health sector, little evidence is available on the technological performance of mobile health programs nor on the willingness of target users to utilize these technologies as intended (behavioral performance). In this case study of the Mobile Technology for Health (MOTECH) program in Ghana, we assess the platform’s effectiveness in delivering messages, along with user response across sites in five districts from 2011 to 2014.

Methods
MOTECH is comprised of “Client Data Application" (CDA) which allows providers to digitize and track service delivery information for women and infants and “Mobile Midwife” (MM) which sends automated educational voice messages to the mobile phones of pregnant and postpartum women. Using a naturalist study design, we draw upon system generated data to evaluate message delivery, client engagement, and provider responsiveness to MOTECH over time and by level of facility.

Results
A total of 7,370 women were enrolled in MM during pregnancy and 14,867 women were enrolled postpa1rtum. While providers were able to register and upload patient-level health information using CDA, the majority of these uploads occurred in Community-based facilities versus Health Centers. For MM, 25% or less of expected messages were received by pregnant women, despite the majority (>77%) owning a private mobile phone. While over 80% of messages received by pregnant women were listened to, postpartum rates of listening declined over time. Only 25% of pregnant women received and listened to at least 1 first trimester message. By 6–12 months postpartum, less than 6% of enrolled women were exposed to at least one message.

Conclusions
Caution should be exercised in assuming that digital health programs perform as intended. Evaluations should measure the technological, behavioral, health systems, and/or community factors which may lead to breaks in the impact pathway and influence findings on effectiveness. The MOTECH platform’s technological limitations in ‘pushing’ out voice messages highlights the need for more timely use of data to mitigate delivery challenges and improve exposure to health information. Alternative message delivery channels (USSD or SMS) could improve the platform’s ability to deliver messages but may not be appropriate for illiterate users.