Extramarital sexual partnerships are a common reason for intimate partner violence (IPV) in sub-Saharan Africa. Despite the fact that IPV requires an interaction between two partners, the majority of the research focuses on individuals rather than the broader relationship context where such violence takes place. Using a sample of 422 married couples from rural Malawi, this study examined the dyadic environment of marital infidelity and two types of IPV victimization: sexual coercion and physical abuse. We considered both self-reported marital infidelity and perceived partner infidelity to assess how well partners knew each other and to compare their respective associations with IPV. Logistic regression was used to test for associations between self-reported marital infidelity and IPV. Multilevel logistic regression was used to examine actor and partner effects of perceived partner infidelity on an individual’s and their partner’s experience of IPV. The results show that self-reported marital infidelity was not significantly associated with IPV for men or women. However, the perception of a partner’s infidelity was significantly associated with both an individual’s and their partner’s risk for sexual coercion and physical abuse. Contrary to the “sexual double standard” hypothesis, women were not significantly more likely than men to report being physically abused when their partners suspected infidelity. Future studies should continue to explore the relationship context of IPV in sub-Saharan Africa in order to understand how spouses mutually shape each other’s experience of IPV and subsequent health outcomes.