Gender based violence is gaining more and more recognition, the world over. The prevalence and factors associated with physical violence as it affects both men and women taking into account the hierarchical nature of the data was reported in this study. A descriptive cross sectional survey was carried out on adult women and men in three selected states of Nigeria. A random effect logistic model was fitted. More female respondents had experienced physical violence compared with males. Behavioural factors of partners were found to greatly influence domestic violence against females such as young age and partner’s smoking and drinking status. These results would provide more insights into the determinants of violence in developing country settings.