This study determined the social vulnerability index (SoVI) of households to climate change impacts for three identified locations (upper, mid and lower) in the Vea catchment, semi-arid Ghana. This study adapted the social, economic and demographic indicator approach. The data used were obtained from a survey of 186 randomly sampled farm households and direct field measurements of 738 farm plots belonging to the same sampled farm households. Information from the literature, expert judgement and principal component analysis were useful for computing and analysing the SoVI. The variables were normalized, weighted and subsequently recombined to determine the index of the three locations towards climate change. Although the SoVI to climate change was highest (0.77) for the upper part of the catchment, the mid- and lower parts of the catchment show a high SoVI of 0.72 each. The overall SoVI for the catchment is 0.73. The study re-emphasizes the high vulnerability level of dry areas to climate change. Moreover, it shows there is variability at micro-scale. There is a need to put appropriate measures to address the vulnerability of households to climate change in the semi-arid areas of West Africa. Factors aggravating dry land’s vulnerability towards climate change should be prevented with implementable policies. Furthermore, it is important to identify conditions that have made some areas less vulnerable to climate change, and then, we can work out the possibility of adapting such to the vulnerable places.