Diabetes prevalence in Tanzania was estimated at 9.1% in 2012 among adults aged 24–65 years — higher than the HIV prevalence in the general population at that time. Health systems in lower- and middle-income countries are not designed for chronic health care, yet the rising burden of non-communicable diseases such as diabetes demands chronic care services. To inform policies on diabetes care, we conducted a study on the health services in place to diagnose, treat and care for diabetes patients in rural Tanzania. The study was an exploratory and descriptive study involving qualitative methods (in-depth interviews, observations and document reviews) and was conducted in a rural district in Tanzania. Fifteen health providers in four health facilities at different levels of the health care system were interviewed. The health care organization elements of the Innovative Care for Chronic Conditions (ICCC) framework were used to guide assessment of the diabetes services in the district. We found that diabetes care in this district was centralized at the referral and district facilities, with unreliable supply of necessary commodities for diabetes care and health providers who had some knowledge of what was expected of them but felt ill-prepared for diabetes care. Facility and district level guidance was lacking and the continuity of care was broken within and between facilities. The HMIS could not produce reliable data on diabetes. Support for self-management to patients and their families was weak at all levels. In conclusion, the rural district we studied did not provide diabetes care close to the patients. Guidance on diabetes service provision and human resource management need strengthening and policies related to task-shifting need adjustment to improve quality of service provision for diabetes patients in rural settings.