Objective In 2004, the province of Aceh, Indonesia was rocked by tsunami and in September 2009, West Sumatra, Indonesia was hit by an earthquake. The aim of this study was to determine the long-term psychological impact on the residents inhabiting these regions and to identify factors associated with it. Methods A cross-sectional study was conducted among the residents. The Depression, Anxiety and Stress Scale (DASS) was used to measure their psychological well-being. Methods Out of 200 respondents, 1 in 5 (19%) was found to suffer from a high level of depression, 1 in 2 (51%) had anxiety and 1 in 5 (22%) experienced stress. Factors found to be significantly associated with depression, anxiety and stress were female, young age, unemployed, and single (p < 0.05). Conclusion The psychological impact following the tsunami persisted in the population after many years post-disaster. It is recommended that the psychological profile of the population be evaluated for the vulnerable group following any natural disaster.