Objectives: In the era of dose-escalated prostate radiation therapy (RT), the use of androgen deprivation therapy (ADT) is undefined for intermediate-risk (IR) prostate cancer. There is growing concern of the risk of ADT to be detrimental to quality of life. This single-institution retrospective analysis aimed to evaluate outcomes of IR patients treated with dose-escalated intensity modulated radiation therapy (IMRT) with or without concurrent/adjuvant short-term ADT. Materials and Methods: Data was collected from 260 consecutive patients treated with dose-escalated IMRT with daily image-guided RT for newly diagnosed IR prostate cancer. Biochemical recurrence-free survival (BCRFS), distant metastasis-free survival, prostate cancer-specific survival, and overall survival (OS) were calculated using Kaplan-Meier methodology. Results: Median follow-up was 93 months. A total of 181 patients had unfavorable IR disease, and 36.2% (N=94) received ADT, with median ADT duration of 6 months. Seven-year BCRFS was 94.1% vs. 86.2% (P=0.067), for ADT and no ADT, respectively, and no difference in distant metastasis-free survival or prostate cancer-specific survival was observed. ADT was associated with significantly worse 7-year OS (80.0% vs. 91.3%, P=0.010). Analysis of the unfavorable IR cohort alone, showed similar results; 7-year BCRFS and 7-year OS in patients who received ADT versus no ADT were 93.7% vs. 85.9% (P=0.093), and 79.0% vs. 90.6% (P=0.019), respectively. Conclusions: In our 15-year experience treating IR prostate cancer with dose-escalated IMRT with daily image-guided RT, short-term concurrent ADT was associated with a statistically significant worse OS. Additional studies are needed to determine if ADT is beneficial or detrimental for patients with IR prostate cancer treated with dose-escalated radiation.