Loss of the fibroblastic reticular cell (FRC) network in lymphoid tissues during HIV-1 infection has been shown to impair the survival of naive T cells and limit immune reconstitution after antiretroviral therapy. What causes this FRC loss is unknown. Because FRC loss correlates with loss of both naive CD4 and CD8 T-cell subsets and decreased lymphotoxin-β, a key factor for maintenance of FRC network, we hypothesized that loss of naive T cells is responsible for loss of the FRC network. To test this hypothesis, we assessed the consequences of antibody-mediated depletion of CD4 and CD8 T cells in rhesus macaques and sooty mangabeys. We found that only CD4 T-cell depletion resulted in FRC loss in both species and that this loss was caused by decreased lymphotoxin-β mainly produced by the CD4 T cells. We further found the same dependence of the FRC network on CD4 T cells in HIV-1-infected patients before and after antiretroviral therapy and in other immunodeficiency conditions, such as CD4 depletion in cancer patients induced by chemotherapy and irradiation. CD4 T cells thus play a central role in the maintenance of lymphoid tissue structure necessary for their own homeostasis and reconstitution.