A federal judge on Tuesday ruled the U.S. Bureau of Prisons must keep providing transgender inmates gender-affirming care, despite an executive order President Donald Trump signed on his first day back in office to halt funding for such care.

U.S. District Judge Royce Lamberth in Washington, D.C., allowed a group of more than 2,000 transgender inmates in federal prisons to pursue a lawsuit challenging the order as a class action. He ordered the Bureau of Prisons to provide them with hormone therapy and accommodations such as clothing and hair-removal devices while the lawsuit plays out.

The ruling does not require the bureau to provide surgical care related to gender transitions.

White House spokesperson Harrison Fields said the Trump administration expects to ultimately prevail in the legal dispute.

“The District Court’s decision allowing transgender women, aka MEN, in women’s prisons fundamentally makes women less safe and ignores the biological truth that there are only two genders,” Fields said in an email.

The American Civil Liberties Union, which represents the inmates, said the ruling was “a critical reminder to the Trump administration that trans people, like all people, have constitutional rights that don’t simply disappear because the president has decided to wage an ideological battle.”

About 2,230 transgender inmates are housed in federal custodial facilities and halfway houses, according to the U.S. Department of Justice. About two-thirds of them, 1,506, are transgender women, most of whom are housed in men’s prisons.

The named plaintiffs, two transgender men and one transgender woman, sued the Trump administration in March to challenge Trump’s January 20 executive order aimed at combating what the administration called “gender ideology extremism.”

The executive order directed the federal government to only recognize two, biologically distinct sexes, male and female; and house transgender women in men’s prisons. It also ordered the bureau to stop spending any money on “any medical procedure, treatment, or drug for the purpose of conforming an inmate’s appearance to that of the opposite sex.”

Lamberth, appointed by Republican President Ronald Reagan, said in Tuesday’s ruling that the plaintiffs were likely to succeed in their lawsuit because the bureau did not perform any analysis before cutting off treatment that its own medical staff had previously deemed to be medically appropriate for the inmates.

Even if it had extensively studied the issue before deciding to stop gender-affirming care, the decision might still violate the U.S. Constitution’s Eighth Amendment’s protections against “cruel and unusual” punishment, Lamberth wrote.

The Department of Justice had argued that the judge should defer to the policy decision of a democratically elected president, but Lamberth said a functioning democracy requires respect for “all duly enacted laws,” including those that blocked the executive branch from acting in an “arbitrary and capricious” manner.

Democratic self-governance “does not mean blind submission to the whims of the most recent election-victor,” Lamberth wrote.

The executive order said it was meant to promote the “dignity, safety, and wellbeing of women, and to stop the spread of “gender ideology” which denies “the immutable biological reality of sex.” But the inmates receiving hormone treatments had little interest in promoting any ideology, and were instead taking “measures to lessen the personal anguish caused by their gender dysphoria,” Lamberth wrote.