Bellevue Hospital is moving 500 patients due to storm damage, Mayor Mike Bloomberg said.
Officials are in the process of finding beds for the patients, the mayor said.
Workers have been pumping about 17 million gallons of water out of the basement.
Bellevue lost power during Monday night's storm and had been operating on backup generators since then.
Even with backup power operating, the outage left many hallways and rooms in the hospital dark and knocked out key medical equipment.
It has been diverting some patients since the initial outage.
Elsewhere on Monday night, rescuers and staff at New York University Langone Medical Center evacuated 300 patients after it lost generator power due to the storm.