Officials said the hotel is under construction. The workers were pouring concrete when the incident happened.
Of the two surviving workers, one man in his 30s suffered minor injuries but refused treatment. The other worker was not injured.
"There are rules for scaffolding,” construction worker John Jennings said. “There’s rules hooking off, there’s rules for the workers. They do participate in a safety program, so these sort of of things don't happen or happen less.”
The Occupational Safety and Health Administration (OSHA) will investigate the incident, authorities said.