Ghanem, 69, had been chairman of Libya's state-owned National Oil Corporation (NOC) before defecting last year several months after opponents of Muammar Gaddafi had risen up against the Libyan leader and begun a rebellion.
As NOC chairman since 2006, Ghanem helped steer Libya's oil policy and held the high-profile job of representing Libya at OPEC meetings, often visiting Vienna for meetings in that role.
"He was found dead in the Danube river at 8:40 a.m. (0640 GMT). There is no suspicion at all of foul play at this stage. The corpse exhibited no signs of violence," a Vienna police spokesman said.
He said an autopsy would be performed to determine the cause of death.
After making a final break with the Gaddafi administration last year, Ghanem first appeared in Rome, saying he had defected because of the "unbearable violence" being used by government forces to try to put down the rebellion.
He was believed to have been living in Europe in exile since then but was still closely associated with Gaddafi's rule by Libya's new leaders and had ruled out returning home.