The gradual repatriation of more than 650,000 Rohingya Muslim refugees back to Myanmar from Bangladesh, scheduled to begin on  Tuesday, has been postponed amid widespread fears that refugees would be forced to return, a Bangladesh official said on Monday.

“The main thing is that the process has to be voluntary,” said Abul Kalam, the refugee and repatriation commissioner, adding that paperwork for returning refugees had not yet been finalized and transit camps had yet to be built in Bangladesh.

It was not immediately clear when the process would start.

His announcement came amid growing concerns among international aid workers and the Rohingya that they could be coerced to go back to Myanmar, which most fled just months ago, escaping attacks by security forces and Buddhist mobs.

“If they send us back forcefully we will not go,” Sayed Noor, who fled his village in Myanmar last August, said over the weekend, adding that Myanmar authorities “have to give us our rights and give us justice.”

“They will have to return all our wealth that they have looted and hold people accountable. They will have to compensate us. We came here because we are fighting for those things,” he said. “If we don’t get all of this, then what was the point of coming here?”

Eventually, the more than 650,000 Rohingya who have fled Myanmar since August were to leave Bangladesh in a process that guarantees them “safety, security and dignity,” according to an agreement both countries signed late last year.

David Mathieson, a longtime human rights researcher who has spent years working on Rohingya issues, heaped scorn on the agreement ahead of the latest announcement.

“It’s a fantasyland, make-believe world that both governments are in,” he said in an interview in Yangon, Myanmar’s main city, noting that security forces there had just forced hundreds of thousands of Rohingya across the border. “Now you’re expecting them to come back, as if they’re in a conga line of joy after what you did to them?”

The Rohingya Muslims have long been treated as outsiders in largely Buddhist Myanmar, derided as “Bengalis,” illegal immigrants from Bangladesh, even though generations of Rohingya have lived in Myanmar. Nearly all have been denied citizenship since 1982, effectively rendered stateless. They are denied freedom of movement and other basic rights.

Many of the people who fled earlier violence and moved into displacement camps inside Myanmar have been unable to leave those settlements for years.

Most Rohingya lived in poverty in Myanmar’s Rakhine state, near the Bangladesh border. Marked by their religion and their language most Rohingya speak a dialect of Bengali, while most of their neighbors speak Rakhine they are easy to target.

The recent surge of violence erupted after an underground insurgent group, the Arakan Rohingya Salvation Army, attacked at least 30 security outposts in Myanmar in late August. The military and Buddhist mobs then launched retaliatory attacks on Rohingya across Rakhine in a frenzy of killings, rapes and burned villages. The U.N. has described the violence as “textbook ethnic cleansing.