There are roughly 240 Tamils on Nauru and Manus Island, from where America has agreed to take up to 1,250 refugees.
Earlier this year, US President Donald Trump described the agreement as a "dumb deal", and his administration said it would only honour it to maintain a strong relationship with Australia and then only on condition refugees satisfied strict checks.
But America's post 9-11 anti-terror legislation, the Patriot Act, will likely disqualify a large number of the Tamils.
Because the since-defeated Tamil Tiger rebels were so heavily enmeshed in the Tamil-dominated north, they would be captured a provision which excludes people deemed to have provided "material support" to terror groups.
Kathy Newland, co-founder of the Washington DC Institute for Migration Policy, described material support as a "really broad" category.
"Its everything from money, food, shelter, almost anything, training, to someone associated with a terrorist group," she said.
At her home near Jaffna in Sri Lanka's Tamil-dominated north, a mother and widow Mala (not her real name) waits anxiously news from her son.
The 24-year-old man is on Nauru, having tried to reach Australia by boat five years ago.
He had gone there to seek help for his sister, a Liberation of Tigers of Tamil Eelam (LTTE) cadre injured in the last days of the war. The family said they were denied proper help.
The man's mother worries what will happen to her son if he is to be sent back to Sri Lanka.
"If he [is] deported to Sri Lanka, I don't know what will happen," she said.
"The villagers say the people who [are] deported [are] arrested and detained."
Last week, the United Nations found she had good reason to be fearful.
Ben Emmerson, the UN's special rapporteur on counter terrorism's impact on human rights, said the Tamil community had "borne the brunt of the State's well-oiled torture apparatus," something he said remained the case today.
"Any person suspected of association, however indirect, with the LTTE remains at immediate risk of detention and torture," Mr Emmerson said.
The envoy said meetings with prisoners detained under anti-terror laws had revealed disturbing details of ongoing mistreatment.
"I heard distressing stories of extremely brutal methods of torture," he told a media conference in Colombo last Friday.
Mr Emmerson said the methods being used included "beatings with sticks, the use of stress positions, asphyxiation using plastic bags drenched in kerosene, the pulling out of fingernails, the insertion of needles beneath the fingernails, the use of various forms of water torture, the suspension of individuals for several hours by their thumbs, and the mutilation of genitals".
'I was beaten during detention'
The ABC spoke yesterday to one Tamil deportee — who did not want to be identified — who also said he had been abused by authorities.
After being deported from Europe in April, the man said police arrested him, and over the course of two weeks in jail he was beaten and robbed because his friends were suspected of having Tiger links.
"I was detained after my deportation," he said.
"I was beaten up during my detention — they beat me using shoes and sticks. I have the scar marks."
The new allegations raise questions about the value of assurances given by Sri Lanka that returnees will not be targeted or mistreated.
Under the refugee convention, receiving countries are not allowed to send someone back into danger.
Australia has previously been given assurances by Sri Lanka that potential returnees would not be mistreated.
But the new allegations raise questions about those guarantees, and what alternatives Australia has for the detainees in its care.
"We're continuing to work with Nauru and with Papua New Guinea to find third countries to take them, otherwise they do have options to be resettled in PNG," Foreign Minister Julie Bishop said on Wednesday.
Ms Bishop is in Colombo meeting Sri Lankan officials today, and said Australia sought to "encourage" Sri Lanka to live up to its human rights commitments.