Deputy Opposition Leader Julie Bishop says many Sri Lankan asylum seekers turn out to be economic migrants, not refugees.
She says Sri Lankan asylum seekers should be deported before they get access to Australia's legal system.
"Sri Lanka is already making a significant effort by preventing many boats from leaving their shores, however those who make it through should be the subject to an immediate arrangement to be transferred back to Sri Lanka without coming to Australia," Ms Bishop said.
"There is an extremely high rejection rate for Sri Lankan asylum seekers with the vast majority proving to be economic migrants.
"But once they are in Australia they can pursue their claims for asylum through our courts regardless of the merit."
Ms Bishop was speaking after it was revealed Indonesian police were holding around 50 Sri Lankan asylum seekers, including seven children, whose boat broke down on the way to Australia.
The 50 Sri Lankan asylum seekers were discovered adrift by fisherman near the Mentawai Islands. After nine days at sea, they were dehydrated and hungry.
They told their rescuers they were fleeing violence in Sri Lanka and and had hoped to reach Australia.
The fishermen towed them ashore where they were given food, water and medical attention and handed over to police.
Earlier on Sunday, Greens Leader Christine Milne told ABC's Insiders program that authorities need to better advertise Australia's increase to the refugee intake.
She says there needs to be a greater focus on Australia's pledge to increase the refugee intake to 20,000 a year.
"Let's get into the camps and tell people not to get on the boats because the humanitarian increase is coming in and they will soon be able to get safe passage to Australia. That's what they need to hear," she said.
"The Prime Minister said 20,000 places will be made available, and we are doing everything in our power to bring that forward so that people have got the promise of a safe pathway.
Senator Milne says the Labor Party and Coalition's policies of deterrence have routinely failed to stop asylum seekers making the dangerous journey to Australia by boat.
"[Asylum seekers] have said themselves they are desperate to come to Australia and they don't see any hope," she said.
"While the Government's putting out videos of deterrence, it is not out there saying to people 'don't get on the boats because we're about to increase the humanitarian intake'."
Meanwhile a coalition of Australian church leaders has called on the Government to further increase its refugee intake.
Leaders from denominations including the Salvation Army and Anglican, Catholic and Uniting churches say they are concerned about the Government's new legislation to allow offshore processing.
President of the Uniting Church, Reverend Andrew Dutney, says offshore processing can be traumatic for those involved.
"It's too late to change the legislation at this point and we accept that but there are other things to consider so we're very concerned that there be a genuine commitment to follow through on all of the recommendations of the Houston report including the increase of the numbers of asylum seekers and refugees to be accepted in the program," he said.
A panel chaired by former defence force chief Angus Houston recently recommended increasing Australia's annual humanitarian intake from 13,000 to 27,000 refugees within five years.