Opposition leader Belden Namah says the facility is illegal because it authorises the deprivation of the asylum seekers' personal liberty.
He has begun legal proceedings against the government of Papua New Guinea, seeking to have the centre declared unconstitutional.
Ian Rintoul, from the Refugee Action Coalition in Australia, says if the challenge succeeds, it will be a blow for Australia's international human rights record.
'On the nose'
He said: "It's already, I think, on the nose because it has tried to draw PNG into undermining the refugee convention.
"If it turns out that Manus Island is unlawful in PNG, then I think (Australia's) human rights record will be even more obviously revealed."
Lawyers for Mr Namah filed a summons with the National Court last Friday.
The summons seeks a declaration that the agreement between the PNG and Australian governments to establish the processing centre is unconstitutional because it authorises the deprivation of the asylum seekers' personal liberty.
Last month, Mr Namah said people could only be detained for lengthy periods in PNG if they had broken the law.
He argued the asylum seekers on Manus Island had not broken any immigration laws so they should not be detained behind the centre's high fences.
The summons asks the court to permanently restrain the PNG government from continuing to detain people at the centre.
It also seeks a temporary injunction preventing any more asylum seekers being sent to Manus Island while the case is heard.The National Court is yet to set a date to hear the challenge.
Prime Minister Peter O'Neill has said the detention centre is being run in accordance with PNG's laws.
PNG's attorney-general, Kerenga Kua, said earlier this month that asylum seekers had effectively consented to being held in a processing centre.
"When they left the shores of their own country, they understood fully well the kind of processing they would have to put through in order to be formally, officially approved as refugees," he said.