Amnesty's refugee coordinator, Graham Thom, met with officials from Immigration Minister Chris Bowen's office on Monday morning to discuss the organisation's inspection of the island last week.
"I think they acknowledged that the situation in the centre at the moment is far from ideal. They want permanent structures there, they want freedom of movement, they want processing," Dr Thom told Radio Australia's Pacific Beat program.
Immigration Minister Chris Bowen's office denied that was said.
"This is inaccurate and a misrepresentation of what was said. The Government has always said - as the expert panel recommended - that with the urgent setup of Nauru we would begin with a temporary facility and move to permanent structures as soon as possible," a spokeswoman said in a statement.
"It is unfortunate that Mr Thom [sic] has seen fit to relate details of a private meeting."
Dr Thom said asylum seekers on Nauru were becoming increasingly distressed at the lack of progress towards processing their claims.
He said they were upset by the Australian Government's announcement last week that it would release asylum seekers into the community in Australia on bridging visas while they wait for protection visas.
"We were on Nauru when that announcement was made and it was heartbreaking for the men that were there. None of them could understand why, it was genuinely a kick in the guts for those guys."
Fears are growing among refugee advocates for the wellbeing of an Iranian asylum seeker, known as Omid, who has been on a hunger strike for over a month.
He left Nauru Hospital yesterday and was taken back to the detention centre.
Refugee advocates said it was because the hospital could no longer care for him.
An Immigration Department spokeswoman said the man was being given care by the health provider at the processing centre, and was being encouraged to drink and eat.
Graham Thom said Omid was in a poor condition last week.
"This is a man who was clearly suffering both physically and mentally because of what's happening to him."
Dr Thom said Omid had lost more than 19kg, was complaining of severe stomach pains and looked "very, very unwell".
"We are not medical practitioners but we have serious concerns about his ongoing health."