The decision was taken at a meeting of federal cabinet this afternoon.
Ms Gillard had been under pressure to act following growing calls for a national inquiry into explosive allegations by a senior New South Wales police investigator that the Catholic Church covered up evidence involving paedophile priests.
A number of senior Labor MPs, as well as key independents, had already voiced their support for action on a national scale.
Opposition Leader Tony Abbott also declared his support for a "wide-ranging" royal commission into child sex abuse but said it should not just focus on claims involving the Catholic Church.
Ms Gillard said the Government would take the coming weeks to consult stakeholders before announcing the terms of reference.
The Prime Minister said the commission would look at all religious organisations, state care providers, not-for-profit bodies as well as the responses of child service agencies and the police.
"The allegations that have come to light recently about child sexual abuse have been heartbreaking," Ms Gillard told reporters in Canberra.
"These are insidious, evil acts to which no child should be subject.
"Australians know... that too many children have suffered child abuse, but have also seen other adults let them down - they've not only had their trust betrayed by the abuser but other adults who could have acted to assist them have failed to do so.
"There have been too many revelations of adults who have averted their eyes from this evil.
"I believe in these circumstances that it's appropriate for there to be a national response through a royal commission."
Ms Gillard said she would speak with relevant premiers and chief ministers in the coming days to discuss how the royal commission would relate to any similar inquiries proposed or underway.
"Discussions will also take place with victims' groups, religious leaders and community organisations," she said.
"Further announcements, including the proposed commissioner and detailed terms of reference, will be made in coming weeks.
"I commend the victims involved for having the courage to speak out.
"I believe we must do everything we can to make sure that what has happened in the past is never allowed to happen again."
Pell welcomes inquiry
Catholic Archbishop of Sydney Cardinal George Pell has welcomed the announcement of a royal commission, saying the church will cooperate fully.
"Public opinion remains unconvinced that the Catholic Church has dealt adequately with sexual abuse," he said in a statement.
"Ongoing and at times one-sided media coverage has deepened this uncertainty.
"This is one of the reasons for my support for this royal commission.
"I welcome the Prime Minister’s announcement. I believe the air should be cleared and the truth uncovered."
The investigator whose allegations sparked calls for a national inquiry, Detective Chief Inspector Peter Fox, has told the ABC's triple j that the royal commission was what he had been pushing for.
He said he was stunned it had happened so quickly.
"And delighted, absolutely delighted, for all those victims out there because this gives us so much of an opportunity to get things right, to look at recommendations for laws that should be changed to protect kids," he said.
The NSW Government has set up an inquiry in response to the claims raised by Detective Chief Inspector Fox. A separate parliamentary inquiry into church sex abuse is already underway in Victoria.
Chief Inspector Fox today said he was "saddened" by the narrowness of the inquiry set up by NSW Premier Barry O'Farrell in response to his concerns.
"I'm not politically aligned to anyone, and I haven't always been a big fan of Julia Gillard personally, but my God she's had some guts this afternoon, a lot more guts than Barry O'Farrell," Inspector Fox said.
"He was so disappointing and some, thank heavens, have stood up and done the right thing."
'We need to do better'
Acting Minister for Families, Brendan O'Connor, told the ABC's 7.30 program the Government wants to ensure a broad range of institutions be examined.
"The child sex abuse offences, and indeed allegations of child sex abuse, are not confined to one church," he said.
"They're not confined to one religious organisation.
"Unfortunately, offences against children have occurred to children in state care and indeed, have occurred to children under the care in other religious organisations and of course also not-for-profit organisations.
"It would be very unfair and quite cruel to confine the examination - or the commission's examination to one body."
He added the royal commission decision was "a very significant one".
"This Government is responding I think to the need to ensure that voices are heard by the victims and their families, that claims are properly investigated," he said.
"This Government wants to provide this forum, this process, so that victims get an opportunity to tell their story, to have their story heard.
"Beyond that I think as a government we want to ensure through this vehicle, the royal commission, that institutions that are caring for children are responding properly and adequately to those allegations that have been made.
"Now what we know to date is that has not been the case. We need to do better."
Earlier this afternoon Mr Abbott pledged his support if the Government decided to establish a "wide-ranging" inquiry.
"Wherever abuse has occurred it must be tackled and it must be tackled vigorously, openly and transparently," Mr Abbott said in a statement.
"It's clear that for a long period there was insufficient awareness and insufficient vigilance when it came to predatory behaviour by people in positions of authority over children.
"Any investigation must be wide-ranging, must consider any evidence of the abuse of children in Australia, and should not be limited to the examination of any one institution.
"It must include all organisations, government and non-government, where there is evidence of sexual abuse."
Several Labor MPs had publicly declared their support for a nationally constituted royal commission, with some pushing for terms of reference that would allow it to look beyond the immediate allegations involving the Catholic Church.
Call for action
Fellow Labor MP Joel Fitzgibbon, whose NSW electorate covers the Hunter Valley, earlier said it had become clear to him that a royal commission would be in the best interests of victims, their families and the church.
"The victims and their families need and deserve an assurance that no crimes or serious breaches of trust and responsibility have gone unpunished and that action has been taken to ensure that no such crimes or breaches can occur in the future," he said in a statement.
"It is clear that only the powers of a royal commission can adequately deliver those assurances.
"Further, the Catholic Church will struggle to rebuild its reputation while ever questions about institutional wrongs are not properly tested.
"Worse, the church will remain subject to all kinds of allegations and innuendo unless a formal process is established, and all the church's ongoing good work will pass unnoticed and unappreciated until the allegations are properly dealt with."
Earlier today, independent MP Tony Windsor said he had written to the Prime Minister urging her to take action to deal with the issue on a national scale.
Fellow independent Rob Oakeshott also backed calls for a national royal commission, saying there needed to be an inquiry with coercive powers to get to the bottom of the problem.
"No-one or no entity should be able to hide from the police in the gathering of evidence, particularly in an area that involves sexual abuse and children," Mr Oakeshott told ABC News Online.
"If it's not a national royal commission with wide powers, it would certainly be something of an equivalent such as a special commission of inquiry."
The Greens and independent senator Nick Xenophon have also publicly backed calls for a national royal commission into sexual abuse claims.