Numerous consumers have complained about excessive charges, with one ABC listener reporting being initially billed $11,000 for a document download.
Senator Conroy says the profit for international roaming stood at 1,000 per cent when Australia started a joint study with New Zealand into the problem.
That has now fallen to 300 per cent, but Senator Conroy says a paper out today proposes price caps and other measures to bring it down even further.
"We're announcing two things today. We're releasing adiscussion paper which we've worked on for the last couple of years with the New Zealand government which is saying we need to act now," he told ABC News 24's Breakfast program.
"We've done a study and what we found was at the beginning of the study the mark-up, the margin for international cost calls between here and New Zealand was 1,000 per cent.
"With the spotlight on, it's come down to 300 per cent but that is not good enough."
Senator Conroy says Australians are being "ripped off" and Australia and New Zealand are negotiating an agreement to introduce legislation looking at possible price caps to "stop this rort".
"I'm directing the ACMA to put in place a standard which will see mobile phone companies notifying their customers when they're overseas of the cost of a call, the cost of sending a text, the cost of going online, and giving them the option to opt out," he said.
"As many Australians will know, you've come back from overseas and gone 'oh my God!'"
Senator Conroy says in one example, an Australian woman who was ill racked up a $4,000 phone bill while in New Zealand.
"That is more than the cost of the holiday - and she desperately needed the holiday," he said.
"You're talking about thousands of dollars in some cases for doing the simplest of things.
"And some people in the community have gone into the shop of the telcos and said 'please, can you turn this off' and they get back and they still have a phone bill.
"In those examples the telcos usually are reasonable, but the fact you can run up these bills without Australians' knowledge of the costs is not good enough."