After seven years of negotiations between the two nations, Mr Abbott and his Japanese counterpart Shinzo Abe have settled on the major points of the bi-lateral agreement.
On the Australian side beef - which was a major sticking point in the agreement - is the big winner.
A hefty 38.5 per cent Japanese tariff that currently applies on frozen beef will be halved to 19.5 per cent over 18 years, with deep cuts in the first year.
The chairman of the Australian beef industry's free trade taskforce, Lachie Hart, says the deal will be worth $5.5 billion to the industry over 20 years.
There are also significant advantages for other agricultural products, with fruit and vegetables, seafood, sugar and wine among the winners.
The duty-free quota for cheese - Australia's single largest dairy export to Japan - will be boosted from 27,000 tonnes per year to 47,000 tonnes annually.
Tariffs will remain on rice, though, with its special cultural significance to Japan.
Japanese exporters will see Australian tariffs lowered on electronics, whitegoods and cars, and Australian consumers will see prices lowered as a result.
Under the deal, Japanese-made cars will be, on average, $1,500 cheaper.
Speaking in Tokyo on Monday evening, Mr Abbott said the deal was of historical significance.
"This is the first time that Japan has negotiated a comprehensive economic partnership agreement or free trade agreement with a major economy, particularly a major economy with a strong agricultural sector," he said.
"I hope that, thanks to this agreement ... Australia can be pivotal to ensuring that in the years and decades to come, the people of Japan have energy security, resource security and food security."
Mr Abe hailed Australia as "strategic partner" that shared "universal values" with Japan.
"In today's meeting, I have confirmed with Mr Abbott that we will elevate this strong bilateral relationship between our two countries to a new special relationship, so that we can work on forging an even stronger partnership together," he said.
The deal is expected to be formally signed when Mr Abe visits Canberra in July and will come into effect later this year.
Both countries gain from agreement, experts say
International trade specialist Dr Andy Stoeckel says the deal makes sense for both Australia and Japan.
"We are good at beef production, and also wine production because we've got the land and the climate and so forth, but not very good at making motor cars because we don't have the scale of operation," he told PM.
"It makes sense, and the efficient use of resources, if we concentrate on our strengths, which is beef and wine, say, and Japan concentrates on their strengths which is making motor cars. We both gain and we have a better allocation of resources globally, also within each economy."
The announcement came after a day of pomp and ceremony, including Mr Abbott's meeting with Japan's emperor.
Australian business groups are already urging the Government to release all the details of the deal.
The Chamber of Commerce and Industry says the negotiations have not been transparent, and says the implications for Australian companies remain unclear.
"Obviously through the course of negotiations some things are able to be sealed and some things are harder, and perhaps don't make it to the final deal," the chamber's director of trade and international affairs Bryan Clark told PM.
"So it's important that we scrutinise that final negotiated text so that we understand exactly what the detail is."
He says the Government has released thorough information about previous trade deals.
"We have seen some of the previous free trade agreement texts, most recently the Korean one, where there are some aspects we're pretty disturbed at," he said.
"We would prefer it if we could see the text before it's ratified."
The week-long trade delegation heads to South Korea tomorrow, where Mr Abbott is expected to sign a FTA with that country.