Japanese media has reported at least three whaling vessels left the country on Friday for the Southern Ocean, where they will begin their annual hunt.
The Opposition has called for the Australian Government to send a customs vessel to observe the hunt.
The Australian Environment Minister, Tony Burke, says customs vessels are engaged in other important work.
"I'm not sure which drug bust or which drug operation the Liberal party would like customs to abandon so they can go whale watching in the Southern Ocean," he said.
"There's no proposal from the Opposition that a customs vessel be allowed to do anything, it's simply they want one to go down there and do some whale watching. Well that is absurd."
'No backwards steps'
The Australia Government has condemned the whaling fleet, and says it is doing everything in its power to resolve the situation diplomatically.
But Greenpeace spokeswoman Julie Macken says the government is not backing its words up with action.
"They seem keener to repel boats carrying asylum seekers than they are to stop Japanese whalers from slaughtering these extraordinary animals," she said.
"If they had half the will to end this that they show about being tough on our border protection, this would have been finished a long time ago."
Mr Burke says the Australian Government has shown it is serious about stopping whaling by taking the matter to the International Court of Justice.
"The approach of the Australian Government has been to take every diplomatic action that a government can take.
"We have been known throughout the International Whaling Commission as a government that takes no backwards steps and a government that takes a very hard line on trying to get the commission to a pro-conservation position."
The government expects the case is likely to be listed for hearing in the second half of 2013.
The Japanese fleet plans to hunt up to 935 Antarctic minke whales and up to 50 fin whales through March.
The fleet's departure comes weeks later than expected and days after a US court ordered environmental group Sea Shepherd to stay at least 500 metres from whaling vessels after a complaint from Japan.
Sea Shepherd, led by captain Paul Watson, each year disrupts the expeditions.
In a statement on its website, Sea Shepherd called the US court ruling "the first shot of the season" by Japanese whalers.
Confrontations between the whalers and activists have escalated in recent years, and the Japanese cut their hunt short in early 2011 due to Sea Shepherd harassment.
Japan hunts whales using a loophole in a global moratorium that allows killing the sea mammals for what it calls "scientific research", although the meat is later sold openly in shops and restaurants.
Sea Shepherd's ninth campaign, named Operation Zero Tolerance, is its largest ever against Japan's whale hunt and involves four ships, a helicopter, three drones and more than 100 crew members.
Three of the vessels, the Steve Irwin, Bob Barker and Brigitte Bardot, are all at sea while the Sam Simon is at an undisclosed location.