Ryuji Sonoda said that in the wake of the allegations it would be "difficult" for him to stay as coach.
"I would like to deeply apologise for causing trouble to all the people concerned with what I have done and said," he told a news conference.
"I think it will be difficult for me to continue being engaged in the training program any longer. I wish to submit my resignation."
The scandal came to light when 15 of Japan's top women judokas wrote to the country's Olympic committee, accusing the head coach of physically abusing them.
It's alleged that he swore at them, slapped them, and struck them with bamboo swords.
Others say they were forced to compete even though they were injured.
The All Japan Judo Federation has apologised to the country's top female judokas and issued Mr Sonoda with a written warning.
Japan's sports minister Hakubun Shimomura has instructed the country's Olympic Committee (JOC) to launch a fresh inquiry into accusations of physical abuse against female judokas by coaching staff.
Earlier Mr Shimomura said the abuse claims have prompted the need for a rethink of physical discipline in education.
"It is time for Japan to change the idea that use of violence in sports including physical discipline is a valid way of coaching," he said.
He has asked the JOC to "investigate the matter using international guidelines as [a] measure and make a proper judgment."
Japan, birthplace of the martial art, endured one of their worst Judo medal hauls in London, taking home only one gold despite being tipped to win at least half of the 14 events.