Mr Inouye, a decorated veteran of World War Two and chairman of the powerful Senate Appropriations Committee, died at the Walter Reed National Military Medical Centre of complications from a respiratory illness.
A statement issued by his office said "Aloha" was the last word he spoke.
Mr Inouye's death was announced on the Senate floor by Majority Leader Harry Reid of Nevada, who hailed him as one of the "greats of this body."
Under Hawaii law, the governor, Democrat Neil Abercrombie, will name a successor to fill Mr Inouye's seat until a new senator is picked in the 2014 general election.
First elected to Congress as Hawaii's first full-fledged member of the US House of Representatives, Mr Inouye took office on August 21, 1959, the date Hawaii became a state, and he went on to win election to the US Senate in 1962.
After nine consecutive Senate terms, he was the only member of a state's original congressional delegation still serving on Capitol Hill at the time of his death.
He was also the second-longest-serving member of the US Senate after the late Senator Robert Byrd of West Virginia, and became the most senior senator when Mr Byrd died in June 2010.
Mr Inouye thus assumed the post of president pro tempore of the Senate, making him third in line to succeed to the US president, after the vice president and speaker of the House.
Mr Inouye began his public service at the age of 17, when he enlisted in the US Army shortly after the Japanese attack on Pearl Harbor and joined the 442nd Regimental Combat Team, a unit consisting entirely of US soldiers of Japanese ancestry.
Serving in Europe with distinction, Mr Inouye lost his right arm while charging a series of German machine-gun nests on a hill in Italy in April 1945.
It injury earned him a Purple Heart but ended his ambitions to study to be a surgeon.
Instead, he earned a law degree and entered politics after the war.
When asked in recent days how he wanted to be remembered, according to his office's press statement, Mr Inouye replied, "I represented the people of Hawaii and this nation honestly and to the best of my ability. I think I did OK."
Former head of news with Hawaii Public Radio, Kayla Rosenfeld, told Radio Australia's Pacific Beat that everything Mr Inouye did was as a "proud Nisei" - a first-generation Japanese on Hawaii.
"Even here in Hawaii he had to defend his position as a gentleman of Japanese ancestry, he had to work his way up the ladder," she said.
"All of that was his inspiration for what he did and his inspiration filtered down into so many different things."