JP Morgan's Australian Mortgage Industry Report surveyed 52,000 households and identified that around 10 per cent of investor loans could be at risk.
The report warns that pressure for banks to increase capital levels on certain types of investor lending by three or four times current requirements could create "extreme effects".
The most "significant changes afoot" are for investor loans that are "materially dependent on property cash flows" to service the repayments.
As a result, banks with loans dependent on cash flows could be forced to raise rates incrementally to manage the risk, potentially by as much as 3 percentage points in an extreme scenario.
Investors dependent on cash flows from rental income would also be "most sensitive" to interest rate rises of between 2 and 3 percentage points according to the research.
The warning comes as banks such as the Commonwealth and its wholly owned subsidiary BankWest consider further restrictions on investor loans to keep under regulatory requirements for growth capped at 10 per cent per year.
NSW poses biggest housing risk
The report's co-author, Martin North of Digital Finance Analytics, said investors most exposed were in New South Wales which has been the epicentre of steep real estate price growth.
"The highest sensitivity has been in New South Wales where house prices are significantly higher, as are borrowing commitments," Mr North said.
"Because the loans are bigger, because property is bigger, (New South Wales) households are more exposed in a rising interest rates environment.
"The risk in New South Wales is significantly higher than its perceived to be the case."
The report also identifies risk in Western Australia and Queensland because of the mining downturn and Victoria due to an oversupply of inner-city apartments.
Mr North also said current risk analytics used by the banks might not tell the full story because they look at historical factors based over the past 10 to 15 years rather the current real estate dynamics.
"The way that we think about this risk problem is different to the way that many of the current analytics that banks run with," he said.
It warns that "wealthy seniors" and "young affluent" investors could be the most severely affected, while others such as rural families and low-income households would be least exposed.
JP Morgan banking analyst Scott Manning said it will be important for the Australian Prudential Regulation Authority (APRA) to determine how it defines loans that are "materially dependent" on cash flows flows.
Mr Manning said APRA could "reverse engineer" concerns about investor loan exposure for outcomes that would meet the need for banks to be "unquestionably strong".
The report found that banks will be positioning themselves to minimise exposure and to meet capital requirements.
It identified the Commonwealth Bank and Westpac as having "the most to lose if heightened churn becomes a reality", while ANZ was best placed of the big four.