RBS CEO Ross McEwan characterized the settlement as a "stark reminder" of the heavy price British taxpayer and the bank itself paid for the global ambitions of the institution under disgraced former boss Fred Goodwin.
R.B.S. said it would be reimbursed $754 million of the settlement amount under indemnification agreements with third parties.
The latest settlement relates to $32 billion in so-called residential mortgage-backed securities.
The 71 per cent taxpayer-owned bank will pay the penalty from the Federal Housing Finance Agency as part of long-running disputes with authorities in the USA over the sale of bonds backed by toxic mortgages.
The bank still faces a separate investigation by U.S. Department of Justice over the sale of risky mortgages.
The settlement relates to approximately USD32.0 billion in mortgage-backed securities which were issued by RBS before the 2008 crisis.
RBS had already put by £6.6 billion to cover U.S. mis-selling claims.
Analysts at Jefferies said the settlement was $1 billion more than they had been expecting, although it is largely covered by existing provisions. Following this settlement, RBS's unresolved RMBS litigation matters involve the issuance of less than United States $1 billion of RMBS issued primarily from 2005 to 2007, assuming court approval of an agreed and provided for settlement.
RBS said following this agreement, unresolved RMBS litigation matters involve the issuance of less than $1bn of RMBS issued primarily from 2005 to 2007.
The settlement came after it reached a deal with thousands of investors last month to avoid a high-profile court trial. At the end of its first quarter, RBS held a provision of $8.3 billion against RMBS, of which $4.55 billion related to FHFA.
UK Treasury chief Philip Hammond said earlier this year that the government is likely to lose billions of pounds when it sells of its stake at the bank.
He said: "This settlement clears a major hurdle for the bank, though there remain further significant RMBS (mortgage-backed security) related costs, such as the DoJ".