HMRC has had to return nearly half-a-billion pounds to taxpayers since April 2015, according to analysis of government statistics.
Royal London claims the Revenue is overcharging on two areas of the tax system, and has had to refund £493 million to taxpayers as a result.
Savers using the new pension freedoms are among the hardest hit, as income tax on pension withdrawals is taken on an emergency tax basis.
Taxpayers have had to claim back £262 million since the pension freedoms were introduced in April 2015.
The other area highlighted by the analysis was extra stamp duty charged on ‘second homes', which are not actually second homes.
This applies to people moving home, who have completed the purchase of a house before selling their existing house, incurring an extra 3% stamp duty charge on the transaction.
Since April 2016, a total of £231 million has been refunded to homeowners who were overtaxed in this way.
Helen Morrissey, personal finance specialist at Royal London, said:
"Taxes like the second home stamp duty were designed to clamp down on buy-to-let landlords, not people whose house sale took longer than they expected.
"Moving house is already stressful enough without having to pay thousands in extra tax to HMRC and then have to claim it back.
"And these figures do not even include the people who are overtaxed without realising it and never claim a refund."
Steve Webb, director of policy at Royal London, added:
"HMRC operates a system of tax first, ask questions later. It is time to speak up for ordinary citizens who are forced to pay excessive amounts of tax and then go through the hassle of claiming it back."
Contact us to discuss your tax issues.