The Labour Party has criticised justice secretary Chris Grayling for pressing ahead with legal aid cuts despite an underspend on the budget last year.

Shadow justice secretary Sadiq Khan insists the government is wrong to claim legal aid costs too much in a bid to justify cutting £220m from the budget.

Mr Khan pointed to figures in the Ministry of Justice’s 2012/13 accounts which show an underspend of £56.4m last year.

This was made up by £28.9m from the criminal aid fund an £27.5m from the civil legal aid fund.

Mr Grayling has previously labelled the £2bn legal aid bill as “unsustainable”.

Mr Khan thinks the cuts could result in an unfair justice system.

He said: “Chris Grayling has some cheek. He is cutting legal aid, which could lead to many wrongs not being righted and innocent people going to prison. His justification is that legal aid costs too much, but his own figures show that less was spent last year than he predicted.

“Those who work in our justice system are already making big savings. Rather than attacking legal aid lawyers as fat cats, he should listen to them. This government’s legal aid plans would devastate our justice system. Yet Chris Grayling isn’t bothered – he would be quite happy if our justice system was only for the richest.”

The ministry said the reasons for the civil fund underspend included changes in provider behaviour and the introduction of remuneration cuts.

Lower court activity was one of the reasons cited for the criminal aid underspend.

A Ministry of Justice spokesperson said: “We have one of the most expensive legal aid systems in the world, paying around £2bn a year, for several years.

“The fact we spent less than feared last year is helpful but at only 1/40th of what is normally spent doesn’t mean we can avoid making further savings.

“Our proposals will ensure a legally-aided lawyer is available to those who most need one – and that legal aid remains sustainable for hard-working taxpayers.”

Mr Grayling will defend his changes to the justice system in front of parliament’s joint human rights committee this week.