The Mayor said a new base in the Royal Docks – the riverside area near London City Airport – will save the Greater London Authority (GLA) £61 million over five years.
But Conservatives said the plans were a "PR stunt" based on "dodgy numbers", and the cost of the move would likely spiral.
The GLA has a budget gap of almost £500 million over the next two financial years because of coronavirus, with police and fire services facing cuts.
In June, Mr Khan proposed moving City Hall to the Crystal – an eco-friendly events centre owned by the GLA – at the end of 2021.
The current City Hall building, custom built as the seat of London government in 2002, is rented from Kuwaiti landlords St Martins on a 25 year lease.
The rent was due to go up to £9.6 million next year, with an extra £3 million of business rates and utilities – but Mr Khan will use a break clause to abandon the lease.
Even with a rent reduction the financial benefits of moving were "impossible to ignore," according to City Hall.
The Mayor first said the move would save £55 million over five years, but now believes it will be £61 million, with £126 million savings over a decade.
"I know that City Hall is a landmark building for many," Mr Khan said.
"But as Mayor, I will always focus my severely limited budget resources on frontline public services and supporting Londoners and our recovery from this pandemic, rather than on high City Hall building costs.
"The Royal Docks is an amazing place, and we have the opportunity to turbo-charge the regeneration of the area, just as the opening of City Hall did for its surroundings."
But critics fear an east London base will make the new City Hall less accessible to many Londoners, as well as to businesses and the media – diminishing the power of city government.
Some London Assembly members also feared being cut off from the Mayor, as the original plans gave Mr Khan two offices – one at the Crystal and one in central London.
The Mayor has now confirmed that he will be based in the Royal Docks. Assembly members can choose offices at the Crystal or in fire brigade buildings in Southwark – though public meetings will be in Newham.
Labour group leader Len Duvall – who has served on the London Assembly since it was set up in 2000 – said the Mayor and Assembly "must remain together".
"We believe this is essential for maintaining strong scrutiny, and though we account for twelve of twenty five Assembly members, we believe our view is held by the majority," he said.
"Though our preference was to see the heart of London government remain in the heart of London – and in City Hall, an iconic building – we recognise that Covid-19, and future budgetary pressures, have rendered that incredibly difficult."
Conservative Assembly leader Susan Hall also indicated that her group would move to the Crystal – but said Mr Khan's plans were "disappointing" and "half baked".
"The Mayor was offered a substantial rent reduction by the landlord of the existing building," she said.
"Instead of accepting it, Khan has chosen a flawed plan to move. We will doubtlessly see costs spiral and fewer savings than he promises.
"Instead of symbolic gestures, we need Khan to identify significant savings to protect London's frontline services," she added.