Belhaj - who was seized with his then-pregnant wife Fatima Boudchar and four children while on their way to the United Kingdom - has said that a tip-off from MI6, the British foreign intelligence agency, led to their capture.
Ms Boudchar, who visited parliament yesterday with son Abderrahim, was pregnant with him at the time and was freed shortly before giving birth.
MPs also sought assurance that such a case would not happen again from attorney general Jeremy Wright, who argued that it was hard to balance the issue of national security and its cooperation with foreign agencies who may use torture.
He did not receive any compensation in the settlement, which Mr Wright said did not involve any admissions of liability, including concerning allegations against former Foreign Secretary Jack Straw and Sir Mark Allen, former head of counter-terrorism at MI6. Belhaj has said he will settle for £1, but insists he and his wife also receive an apology.
Speaking in Istanbul, where he is involved in peace talks, Mr Belhaj said: "The wording of the apology was heartfelt.There was a feeling of concern, an admission of the shortcomings, an expression of unreserved apology, lessons learned, admission of failings and an expression of disappointment towards the global partners that I was handed over to. We accept this was a failing on our part".
"Today's candid apology from the government helps restore the humanity and dignity so brutally denied to my clients during their ordeal, and is warmly welcomed". Mr Belhaj, who is now a politician in Libya, was imprisoned for six years, repeatedly tortured and sentenced to death. "I welcome and accept the prime minister's apology, and I extend to her and the attorney general my thanks and honest goodwill".More news: Delta Air Line passengers evacuated on tarmac
More news: Man Utd boss Mourinho ready to allow Martial leave
More news: Walmart To Acquire Majority Stake In Flipkart
"On behalf of Her Majesty's Government, I apologise unreservedly. Now, at last, justice has been done", he concluded. "I accept the government's apology". We hope our case will serve as a marker for future generations. "Britain has made a wrong right today, and set an example for other nations to follow".
"By today's settlement, I look forward to rebuilding my life with dignity and honour, and living free from the weight of these events with my husband and our five lovely children". A great society does not torture; does not help others to torture; and, when it makes mistakes, it accepts them and apologises.
The UK allowed the USA to conduct air strikes against Libya from bases in the UK in 1986, while MI6 is alleged to have been involved in a plot to assassinate Gaddafi in 1996.
"This is a historic day", Boudchar told reporters outside Britain's parliament.
"The UK Government has learned many lessons from this period".
The prime minister added that "we should have understood sooner the unacceptable practices of some of our worldwide partners".