He first attended an important committee of the House of Commons in 1822, and continued to take notes till 1872. On his father's resignation in 1849, he was appointed shorthand writer to the Houses of Parliament. Like his father, he was interested in religious and philanthropic movements. He was for more than fifty years a member of the committee of the Religious Tract Society, and latterly its treasurer. He was also treasurer of the Baptist college in Regent's Park.
On Gurney's retirement as shorthand writer to the houses of parliament in 1872, the post was given to his nephew William Henry Gurney Salter. Gurney died at Tyndale Lodge, Wimbledon Common, on 12 August 1879, and was interred at the Norwood Cemetery.
Gurney brought out popular commentaries on the Bible. The best known of these was The Annotated Paragraph Bible, containing the Old and New Testaments according to the authorised version, with explanatory Notes, Prefaces to the several Books, and an entirely new selection of references to parallel and illustrative Passages, two vols., London, 1850–60, published by the Religious Tract Society. It was successful, and widely praised. The notes were prepared under Gurney's supervision.
Besides other bibles, Gurney published also The Revised English Bible, London, 1877, closely resembling the Revised Version (1885). The profits of his literary works he gave to the Religious Tract Society.
- Dictionary of National Biography. London: Smith, Elder & Co. 1885–1900. .