Web Sites of Interest
Was your ancestor in the entertainment business? This site could well be the place to search.
The Rootsweb hosted site which is run by David Jassie, includes extracts from the 1851 and 1881 census by entertainment occupation and has a search engine which allows you to search for your ancestor by name. It is not particularly aimed at Hertfordshire - but as people in this profession tend to move about it could be worth looking at.
If you can add to the information given above tell me.