First, we have the birth record of his son, Ezra Riley for 1862
This shows his birthplace to be Woolwich Arsenal. His father's occupation is listed as Journeyman Blacksmith.
Another piece of evidence to verify the family's residence is this letter obtained by Mary Jane Riley Lacke.
This letter shows that he was a Manager of Cable and Telegraph Works and lived in North Woolwich in Dec 1865. It is interesting to note that he paid dues through 1878. He had moved away at some point as we know he was in Wales in 1871.
There are always alternatives to get at some of the information we need to put together the story. It just isn't as easy as going to a Census Record.