10/16/1850 Montgomery Co, TX. Listed in that census were William Whitley, 31, wife Emily, 23, children Mary, 8, Robert, 6, Elijah, 4, brother Josiah Whitley, 24, and Emily's mother Mary Collard, 60. William and Josiah were farmers, William's real estate totaled $1100. William, Josiah, and Mrs Collard were born in IL, Emily in MO, the little guys in TX.
7/9/1860 Same place. That year the census taker saw fit to only record initial: W Whitley, 41; E, 33; M, 16; R, 14; E, 12; J, 10; W, 7; W, 4; and M Collard, 75. William had amassed an estate valued at $2665/3000 and was still farming. Mrs. Collard had $1000 in personal property. Next door was the W Watters family with the future wife of son William Jr.
8/6/1870 Pct 4, PO Huntsville, Walker Co, TX. Recorded in the census were Wm Whalley, 46, farmer with property worth $1500/500; wife Emily, 42; Robt, 24, merchant with $800 in personal property; Lige, 22, and John, 19, farmers; Wm, 16, and Warrick, 14, at school; and Oppiera Owens, 17, house worker. Next door was the black George and Mariah Whitley family. He was a farm hand without property so may be William's helper. Brother Sharp was in the same area.
According to son William's death certificate, both parents were born in Miss.