1840 Southern Division of Bracken Co, KY. He lived in the same area as his father according the census records but the census is too blurry to read.
8/14/1850 Dist 1, Pendleton Co, KY. By the next census they had moved and apparently it was recent as James had no occupation or property. He was listed at age 34, Mary was 32, and the children were Andrew, 11, Susannah, 8, James W, 7, daughter Ricy, 4, Arthur, 2, and Lewis, 1.
7/10/1860 Same place, PO Morgan. That year the census recorded James at 45, Mary A at 43. The family had grown to include: Andrew J, 21, Susan, 18, James W, 16, Rosa, 14, Arthur, 12, Lewis, 9, Sarah, 4, and Margaret, 1. James was a farmer but had only personal property valued at $300.
6/20/1870 Still there. According to the census, James was a 55 year old farmer and owned real estate worth $850, personal property worth $300. Mary A was 52 and still at home were Ricean, 24, Arthur M, 22, Lewis W, 18, Sarah J, 14, Andrew J, 30, Martha, 22, Ada F, 3, and Jas T, 8 months. Martha was probably Andrew's wife and the smallest children were theirs. Also living with them was 91 year old Mary Woodyard, Mary Ann's mother.
1880 Dist 165, Callensville, Pendleton Co, KY, could be the same place as previous census records. James was 65 that year, Mary A, 62 and living with them were Lewis, 30, and Sarah, 24. Both men were farmers.