He was the son of James Leonard Denman (b.1830) and Lucy Ann Martha Stovall (1832-1866). He was married (18 Dec 1879) to Naomi Agnes Lewis, who was the daughter of Harleston Davis Lewis (1837-1913) and Jane A. (Walker) Lewis (1841-1867).

The 1860 census show James in the home of his parents in Franklin County, Georgia. 1870 census shows him living in the family home with his elder siblings, both parents having died by then (his father died in the Civil War). 1880 census shows James & Agnes living in Little Prairie, Bartow County, Georgia. 1900 census shows them living in Sonoraville, Gordon County, Georgia. 1910 census shows Agnes head of household with eight children living in Cassville, Bartow County, Georgia. 1930 census shows Naomi (widowed head of household with 2 children and a boarder) in Center Hill, Fulton County, Georgia. Her death certificate (15 Mar 1932) shows last place of residence and place of death as Atlanta, Fulton County, Georgia.

James Patrick and Naomi Agnes had 13 children:

Harlston Leonard 11/5/1880 [NOTE: the 1900 census shows this person as a female, yet         a WWI draft registration shows this person as a male. The names Harleston and                 Leonard are male Denman family names--grandfathers of this person, in fact.]
Alice Louise (Stewart-Dobson) 2/12/1882 - 3/13/1961
Ona Annette (m. Jesse M. Stewart) 7/4/1883 - 12/12/1918
James Lonnie 8/23/1885 - 9/8/1886
Martha Mazie 11/16/1886 - 10/25/1907
Lida Jenness (Stewart) 2/23/1888 - 7/31/1971
Benjamin Franklin 1/5/1891 - 2/3/1931
Julia Ella (Roberts) 8/31/1892 - 12/1/1962
Luther Theodore 3/10/1895 - 4/16/1918
Abbie Leitha 7/28/1898 - 12/1977
Agnes Floy (Cooper) 2/28/1900 - 10/1982
George Young 1/15/1902 - 3/10/1902
William Bradley 6/14/1904 - 6/21/1962