He first married Matilda Viers on August 7, 1828 at New Rumley, Ohio. They had three children:
1). Hannah Custer ( - ) died in infancy.
2). Brice William Custer (April 23, 1831-April 6, 1904).
Married Marie Stockton in 1850.
3). John Viers Custer (April 22, 1833-July 29, 1836). Died in childhood.
Matilda Custer died on July 18, 1835. He then married Marie Kirkpatrick Ward on February 23, 1836 at New Rumley, Ohio. They had seven children together:
1) James Custer (1836-1837) Died in infancy
2) Samuel Custer (1838-1839) Died in infancy
3). George Armstrong Custer (December 5, 1839-June 25, 1876). Brevet Major General, US Army, Civil War. Lieutenant Colonel, 7th US Cavalry, died at the Battle of Little Big Horn.
4). Nevin Johnson Custer (July 29, 1842-February 25, 1915).
5). Thomas Ward Custer (March 15, 1845-June 25, 1876). Double Medal of Honor Recipient, Civil War. Captain, 7th US Cavalry, died at the Battle of Little Big Horn.
6). Boston Custer (October 31, 1848-June 25, 1876) Died at the Battle of Little Big Horn.
7). Margaret Emma Custer (January 5, 1852-1910). Married 1st Lieutenant James Calhoun, 7th US Cavalry. He died at the Battle of Little Big Horn.
John Fedele Custer