William Elsworth, son of John and Anne Knight Elsworth, was born in England 7 September 1822 in Northampton, Northamptonshire, England. The family moved from Northampton to Leicester, Leicestershire about 1827. He married Eliza Clarke, daughter of William Clarke, 21 May 1844 at Harvey Lane Chapel (Baptist) in Leicester, Leicestershire, England. William and Eliza had three children Sam, Elizabeth, Catherine born in Leicester prior to their immigration to the United States. William's occupation in England was a butcher; he was listed as a butcher in Grey Street in the 1846 Leicester Trade Directory.
William and his family emigrated in June 1848 and settled in Wisconsin where he turned to farming for a living. William declared his intention of becoming a United States citizen on 17 March 1853 and was admitted as a citizen 6 May 1868. In 1850 the family resided in Springfield, Dane County, Wisconsin with the Abram Smith family of England (no known relationship between the two families). In 1860, the family was found in Dane, Dane County, Iowa, with an additional five children: William, Roxanne, Betsy, Sarah and Benjamin. Four more children Priscilla, George, James and Arthur were born before Eliza's death 7 June 1868. In the fall of 1869 the family moved to Arena, Iowa County, Wisconsin; the 1870 census found the family with all the children at home.
William married again in Black Earth, Dane County, Wisconsin 15 February 1872 to Mary Ann Weston, the daughter of Jabez and Mary Weston. She was born in Lincolnshire, England 13 May 1836 and immigrated in 1836 to the United States with her parents. Two children were born to William and Mary, Fred Weston and a child who died in infancy. March 12, 1888, the family moved from the farm into Mazomanie where William died 17 February 1900 and Mary died 28 July 1914.

The family surname was spelled Elsworth in England but Ellsworth in America.

Children in addition to those listed below include:
* Sam, Born 18 March 1845 in Leicester, Leicester, England, and died 17 Jun 1846 in Leicester, Leicester, England .
* Sarah Ann "Sadie", born 5 Mar 1858 in Dane County, Wisconsin, and died 14 Jun 1947 in Los Angeles County, California.
* Arthur Clarke, born 4 Jun 1867 in Dane County, Wisconsin.
* James Lincoln, born 8 Aug 1865 in Dane County, Wisconsin, and died 20 May 1944 in Council Bluffs, Pottawattamie, Iowa