Anson was the son of Flint Barton of Sutton, Worchester County, Massachusetts and Lydia Crosby Barton. Flint and Lydia were married on 10 June 1778 in Sidney, Kennebec County, Maine. Lydia and Flint had twelve children. They were: Edmond Barton was born on 08 April 1779 in Sidney, Kennebec County, Maine; Amos Barton was born on 12 November 1780 in Sidney, Kennebec County, Maine; Jonah Barton was born on 13 July 1782 in Sidney, Kennebec County, Maine; Otis Barton was born on 24 May 1784 in Sidney, Kennebec County, Maine; Rufus Barton was born on 28 August 1786 in Sidney, Kennebec County, Maine; Persia Barton was born on 03 July 1788 in Sidney, Kennebec County, Maine; Crosby Barton was born on 20 February 1791 in Sidney, Kennebec County, Maine; Stephen Flint Barton was born on 27 January 1793 in Sidney, Kennebec County, Maine; Alfred Barton was born on 18 March 1795 in Sidney, Kennebec County, Maine; Franklin Barton was born on 12 May 1797 in Sutton, Worcester County, Massachusetts; Anson Dean Barton was born on 04 December 1799 in Sidney, Kennebec County, Maine; and Dean Washington Barton was born on 21 December 1802 in Sidney, Kennebec County, Maine.

Anson married Rhoda Sisson on 05 January 1826 in Douglas Parish, York County, New Brunswick, Canada. Rhoda and Anson had twelve children. They were: Albion Barton was born on 03 October 1826 in Douglas Parish, York County, New Brunswick, Canada; George Sisson Barton was born in August 1828 in Douglas Parish, York County, New Brunswick, Canada; Greenleaf Barton was born on 02 May 1831 in Sidney, Kennebec County, Maine; Herrick S. Barton was born on 14 March 1833 in Hallowell, Kennebec County, Maine; Jonathan Quincy Barton was born on 20 March 1835 in Sidney, Kennebec County, Maine; Anson Bliss Barton was born on 27 November 1836 in Sidney, Kennebec County, Maine; Mary Hellen Barton was born on 26 October 1838 in Sidney, Kennebec County, Maine; William Henry Harrison Barton was born on 13 March 1840 in Sidney, Kennebec County, Maine; Charles Carroll Barton was born on 21 March 1842 in Sidney, Kennebec County, Maine; Ellen Victoria Barton was born on 08 May 1844 in Sidney, Kennebec County, Maine; Mary Ellen Barton was born on 07 October 1845 in Sidney, Kennebec County, Maine; and Caroline Augusta Barton was born on 01 March 1848 in Sidney, Kennebec County, Maine.