In 1822 Smith married Ann Carroll Fitzhugh, whose father was one of the founders of the City of Rochester. She was called Nancy by family members. Their marriage lasted till their deaths, which followed in rapid succession. Gerrit Smith died on December 28, 1874. Ann Carroll Fitzhugh Smith died on March 6, 1875, which would have been Gerrit's 78th birthday.

Ann C. Smith bore eight children, a fact that has not been correctly reported by any published source. The correct number was identified by Peterboro Historian Donna Burdick, from tombstone rubbings that identified a previously unreported set of twins who "died too young to have names in this world." Only two of the Smith's children lived to adulthood, the more prominent being their daughter, Elizabeth Smith Miller. Of those who died in their youth, their son Fitzhugh was the oldest when he died in 1836 at the age of 12.

Ann Smith's relationship with her husband appears to have been loving, and their correspondence suggests her willingness to advocate on behalf of their children. A letter in the NY State Archives in Albany urges Gerrit's tolerance of their son Greene, whom she reported struggling through a serious case of withdrawal in his attempt at abstinence from cigars.

In 1822 Smith married Ann Carroll Fitzhugh, whose father was one of the founders of the City of Rochester. She was called Nancy by family members. Their marriage lasted till their deaths, which followed in rapid succession. Gerrit Smith died on December 28, 1874. Ann Carroll Fitzhugh Smith died on March 6, 1875, which would have been Gerrit's 78th birthday.

Ann C. Smith bore eight children, a fact that has not been correctly reported by any published source. The correct number was identified by Peterboro Historian Donna Burdick, from tombstone rubbings that identified a previously unreported set of twins who "died too young to have names in this world." Only two of the Smith's children lived to adulthood, the more prominent being their daughter, Elizabeth Smith Miller. Of those who died in their youth, their son Fitzhugh was the oldest when he died in 1836 at the age of 12.

Ann Smith's relationship with her husband appears to have been loving, and their correspondence suggests her willingness to advocate on behalf of their children. A letter in the NY State Archives in Albany urges Gerrit's tolerance of their son Greene, whom she reported struggling through a serious case of withdrawal in his attempt at abstinence from cigars.