"Aunt Sally Dye," as she appears in the notes of my Great Aunt H.E. Mott, was the youngest child of Cornelius and Charity (Griffin) Dubois, born in 1796 the young sister of my 3rd Great Grandfather Jonathan Dubois.

Her husbands and children are a bit of an enigma. Many of the family trees and Find-a-Grave
have it that she had five children that survived, three with J., or William, Palmer, and two with James Dye. I find that hard to believe considering the documented birth years of the five children, all whom appear to have been in New York State, likely Tioga County, before Palmer died and Sally and went to Wisconsin about 1847 (per the obituary of son William G. Palmer) where she married the widower James R. Dye who died between 1860 and 1870 (censuses).

Children ATTRIBUTED to Sally Dubois (1796-1877):

born 1835: James Walter Dye
born 1836: Deborah Palmer (married ____Kings per the 1860 census living with her mother and young son. FaG memorial says "Kinney."
born abt 1838: William G. Palmer (obit says he came to Wisconsin about 1847)
born 1838: Liberty Dye (1862 war registers and enlistment register (age 25) identifies father as James____, mother Sally Palmer.
born 1841: Dewitt G. Palmer

Sally (Dubois) Palmer moved to Hebron in Jefferson County Wisconsin about 1847. James had had two wives before Sally, the first being Sarah Fox. According to the notes on James' first wife's Find-a-Grave, Sally Fox, she died in 1833 and he married "an Unk woman" who died shortly after they moved to Wisconsin. Sally Fox. I would submit that the 1840 census of Tioga County of James Dye is with the "Unk woman" and the two boys under 5 are Liberty and James W. Dye and not Aunt Sally Dye's sons but would become step-sons.

So two options seem evident as to the parentage of James Walter Dye and Liberty Dye: they were either the sons of James Dye with an earlier wife, or they were actually sons of Palmer and Sally and took the surname of their stepfather after 1847. I would think the first option is the more likely.

1840 census of Tioga County New York:
I suspect that the 1840 family of William Palmer in Tioga County contains Sally Dubois, his wife. The census record has 1 male under 5 (William G. Palmer), 1 male 15-19 (unknown or mistake), 1 male 50-59 (William Sr), 1 female under 5 (Deborah), one female 40-49 (Sally Dubois Palmer). (Dewitt was born in 1841.)

The same census of Tioga County has the rather large family of James Dye: 2 males under 5 (James W. & Liberty), 1 male 10-14, 1 male 15-19, 2 males 20-29, 1 male 40-49 (James), 2 females 10-14, 1 female 15-19, 1 female 20-29, 1 female 40-49 (the "Unk" woman?), 1 female 70-79 (likely a mother.)