Son of Jonathan Ballard and Sophia (Brown) Ballard.
He married, first, Adelia A "Addie" Rice on Nov 26,1857 at Charlemont,MA.
He married, second, at Monroe,MA on Jun 4,1882, Nancy Ellen (Tower) Sherman. She had married, first, on Jun 19,1864, Jacob Sherman III of Rowe,MA.
Children(by first marriage): Ida Josephine Ballard Roberts, Adelia Emeline "Lettie" Ballard Hillman(whose first husband was Francis H Hillman), Ellen Zerviah "Nellie" Ballard Ruberg, and Julia R Ballard Strong.
Children(by second marriage): Susan May Ballard and Nathan Brown Ballard Jr.