Carroll Pitallo Garner was the son of James William Garner and Fannie N. Pattillo. He had seven siblings: Helen H. Garner, Sarah N. Garner, Mamie Garner, Jane O. Garner, Arthur Glenville Garner, Irene D. Garner and Tressie Garner.
Carroll Pitallo Garner married Avis N. Wagstaff. They had two children: Alvin William "Billy" Garner and Hilmer Grey Garner, Sr.
Carroll Pitallo Garner and Avis N. Wagstaff Garner have four grandchildren and four great-grandchildren.