Virginia was the youngest of six children born to Tom Patton and Ludie Wells. Her siblings were Jake Patton born in 1906; Mary Jane Patton born in 1908; Jessie Bell Patton born in 1910; Georgia Leah Patton born in 1913; and Eddie Lee Patton born in 1919. Her grandparents were Jacob Patton and Nellie Sarah Turner, and Mose Wylie Wells and Mary Mason. Virginia married Theron (Tig) Weaver; then she married Leon Lusk in 1954. At the time of her death in 2008 she was a widow of Theron Weaver.