Frank was one of three children born to Jimmie Asbury and Nellie (Watts) Asbury. His father Jimmie passed away on March 23, 1973 and is buried in Hillcrest Cemetery in Weleetka Oklahoma. His mother Nellie is still living. He has a brother named Howard who lives in Okmulgee Oklahoma. Franks sister Patsy (Asbury) Verucchi passed away on September 3, 2001 and she is buried in Hillcrest Cemetery in Weleetka.