Faber was the first child of Claude Sherman and Alice Pearl (Moser) Fitch. His siblings were Florus Edwin (known to many as Shorty), and Donald Dean. Faber was first married to May Banks, who died as young woman. He next married Venice L. Taylor, by whom he had six sons and one daughter. Faber is buried next to May.