Fred Swartz was born in Gerard, Kansas. In 1890 his father brought the family to California to work a surveying project on the coast for the Santa Fe Railroad. The project fell-though, but the family settled in Fresno.
Swartz was educated in Fresno, attending the elementary grades at the old White School (situated on the site where Fresno Memorial Auditorium now stands), then Fresno High where he played football from 1901 to 1903. He studied architecture through a correspondence course while working as a logger at Shaver Lake. He worked for an architectural firm in San Francisco for a time and then enrolled in the University of Pennsylvania, graduating in 1909. Swartz worked with his father, Alexander C. Swartz until the senior's death in 1919 (flu epidemic), then worked with C. J. Ryland until 1934 when the partnership dissolved. He worked alone until 1946, when he took on William G. Hyberg (of San Mateo) as a junior partner. He finally retired in 1965.
During World War II, he was a civilian architect for the U.S. Navy. He helped design the giant SeaBees base at Camp Parks near Pleasanton and the 3,000 bed Camp Shoemaker Hospital.
He was a prominent architect who designed many public buildings in Fresno. Among the buildings his firm designed were the Fresno County Public Library, the (old) Fresno County Jail, the Elks Lodge, the Science Building at Fresno State, the Greyhound Bus Terminal, McLane High School, and several hospitals around the valley. He also helped design the Scottish Rite Temple in Fresno. Before he would design a building, he would interview everybody who worked or had business in that building to determine their needs so that the building would be most functional.
He petitioned Las Palmas Masonic Lodge No. 366 for the degrees of Masonry in 1909 at the age of 24. He then served as its thirteenth Master in 1917 at the young age of 32.
Sponsored by Ancestry