Harley was the first child of George and Mattie Conrad. He had six younger siblings. His mother's sudden death in 1913 left him and the older children to care for the younger ones. Though he was a life-long farmer he was no stranger to the kitchen. He was especially good at candy making.
He married Ethel Peterson in 1917. They raised three daughters; Doris, Donnis, and Betty.
He lost his wife in 1945 and was lonely for several years until he met and married his second wife, Isabelle, in 1950.
He died as the result of a farming accident in 1961.