Wiley was born in Kentucky the son of Samuel Howard and Delana Jones. In 1869 he married his first wife Minerva Jane Patrick. Their children are; Laura Belle (Howard) Miller, McKendall Howard, Queen Victoria (Howard) Adams, Marmaduke "Duke" Howard and William P. Howard. Wiley married his second wife Parrot Allen. Their children are; Dove "Dovie" (Howard) Miller, Mary Ann (Howard) Arnett, Dona Howard and Leora Howard.