Lizzie was born either May or June 5, 1840 in then Clinton Co (now DeKalb) in MO to James Gregg and Jane Stewart. Her parents died of yellow fever when Lizzie was four. Her eldest sister, Harriett raised her. She married in 1859 to John A. Smith. Lizzie had 10 children before dying in Los Angeles at the age of 65.