Married Mary Ann Arnold (1827-1860) on 9/19/1844. Married Rachel Paulina Riley (1841-1930) on 12/13/1860. (Rachel was Mary Arnold's niece.) Acted as the guardian for his brother William Henry Harrison Armstrong when their parents died in 1859/60.
Served as Major in the Mississippi Militia during Civil War. Was commonly referred to as "Major" Armstrong after the war.