Mathew was born Mathias Augenstein, the son of Matthias Augenstein (1779-1870) and Barbara Leonhardt. They were both born in Germany. His father died in Baldwin Township, Allegheny County, Pennsylvania. It is unknown if his mother ever came to the United States.
Mathew married Katharine Margaretha Neu. They had the following thirteen children: John 1852-1922; Catherine 1853-1909; Charles W. 1854-1926; Mary J. 1854-1918 married John Dewalt; Margarethe Katharine 1857-1880; Elizabeth 1858-1863; Carolina 1860-1865; Sarah 1862-1916; Anna Katharine 1864-1873; Susanne Elisse (twin to William) 1866-1916 married John Lachman Jr.; William (twin to Susanne) 1866-1941; Jacob 1868-1934; Louise Elizabetha 1870-1931 married Charles Kroll.