Son of Volney Jackson Brown and Susan (Butler) Brown. His parents are buried in Greenlawn Cemetery in Brownsburg. Susan's parents are William Butler and Nancy Henshaw who died in Franklin County, Indiana. Isaac's brother, William M. Brown, was with him when he died. William was discharged from the military for illness. It is believed that William brought Isaac home from Maryland where he died.