Joseph was born on November 1, 1829, in Mönchzell, Baden, Germany (not to be confused with Münchzell, Bavaria, German). He was a son of Johann Gottlieb Umbach and Anna Catharina Lachsinger. When Joseph was 13 years old, his family left Germany for Canada, where other relatives had already settled. The Umbach family, consisting of the parents and four sons, sailed aboard the Brig Historian, departing Rotterdam in tthe spring of 1843 and arriving in the port of New York on July 12, after 54 days at sea. 

Having reached Canada West (now Ontario), Johann Gottlieb bought a small farm near Conestogo in Woolwich Township but traded it for a larger tract of woodland in the northern part of township. The family was isolated, with few neighbours nearby, as this area was just beginning to be inhabited by settlers. 

Joseph's father was Lutheran and his mother, Catholic. As a young man, Joseph converted to the Methodist (Evangelical) movement that was growing in the area. Believers gathered to worship in their homes, and it was there that Joseph gained skill as a respected and gifted preacher. 

In 1851, Joseph, age 22, and Elizabeth Menger, 17, were married. Like the Umbach family, the Mengers came from Baden, Germany, and were among the early pioneers of north Woolwich.

In the 15 years following their wedding, Joseph and his growing family changed residencies according to where his ministerial work took him—in Canada West (Ontario), the family lived in Woolwich Township, and the towns of Waterloo, New Hamburg, and Niagara Falls. They also resided briefly in Utica and Buffalo, New York. Around 1865, the family moved back to New Hamburg, where Joseph and Elizabeth would remain until their deaths. In 1866, Joseph was elected president of the Waterloo Society of the Evangelical Association, a post he held for 29 years until poor health forced him to resign.

Joseph endured illness and several medical operations in his senior years. After his wife's death in February 1906, Joseph lived with his daughter and son-in-law, Mary Ann and John Kaercher. 

Joseph passed away on January 11, 1911, at the age of 81 years. The attendance at his funeral was so large that Zion Evangelical Church in New Hamburg could not accommodate all the people who came to pay their respects. In the service, Rev. Umbach was eulogized for his many contributions to the church.