On November 6th 1863 the first cemetery on the site was registered as the Elmira Union Burying Ground when a half acre of land was separated from the Northwest corner of the farm of George and Susanna Girling. It was a non-denominational cemetery, commonly known as the English Cemetery, serving the Elmira Area.

The Saint James Evangelical Lutheran congregation was organized in 1850. The back yard of the church then served as the cemetery for the church members. On September 10th 1885 the congregation purchased a half acre East of the Elmira Union Burying Ground from Christof and Anna Maria Weidenhammer. After the purchase, the remains and grave markers in the cemetery at the back of the church were moved to the new cemetery by members of the congregation.

Saint James Cemetery was expanded on January 15th 1910 when an additional acre of land was purchased to the South of the two burial grounds from Charles Stroh and his wife Elizabeth Stroh nee Weidenhammer.

On January 2nd 1934 an agreement was signed between the Elmira Union Burying Ground and the Saint James Lutheran Congregation to share the infrastructure of the cemeteries -- roads, buildings, tools, ...

Following discussions in 1951, on February 7th 1952 an agreement was struck between the Elmira Union Burying Grounds and the Saint James Church Council. The church acquired the smaller Union Burying Grounds and the two cemeteries amalgamated as the Saint James Evangelical Lutheran Cemetery. There was now a single cemetery on the site.

Then on May 30th 1960 an additional parcel of two acres of land to the South was purchased from Earl H. and Eileen Stroh to expand Saint James Evangelical Lutheran Cemetery to the boundaries we know today.

Note: Until 1952 when the Elmira Union Burying Ground ceased to exist, a burial reference to the Union Cemetery is likely to refer to this cemetery in the north west corner of St James Evangelical Lutheran Cemetery rather than the Elmira Cemetery (now the Elmira Union Cemetery) across the road. There is another cemetery named Saint James Lutheran Cemetery in nearby Saint Jacobs.