On July 16, 1861, Oliver mustered in as a Private in Company C of the 3rd Vermont Infantry. He mustered out August 29, 1865.

In "Soldiers' Record of the Town of St. Johnsbury, Vermont, in the War of the Rebellion" ( C.M. Stone & Company, 1883 - Saint Johnsbury, Vermont", the authors state that somewhere between Fairfax Court House and Alexandria, Virginia, Oliver fell out of ranks, unable to march any further. Regimental records list him as deserting August 31, 1862. Oliver later said that he didn't desert; he just couldn't endure the long marches, and that he went to Washington DC to find another way to serve. There, he enlisted with the 16th Eastern Virginia Regiment to assist with the defense of Washington. It is believed that the 3rd Vermont knew where he was during that time, but allowed him to remain with his new Company. Oliver participated in the battles at Savage Station and White Oak Swamp, among others.

The 16th Virginia was disbanded in 1863, and Oliver then enlisted in the navy at Philadelphia. Oliver's naval service included being taken prisoner and escaping twice, traveling to England, France, Africa, China, Brazil, and the West Indies. His service ended when he was discharged on October 1865.

Oliver married Sarah Wilson on on May 17, 1866 in Irasburg, Vermont.

On July 16, 1861, Oliver mustered in as a Private in Company C of the 3rd Vermont Infantry. He mustered out August 29, 1865.

In "Soldiers' Record of the Town of St. Johnsbury, Vermont, in the War of the Rebellion" ( C.M. Stone & Company, 1883 - Saint Johnsbury, Vermont", the authors state that somewhere between Fairfax Court House and Alexandria, Virginia, Oliver fell out of ranks, unable to march any further. Regimental records list him as deserting August 31, 1862. Oliver later said that he didn't desert; he just couldn't endure the long marches, and that he went to Washington DC to find another way to serve. There, he enlisted with the 16th Eastern Virginia Regiment to assist with the defense of Washington. It is believed that the 3rd Vermont knew where he was during that time, but allowed him to remain with his new Company. Oliver participated in the battles at Savage Station and White Oak Swamp, among others.

The 16th Virginia was disbanded in 1863, and Oliver then enlisted in the navy at Philadelphia. Oliver's naval service included being taken prisoner and escaping twice, traveling to England, France, Africa, China, Brazil, and the West Indies. His service ended when he was discharged on October 1865.

Oliver married Sarah Wilson on on May 17, 1866 in Irasburg, Vermont.