James Wardlaw worked as a surveyor, a school-master, and a merchant in his early years before he became Deputy of the County Court for the Abbeville District in 1796. From 1800 to about 1840 he was the Clerk of Court for Abbeville County.

James Wardlaw and Hannah Clarke were married in Abbeville on June 9, 1796 at the plantation home of the bride's stepfather John Quay. Their twelve children are linked below.

About 1798 they bought the Quay house and most of their children were born there. Originally a four room log building operated by Hannah's stepfather as a tavern, a stage stop, and a trading post, the Wardlaws added several rooms and made improvements over the years to make it a family home. It is the oldest house still standing in Abbeville (2017).

After buying the Quay house, James continued Quay's trading business of bartering goods in Charleston. His own goods for barter were deer skins, tallow and beeswax which he put in the store building to await a wagoner who hauled his goods to Charleston. James always rode a horse, making the trip each way in about five days, or forty miles a day. On other occasions, James sold goods at the store. James was particularly scrupulous in keeping his books and papers at both his store and as District Clerk. His prodigious knowledge of the statute law of the state was widely recognized.

James Wardlaw was a member of the Presbyterian faith all his life. He attended the Upper Long Cane Presbyterian Church and was Secretary of the Upper Long Cane Society for forty-nine years, from his election at its organization in 1798 until his death in 1842. He served as a Trustee of that church from 1813-1819 and was a member of the first Board of Trustees of the projected Theological Seminary in 1825.

James Wardlaw wrote his a detailed six page will on 6 June 1840, at Abbeville, South Carolina. At the time of his death he was a wealthy man, owning many slaves, almost 2000 acres of land in South Carolina, additional land in Alabama, and bank stocks. His estate sale, exclusive of the land, slaves, stocks, and other items specifically devised, amounted to over $34,000.
Mentioned by name in his will were sons, David Lewis, Robert Henry, Joseph James, and William Alfred Wardlaw. His daughters named were Mary Caroline and Elizabeth Amanda, and Hannah Margaret. His other two children, Francis Hugh and Jane Eliza, were included by a clause in which the remainder of his lands and his slaves were devised equally among his nine living children. Also devised was a bequest in trust to Mary Ann Hodges, wife of Benjamin Hodges. In a codicil added on September 26, 1841, he bequeathed to his granddaughter, Hannah Bonham, a negro girl between eight and fourteen years of age.

He appointed his sons, David Lewis and Robert Henry executors of his will.

Joseph Wardlaw, a grandson, wrote that James Wardlaw "was said to have been a large, stout, fine-looking man, a devout Christian, noted for honesty and reliability; of a fine mind, and greatly respected for uprightness and right living".