Born in West Virginia in 1873, John William Davis went to college and law school at Washington and Lee University taking his LL.B degree in 1895. Having already read law for a year in his father's office, Davis completed the law degree requirements in nine months. After practicing for a year in West Virginia, he accepted a position as the third member of the expanded law faculty at Washington and Lee. During the 1897 school year, Dean John Randolph Tucker died and Davis had to take on the additional load of teaching Tucker's classes. Though tempted to stay on at Washington and Lee under the leadership of the new president, William L. Wilson, Davis chose the "rough & tumble" of private practice. Two years later, when Professor Charles Graves left Washington and Lee to accept a chair at the University of Virginia, he was again invited to join the permanent faculty. Davis again selected private practice over teaching. He remained loyal to Washington and Lee and later served more that two decades on its board of trustees.

Davis practiced law in Clarksburg from 1897 to 1913. During this period he was active in West Virginia and national Democratic politics. He was elected to the West Virginia House of Delegates in 1899 and, from 1911-1913, he served in the U.S. Congress. In 1912 he married Ellen G. Bassell. (He had married Julia McDonald in 1899. She died in childbirth a year later.) From 1913-1918 he served as U. S. Solicitor general. In September 1918, Davis was one of the delegates to the Berne, Switzerland conference on the treatment and exchange of Prisoner. From 1918 until 1921 he was ambassador to Great Britain. In 1921 Davis moved from London to New York to become head of the prominent Wall Street law firm Davis, Polk and Wardwell. Clients included J. P. Morgan and Company, and U. S. Steel.

In 1922, the same year he served as president of the American Bar Association, Davis rejected appointment to the U. S. Supreme Court. In 1924 he became the Democratic nominee for president. He waged a conservative, high-minded and losing campaign against Calvin Coolidge. He left the political arena, only reemerging briefly in the 1930's as an organizer of the anti-New Deal Liberty league.

For the rest of his career, he devoted himself to his private practice. By his death in 1955 he had made 139 oral arguments before the Supreme Court, at the time a 20th century record. Davis was honored in his lifetime by fourteen honorary doctorates. Felix Frankfurter, Learned Hand, and Hugo Black, among others, deemed him one of the two or three finest advocates of the century.

Davis' lifelong fidelity to the conservative legal principles espoused by his father and by the Washington and Lee law faculty at the time he was a student make for a seemingly inconsistent record of advocacy. He may be best remembered for successfully defending the steel industry against government seizure during the Korean War, and for unsuccessfully arguing South Carolina's case for maintaining segregated schools in the school desegregation cases now known jointly as Brown v. Board of Education. But Davis' second case as Solicitor General made a strong argument against Oklahoma's "grandfather clause" excluding blacks from voting (Guinn v. United States). He spoke in defense of religious liberty in the 1928 presidential campaign when candidate Al Smith was attacked because of his Catholicism. In a 1931 pro bono case, Davis defended a Yale divinity professor in a case (United States v. Macintosh) that became a leading precedent in the development of the law of conscientious objection. During the Cold War, Davis was contemptuous of McCarthyite tactics. He was involved both in the Alger Hiss case and in preparing the appeal of J. Robert Oppenheimer to the Atomic Energy Commission for security clearance.