James was born in Pretoria and, as his parents were in the Foreign Service and moved around a lot, he was sent to boarding school at Bishops in Cape Town at the age of eight.
After matriculating, he was conscripted into the SANDF Anti-Aircraft Division, was commissioned to and eventually commanded a citizen force regiment, with the rank of Lieutenant-Colonel. In the mid 1980s, he left the army, which was being used increasingly internally to suppress political upheavals with which he sympathised.
James started working in the PFP research department with iconic figures like Helen Suzman and Colin Eglin late in 1978, at the same time as the Information Scandal. In 1981, he contested his first parliamentary seat, aged 25. In 1988, he became the PFP's Director of Communication, and was responsible for the branding of the Democratic Party, of which the PFP became part in April 1989.
In 1992, he became the Executive Director of the Democratic Party, and was responsible for the challenging task of preparing the Party for the first democratic elections. Selfe was one of only three senators elected in the party in that election and describes the experience of the early years of South Africa's new democracy, where he participated in drafting the country's new constitution, as tremendously exciting.
He worked very closely with the then Leader of the DP, Tony Leon, and in 1999 was elected to parliament where he was appointed spokesperson on Provincial and Local Government. In 2000 he was elected as the Chairperson of the DP's Federal Council and retained the position with the formation of the DA, a post he has held to this day.
He was re-elected to the National Assembly in 2004 and took over the Correctional Services portfolio. He says that the portfolio is not popular but occupies a crucial role in the criminal justice system and, in 2005, he pioneered a new DA policy entitled "Prisons Work where Prisoners Work". On his re-election to Parliament in 2009, he was appointed as the Shadow Minister of Correctional Services.
He attributes his drive and success to the support of his family, including his three daughters, and the professional assistance of his staff team.