secretly he had been afraid that , after retiring from the government and writing his memoirs , he 'd be gradually marginalized , treated as an eminence grise , invited to appear on nightline once in a while , and to write the occasional thumb-sucker for the new york times op-ed page .