It was a different era.  Much, much different.  Much of their fame was due, I believe, to that new medium, TV.
Look at the old footage today (Ed Sullivan, late 40s, Colgate Hour).  Bizarre.  They weren't alone in this, either: Ed Wynn, Milton Beryl, Red Skelton (who soldiered on into the 1960s.  My Grandmother liked him...). Hold-over and hold-outs from the vaudeville era.

I consider Lewis something of a tragic figure.  Not Martin.  It's no coincidence that his first #1 song--Memories are Made of This--was recorded the year he broke up with Lewis.  Lewis: stuck in Grade B comedy flicks.  Martin: super-star status into the late 60s. Martin's 2nd #1 hit: Everybody Loves Somebody came almost a decade after his first, and led to a career resurgence on Sinatra's Reprise label, proving there was room for something other than "The British Invasion."