'X Factor' ratings slide: Where Simon Cowell went wrong Season 3 of Simon Cowell's singing show got a jump-start on the fall season Wednesday night, but the numbers were pretty close to a disaster. An average of 6.2 million viewers tuned in to the one-hour premiere, good only for an anemic second behind CBS' "Big Brother" (6.7 million) -- and that's with ABC running a repeat of "Shark Tank" (3.6 million). In the key adults ages 18 to 49 demographic, Fox's "X Factor" lost more than one-fifth of its audience compared with last year. Remember back just before the show launched its first season and Cowell predicted that he was going to beat "American Idol"? Well, that didn't happen, and it's obvious now that it's not going to anytime soon. So why is "X Factor" going down like a tone-deaf warbler at an audition? Blame it on Cowell, who seems to have no fixed vision for the show. In season 1, he hoped to rekindle his old "Idol" magic with former judge Paula Abdul, only to find that viewers had moved on beyond their warring-siblings act (he wasted no time canning Abdul, fellow judge Nicole Scherzinger and host Steve Jones once the season wrapped). Last year, the show turned into a deliberately crass, over-the-top, "Hellzapoppin"-style revue, with virtually every segment buffed and manipulated to beat a certain emotion out of viewers. This year, the producers have decided to move back to the format's amateur-hour roots. The idea is to focus on those who truly need a break -- such as Wednesday's heartstring-tugging 54-year-old mom Lillie McCloud, above -- as opposed to more jaded singers who've already been through a busted record deal or two. What is "X Factor"? Well, what would you like it to be? True, you could argue that the singing-show format has simply gotten too crowded, what with "Idol" and "The Voice" also competing for viewers. But Cowell had a clear shot competitively this week, and the show belly-flopped anyway. In Britain, "X Factor" was a much bigger hit. Cowell was able to leverage the contestants' personal lives and scandals into tabloid fodder that fed viewership. But the U.S. is a much larger, more cluttered TV market. And "X Factor" had the luxury of years to build its audience, after it drove "Pop Idol" -- the British version of "American Idol" -- out of the market. Cowell brought on new judges Paula Rubio and Kelly Rowland to the stateside version this year. If the ratings keep dropping, don't be surprised if the master of media manipulation starts talking program reinvention yet again. But that's one act that needs to be sent packing. And the repercussions will extend far beyond Cowell and his mini-empire. Fox has spent more than a decade relying on singing shows. After this week's ratings, it's clear that era is quickly winding down. What do you think of "X Factor"? Carson Daly to join 'Today' as host of 'Orange Room' Jimmy Fallon presents 'Breaking Bad' spoof, 'Joking Bad' 'Sons of Anarchy' Season 6 premiere hits series-high ratings