Simon Cowell on ‘X Factor’ Judges: ‘No One Has Any Job Security Anymore, Including Myself’

"I don't envisage this changing for a while," he says of current panel

Simon Cowell doesn't see his current lineup of "X Factor" judges changing anytime soon — but says that "no one has any job security anymore, including me."

Cowell rolled into the Television Critics Association summer press tour — via satellite, anyway — with returning judge L.A. Reid and new panelists Britney Spears and Demi Lovato. He said he had always wanted Spears on the show  and joked that Lovato was "a brat."

Also read: Mariah Carey Named 'Idol' Judge; More Announcements Coming Soon

He once used the same word to describe Cheryl Cole, the judge who was quickly fired from the first season of "The X Factor." He subsequently replaced Cole's replacement, Nicole Scherzinger and Paula Abdul, bringing in Spears and Lovato. But he said he doesn't anticipate similar retooling for his current panel.

"No one has any job security anymore, including myself," Cowell said. "We've always made a point on these shows of changing the shows whenever we think it's necessary… we just felt that we needed a change."

He said of his new panel: "I don't envisage this changing for a while."

Also read: 'X Factor' Shakeup an Admission of Failure; Enter Mariah?

Cowell, in Cowellish fashion, took what opportunities he could to gin up conflict. But he took fewer potshots than he did at last year's TCA, when he predicted — wrongly — that "X Factor" would overtake "American Idol" as television's biggest show.

This time around, he took small digs at "Idol," which he left after its eighth season, and NBC's "The Voice," which outperformed "X Factor" last season.

At one point, Spears and Lovato were asked what other musical competitions they enjoy.

"Don't mention 'The Voice,'" Cowell said.

"I really enjoyed 'American Idol' when it first came out," Lovato said. Spears agreed.

"Seasons one through eight," Cowell said, referring to his stint on the series.

He also blamed Abdul and Scherzinger for 13-year-old's Rachel Crow tearful breakdown following her "X" ouster last season. (Abdul advised Scherzinger to vote to send Crow home, so that the judges' panel would deadlock. That meant viewer votes sent Crow home.)

"That was the girls' fault, unfortunately, because they could have saved her," Cowell said.

This time last year, Cowell tried to play up a spat between the male and female judges on the show. This time, he said Lovato and Spears were tougher judges than him and Reid.

"Britney's quite mean, which you'll discover," he said. "As sweet as a lemon."

Spears was asked if she thought the show would boost her record sales. "That's not my intention," she said, adding that she wanted to do the show because it would be a different experience.

Cowell also said he was happy for Mariah Carey, who was named earlier Monday as the new judge on "Idol." But he said she might have a hard time saying no to contestants because "she's sweet."

Not sweet as a lemon. Just sweet.