Randy Jackson is the only judge returning to "American Idol" next season ... but that almost wasn't the case.

Sources close to production tell TMZ there was "serious talk" of a clean sweep of the judge's panel ... but we're told Idol brass wanted to keep a connection to the original judge's table, so Randy was spared.

As we previously reported, Simon Cowell left for "The X Factor," Ellen DeGeneres stepped down after one season and Kara DioGuardi was let go -- leaving Randy as the only remaining judge.

Welcome back, dawg!