(courtesy of TMZ)

Michael Jackson's long-time best friend and physician -- tells TMZ he did not betray Michael Jackson by claiming to TMZ that MJ had an affair with Klein's office manager ... and he's freaked out that he's getting "a lot of horrible death threats."

Klein and office manager Jason Pfeiffer were featured on "Extra" a few days ago -- during the interview Pfeiffer claimed to have had a romantic relationship with MJ.

Klein called in during TMZ Live last Friday, and said Michael was gay and Jason was "the love of his life."

On Saturday, Klein told TMZ he does not believe he betrayed Jackson because Jackson never tried to hide his sexuality. We called Klein out on his statement, because it seems to us MJ went to extreme lengths to keep his sexuality a secret.

Klein told us Saturday, "Was he gay? Yes." He then went on to say his declaration is meant to shoot down rumors that Jackson was a pedophile. And, Klein says, as far as he knows, Jason's alleged two-month relationship is the only gay relationship Jackson ever had.

Pfeiffer says, "Michael was not ashamed of his sexuality."

Klein and Pfeiffer say they have received numerous "horrible death threats" since their statements became widely public this week. They have both been in touch with the FBI and now say they have bodyguards.