Trump fired Scaramucci, a wealthy NY financier, on July 31, just days after naming him as communications director.
During his brief time at the White House, Mr Scaramucci promised to take a hard line against leakers.
He also called on Trump to distance himself from top adviser Steve Bannon, the divisive former head of Breitbart News who Scaramucci argued was pulling Trump too far to the right and was "not serving the president's interest".
"I've never sat down with Steve Bannon and said, 'Hey are you a white nationalist or a white supremacist?' But I think the toleration of it by Steve Bannon is inexcusable".
STEPHANOPOULOS: You mean Bannon and Breitbart?
He said President Trump needed to bring in "more loyalists" in order to carry out his agenda.
"The fact that [Steve Bannon] is sitting there inside the White House probably makes people think that [Breitbart] has an influence, and I think that's reason enough to probably make some changes".
He did reveal that he spoke with the President this past week.
Scaramucci insisted he was unaware the interview was being recorded and hit out at The New Yorker journalist Ryan Lizza after the interview was published, taking to Twitter to describe Lizza as the "Linda Tripp of 2017" and insisting "He absolutely taped the call without my permission", adding #lowlife. If the president wants to execute the legislative agenda that I think is so promising for the American people, the lower and middle-class people, the middle-class people then he has to move away from that sort Bannon-Bart nonsense.
"Do you believe Steve Bannon does that?"
Mr Scaramucci, who was sacked last month after just 10 days on the job, told ABC there are "elements" within the White House trying to "eject" Mr Trump, according to the BBC.
"I think the president knows what he's going to do with Steve Bannon", he said, adding, "Let's leave it up to the president, it's his decision". "He's got to be more into where the moderates are and the independents are, George, that love the president". And so if he does that, he'll have a very successful legislative agenda that he'll be able to execute. "And if he doesn't do that, you're going to see inertia and you're going to see this resistance from more of the establishment senators that he needs to curry favor with".