Patrick Warburton Says Ellen DeGeneres Confronted Him After He Turned Down Guest Role on Her Sitcom

"You don't expect the star of the show to walk up to you and call you out on your s--- and that's all it was," Warburton said

Patrick Warburton is looking back at an uncomfortable confrontation with Ellen DeGeneres after he returned down a guest role on her sitcom.

On the latest episode SiriusXM's The Spotlight with Jessica Shaw, the Seinfeld alum, 59, said that he was ready to “move on” after making a few guest appearances on Ellen, which ran for five seasons from 1994 to 1998.

“They say you're the flavor of the month, so they keep asking you like, ‘They want you on this show, they want you on this show,’ so I did a guest spot or two on Ellen,” he explained. “And then they asked me to come back and do another one and I was just kind of done doing guest spots at that time and it was sort of time to move on and I just said I had a conflict, all right? So, that was it.”

Related: Ellen DeGeneres Says She's 'Done' After Her Upcoming Netflix Special

Warburton claimed that DeGeneres, 66, was not happy with his decision and confronted him while he was dining with a friend.

“Two weeks later, I'm having lunch at a restaurant called Orso in Beverly Hills with a buddy of mine having lunch and Ellen walks up to the table and she goes, ‘Too big to do my show now, huh?’ and I go, ‘No, Ellen.’ I go, ‘I just had a conflict,’ which she probably sensed was just bulls--- and she walked away.”

“So it was just interesting, but just sort of knowing her, I realized at that point on, there was a reason why I was never ever gonna be invited on her talk show,” he continued, referring to the Ellen DeGeneres Show.

He added: “She was spurned.”

A rep for DeGeneres did not immediately respond to PEOPLE's request for comment.

Related: Ellen DeGeneres Announces Dates for Her 'Final' Stand-Up Tour Across North America

Despite the interaction, Warburton shared he is still “a huge fan” of the comedian.

“What I recall working on that show was that as far as crowd work and talking, she was just this remarkable comedian, you know, at the very, very top of the game,” he said. “She was as good as anybody, but it was just weird.”

“It was just weird because when you say you have a conflict and you don't do something, you seem like you're gonna be fine with that,” he added. “You don't expect the star of the show to walk up to you and call you out on your s— and that's all it was, and then I always knew, it was like, ‘Oh, Ellen's got a great show. It's a lot of fun. I'll never be on it.’”

In fact, Warburton never appeared on the Ellen DeGeneres Show, which aired for 19 seasons before coming to an end in 2022 amid allegations of a hostile workplace.

In a July 2020 BuzzFeed News report, anonymous former and current staffers of DeGeneres' accused her of creating an unhealthy and toxic work environment on her show. Claims included being penalized for taking medical leave, instances of racial microaggressions and fear of retribution for raising complaints.

"I really didn't understand it; I still don't understand it," DeGeneres told anchor Savannah Guthrie in May 2021. "It was too orchestrated; it was too coordinated."

During an L.A. standup set in April, DeGeneres joked about getting "kicked out of show business" for being "mean." She later told the audience during a Q&A, "I'm making jokes about what happened to me, but it was devastating. It took a long time for me to want to do anything again."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer??, from celebrity news to compelling human interest stories.

Most recently, DeGeneres canceled four upcoming show dates just one month after kicking off the stand-up tour. No details were provided for the cancellation besides an alert from Ticketmaster.

For more People news, make sure to sign up for our newsletter!

Read the original article on People.