Southern Charm 's Craig Conover Says Reconnecting with Naomie Olindo Showed Him 'I'm Not into This Anymore'

With Kathryn Dennis's Great Katsby 30th birthday party ending on a not-so-great note, the attendees needed to recover in Thursday's episode of Southern Charm.

Austen Kroll sweated it out at the gym, Craig Conover and Paige DeSorbo chugged water and Kathryn downed coffee. Shep Rose and his girlfriend Taylor Ann Green recapped the evening the next morning when Taylor asked what Austen and Craig spoke to him about at the party.

"I thought Craig was going to like, put his hat in his hand, be like, 'You know what? Maybe I have been a little bit less than desirable as far as friendship is concerned,'" Shep told Taylor. "But we get out there and he was just like deflecting, deflecting, deflecting. And then Craig admitted that he slept with Naomie Olindo in Las Vegas."

Shep added that Kathryn "implied that it wasn't just Vegas — it's been kind of a low-key, ongoing thing."

RELATED: Naomie Olindo on Hooking Up with Ex Craig Conover: 'Neither of Us Thought We'd Get Back Together'

But Paige felt comfortable with boyfriend Craig and Naomie's past, and said she actually spoke to his ex a lot at Kathryn's party.

"I mean, it's obviously not the most comfortable thing, like over the summer you guys slept together. But I'm not in high school. I'm not going to not acknowledge her," Paige said to Craig. "Everyone has an ex. Everyone has dated people before. I'd actually find it more of a red flag if you two didn't get along."

PGA Tour

KATHRYN AND CHLEB'S FALLOUT

While Craig and Paige's relationship appeared stable after encountering Naomie at the party, Kathryn and boyfriend Chleb Ravenell's experienced a strain. Kathryn told Chleb she didn't like that he told her Venita Aspen wanted to talk to her because "it made it made me feel like suddenly you're on her team and not my team."

"I feel like you don't like a lot of people and that's not the way I roll," Chleb told Kathryn. "Like I don't have any problems with any of my friends or nothing. When it comes down to you and your friends, it feels like you have problems with most of them."

Chleb continued to tell Kathryn that he didn't like dealing with the stress of her social situations.

Kathryn Dennis/Instagram Kathryn Dennis and Chleb Ravenell

"Deal with what stress? I don't ever bitch about people," Kathryn said.

But Chleb disagreed. "You always bitch!" he responded.

That put Kathryn over the edge. "Walk the f--- away from me," she shouted at Chleb.

While recalling the conversation to his mom over the phone, Chleb said he felt like "all hell broke loose" when he moved in with Kathryn.

Chleb's mom actually knew Kathryn before he did, so she said she'd try talking to the mom of two.

RELATED VIDEO: Southern Charm's Naomie Olindo Had to 'Beg' to Get Back on the Show After Split from Ex Metul Shah

WHAT HAPPENS IN VEGAS, FOLLOWS TO CHARLESTON

Having just found out at Kathryn's birthday party that Craig and Naomie did, in fact, hook up in Vegas left Austen shocked.

"I literally had the covers pulled over my eyes," Austen told Leva Bonaparte on the phone.

Leva admitted she actually had lunch with Craig and Naomie in Vegas right after they slept together, so she knew about the hookup for months. "I'm calling bulls--- on one and done," Austen said of whether Craig and Naomie hooked up again.

Leva agreed that Craig and Naomie have "way more feelings than those two are admitting to."

Craig did indeed feel a little bit emotional pulling up to Naomie's parents' house for her "fresh start party." At the party (also attended by Summer House alum and Charleston resident Everett Weston), Craig and Naomie fought about chewing ice.

"You know, it's like a really big turn off for some people, apparently," Naomie said to Craig.

Naomie also told vaguely Craig, "You were right. That's all I'm going to say." To which Craig responded, "Well, cheers!"

Craig picked up on a vibe that Naomie had lingering feelings for him. "I think that Naomie still has feelings for me that she had never dealt with," he said in an confessional interview. "She kind of shelved them and I think that when she broke up with Metul and we hooked up again in Vegas, I think those feelings were knocked off the shelf."

Want to get the biggest stories from PEOPLE every weekday? Subscribe to our new podcast, PEOPLE Every Day, to get the essential celebrity, entertainment and human interest news stories Monday through Friday.

Craig Conover Instagram

Madison LeCroy thought Craig loved Naomie, too. "I will always love all my exes," Craig said.

When Madison accused his and Naomie's relationship of being "toxic," Craig admitted he and his ex needed to address certain topics.

"We've got some s--- that we need to talk about because it's festering, but we see each other and we have fun and we act like nothing's happening," Craig said.

"That's your specialty," Austen quipped.

Craig didn't want Austen butting in. "If you don't have things that you want to say sorry to me about, then that's sad," Craig told Austen.

Inside, Naomie recounted the drama that happened at Kathryn's party to Leva, Venita, Patricia Altschul and Olivia Flowers. When Olivia asked Naomie why she came to the party if she and Kathryn had preexisting tensions, Naomie and the rest of the women interpreted that as Olivia taking Kathryn's side.

"I couldn't make sense of the whole thing," Olivia explained.

After the women piled on Olivia, she went back to Austen and informed him about the situation. Venita quickly came over and apologized. "I'm going to apologize because I'm a big girl," Venita told Olivia. "I don't want you to feel attacked. I'm sorry."

Olivia accepted the apology and thanked Venita before the topic turned back to Craig and Naomie's reconnection. This time, Austen spoke with Naomie about the subject.

"I'm convinced you two had like a mini-reconnection," he said, "whether it be two weeks, three weeks, a month. That's his style."

Bravo

Naomie insisted she and Craig only slept together twice, but Austen only knew about the one time. "He was like, 'One time. Never again,'" Austen claimed. "The way that he described what had gone on was like, 'I was right. And she came calling and we hung out and yeah, we slept together.'"

A flashback to Kathryn's party then showed Craig talking to Austen and Shep about the hookup, in which the Pillow Talk: What's Wrong with My Sewing? author referred to it as "closure." "I woke up and I was like, I'm done. I was like, I'm not into this anymore," Craig said.

That description hurt Naomie. "I literally don't believe that he would describe me in that way," she told Austen.

RELATED: Southern Charm's Naomie Olindo Has a 'No-Contact Rule' with Ex Metul Shah Since They Split

Then Craig walked in, so she brought the accusations to him in a one-on-one conversation. "Those words would never come out of my mouth," Craig alleged.

In a confessional, Naomie said she sought out Craig as a familiar face in the wake of her breakup. "When Metul and I broke up, I was so heartbroken and I think in reaching out to Craig I was seeking comfort in somebody that I trust, in somebody that I knew understood me," she said. "But it was just kind of a fleeing thing."

Still, she didn't appreciate how Austen characterized their her and Craig's reconnection. "What hurt me so much is you insinuated that I was your ex for a reason because it just like wasn't fun and whatever," Naomie told Craig.

Craig assured her they had an "amazing experience."

"I don't have any regrets about it," he continued. "I love you to death and you know that. You know I'll always feel that way about you. I wish you would've gotten to know me, like who I am today. You would've liked the guy."

Never miss a story — sign up for PEOPLE's free weekly newsletter to get the biggest news of the week delivered to your inbox every Friday.

Southern Charm airs Thursdays at 9 p.m. ET on Bravo.