Jill Duggar Dillard Opens Up About Distancing from Her Family: 'I Never Expected This to Happen'

For the last few years, Jill (Duggar) Dillard's relationship with her family has changed significantly, and now the reality star is finally opening up about why.

After being a prominent fixture on the family's first TLC series, 19 Kids and Counting, which documented the highly-religious, ultra-conservative Duggar household from 2007 until its cancellation in 2015, Jill went on to star in the subsequent spinoff, Counting On. But in 2017, she made the surprising decision to quietly quit the show. The reason? Jill, 29, tells PEOPLE she and her husband Derick Dillard, 31, realized they'd lost the power to dictate their own lives.

"Our control to choose what jobs we were allowed to accept and even where we were allowed to live was taken away from us," Jill says, adding that goals they'd had as a couple were squashed if they conflicted with the network or family's plans.

Derick, who is now in his final year of law school, elaborates, "The first few years of our marriage, we spent time and money working towards opportunities only to hit a dead end when we'd be told, 'Well, you're not allowed to do that.'"

When the couple told the network and Jill's family about their decision, "It didn't go over very well with anyone," she says. "By that point we'd had enough. We knew we had to pull out completely to reevaluate and get our bearings."

When reached by PEOPLE, TLC declined to comment, and Jill's parents Jim Bob and Michelle Duggar provided the following statement: "Every family has differences of opinion and perspective at times, but families work things out. We all love Jill, Derick, and their boys very much. It is our prayer that our relationship is healed and fully restored quickly!"

• For more on Jill Duggar, pick up this week's issue of PEOPLE, on newsstands Friday

Since their departure from the show, Jill — who shares sons Israel David, 5, and Samuel Scott, 3, with Derick — has begun to embrace a life without some of the many rules she was raised to strictly adhere to. The Duggars discourage not only the use of birth control, but also forbid nearly all physical contact with the opposite sex before marriage, "revealing" clothes (meaning anything exposing shoulders or knees) as well as dancing, alcohol and non-religious music.

She's gotten a nose ring, enjoyed the occasional cocktail or glass of wine, worn jeans (a strict no-no for Duggar women, who don long dresses or skirts only) and even modeled modest swimwear.

"Sometimes," she says, "it's a good thing being okay with other people not being okay."

The couple has also begun sharing more of their lives on their Dillard Family blog and on their Dillard Family official YouTube page, where they say, "We can be the ones to tell our story."

Still, her hope is to one day reconcile with her loved ones.

"I never expected this to happen or for it to get to this point," Jill says. "But I'm realizing I can't put a timeline on healing. I love my family and they love me. I really just have to follow God's lead and take it one day at a time."