Jill Duggar Dillard Says Her Reality Shows Nearly Tore Her Marriage Apart: 'Another Wake-Up Call' (Exclusive)

Jill Duggar Dillard is opening up about the toll that filming her family's reality shows, 19 Kids and Counting and its spin-off Counting On, took on her relationship to her husband Derick Dillard.

"It caused a lot of frustration in our marriage," Jill, 32, exclusively tells PEOPLE in this week's issue. "Especially early on, where he would feel a certain way about filming something. I'd be like, 'I hear you, I feel you, I also don't want to do whatever it is they're asking us to do either. But we have to.'"

In their new book, Counting the Cost, the couple reveal all sorts of bombshells about their time starring on the popular TLC shows about the big, ultra-conservative Christian Duggar family. One of the main takeaways is that after they were married in 2014, they no longer really wanted to be a part of it at all. Instead they wanted to start their own life, have their own jobs and family, and not feel obligated to be available for 20 hours a week of camera time.

"It began to feel like a burden," Duggar writes in the book, which hits bookshelves on Sept. 12.

She tells PEOPLE it doesn't help that she was raised to believe that you should always honor your parents and do as they say, as part of the teachings of the Institute in Basic Life Principles (IBLP), the religious organization that her family followed. As a result, she found it hard to say no to her father Jim Bob Duggar and the show's demanding filming requests, even if she felt like just walking away.

"It definitely got between us," she says of how she and Derick began fighting over the show. "No matter your age, you are to obey your parents wishes and you even have to ask them for their blessing for any major moment in your life. That could be buying a house, moving to a different state, where to go to school. We were dealing with this a lot when we were trying to make decisions for our family , and we were really wrestling back and forth with it."

Jill claims her father began trying to drive a wedge between them, and Derick agrees.

"Whenever we were at odds with what her dad thought we should be doing with filming, he would say things that would be very damaging," Derick alleges. "He would weaponize the relationship and say, 'Is this you Jill, or is this you, Derek? Are you leading your wife astray and doing things that are not supportive of marriage?' And I think that was a red flag."

In a statement to PEOPLE, Jim Bob and wife Michelle Duggar say: "We love all of our children very much. As with any family, few things are more painful than conflicts or problems among those you love... We do not believe the best way to resolve conflicts, facilitate forgiveness and reconciliation, or to communicate through difficulties is through the media or in a public forum so we will not comment."

Jill says relations got so strained that she felt she had to pick sides.

"I saw how deeply we were in an argument one time, and I was like, 'Whoa, this is not okay,'" she recalls of her and Derick arguing over the show, and the subsequent arguments they had with Jim Bob about eventually being paid for their years of work on it.

"When I saw how it was affecting our marriage, I think that was another wake-up call for me," she adds. "It was like, okay, we need to either fight this battle together, or it's going to rip us apart. So yeah, we had to join forces at that point."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

The couple, who now live on the border of Arkansas and Oklahoma and are raising three sons of their own, have found their happy place, away from cameras and out the spotlight. They admit they're nervous about the reaction to the book, but felt like it was time to own their story.

"I know there will be nay-sayers, but I feel called to do this," Jill says of writing the book. "We really wanted to tell our story for my siblings, because some of them are going to face similar challenges, if they haven't already, to what I've faced."

And while Jill and Derick left the IBLP in 2020, are now doing things their way and "trusting in God," they both say it took a bit of therapy to get to such a good place.

"Therapy was the gift we didn't know we needed," Jill says. "We initially went into it with the goal of re-establishing a relationship with my parents, but once we go there, the therapist was like, 'I think we maybe need to do a little more processing, a little more sorting out here. You guys need to figure out who you are.' Which was so wise. It helped us so much."

For more on Jill Duggar Dillard and Derick Dillard, pick up the latest issue of PEOPLE, on newsstands Friday, or subscribe here. 

Jill and Derick's memoir, Counting the Cost, will be available wherever books are sold on Tuesday, Sept. 12.