How Chelsea Handler Was Shaped by the Deaths of Her Brother and Mother: 'That Turned Me Into an Adult at a Young Age'

Misha Gravenor

Chelsea Handler has always felt wise beyond her years.

“I was 15 when I was 2,” the former Chelsea Lately host says in the latest issue of PEOPLE.

“There was a lot of love in our house,” Handler, 40, adds of growing up in New Jersey the youngest of six born to Seymour, a used-car salesman, and Rita, a homemaker.

But her family was rocked in 1984 when Handler’s oldest sibling, Chet, fell off a cliff in Jackson Hole, Wyoming, while hiking.

Handler says her entire family grieved the loss of Chet, who was only in his early twenties, “but no one had the tools to deal with it.”

“You would walk by my parents’ room and just hear them crying,” she says. “I was only 10, so my brother and sisters were, like, ‘Don’t even worry about her. She doesn’t even know what was going on.’ But of course I knew what was going on. I was so close with my brother.”

“My parents were so lost in their grief, they were unavailable. That experience was a defining moment in my life,” Handler says. “That’s when I realized you can’t ever really depend on anybody. They could die at any moment. But even if they didn’t die, those you love the most may not be there for you when you need them.”

At the time, Handler considered this a “negative” experience. But as she entered adulthood, she recognized how it prepared her to move out to Los Angeles on her own to pursue acting.

? For more on Handler – including why she took a year-long break from Hollywood – pick up the latest issue of PEOPLE, on newsstands Friday

Losing her brother also primed Handler for stepping in as a leader in the family when her mother entered hospice after a battle with cancer in 2006.

“I was in London on my first book tour, and my sister called and said, ‘I think you need to come home. She’s dying,’ ” Handler recalls. “I came home and slept in the hospital with my mom for a week. Then they took her to hospice.”

Handler says her dad was in “denial” that his wife was truly dying and Handler’s mother tasked the comic and best-selling author with preparing him for her death.

“That was much easier for me to deal with than the death of my brother because I knew she was in pain and wanted to go. She did not want to be remembered in that way,” says Handler. “I wanted her to be able to depend on me, so I took control of the situation. I told my dad he had to deal with his feelings and I was happy to show up for my mom and my family in that way.”

Handler – whose four-episode documentary series Chelsea Does debuts at the Sundance Film Festival just hours before it becomes available on Netflix Jan. 23 – regrets that her mother never saw her reach the highest points in her career, but she says she found small ways to keep her in mind as she found greater and greater success.

“I used to have this thing I would do before I walked out on stage. I’d look up and just smile at my mom. It was a ritual that I did,” she says. “I had a psychic on [Chelsea Lately] and he said, ‘Your mother is here and she wants me to [pinch your nose]. And my mom used to do that. No one knew that. Not even my father. And I felt my mom in that moment. I haven’t felt her since.”