Diane Keaton Opens Up About Her Mentally Ill Brother: He Was 'on the Other Side of Normal'

Diane Keaton, 74, has forged a decades-long career for herself as an award-winning actress. Her younger brother, John Randolph Hall, has lived a life that couldn’t be more different: “on the other side of normal,” as Keaton puts it, since childhood, Hall received a variety of diagnoses over the years (bipolar disorder, schizoid personality disorder) but nothing definitive. Now 71, he is suffering from dementia and lives in a care facility, where his sister visits him every Sunday.

In an exclusive interview with PEOPLE, Keaton shares memories of her brother and their relationship, the subject of her new memoir, Brother & Sister. The book reveals that Hall was the inspiration for Annie Hall’s scary brother Duane in the Woody Allen classic, and attempts to understand who Hall was before dementia took over: a talented poet and artist, a reclusive alcoholic and a man plagued by never-acted-on violent fantasies about women.

“Over the years people did a lot of measuring of Randy’s mental status, and it all came to naught,” says Keaton, explaining why she decided to write her moving memoir. “He was so hidden. I wanted to explore the mystery of him.”

Hall was born two years after Keaton. The star explains that she was initially close with her brother, though she thought his fears were odd.

“I wondered why he was always crying,” Keaton says. “Why was he afraid of the outdoors? That’s weird!”

Jack and Dorothy Hall were concerned about their son’s behavior but didn’t seek professional help. It was a time when psychological issues were less talked about, and Keaton thinks her civil engineer father found the idea of therapy “kind of threatening.” But when the Vietnam draft began, they arranged for a psychiatrist to see him and he received a deferment.

Meanwhile, from her late teens on, Keaton was busy developing her acting career and came home infrequently. She has regrets about not being a more attentive sister, and about her growing fame’s effect on Hall.

“I think it’s hard for anyone who has a sibling who’s out there, throwing themselves into the world and getting recognition,” says Keaton. “I wouldn’t be happy to have a sister like that. Nobody wants to be compared.”

Hall was married for several years. But after his wife left him, he lived alone in a house purchased by his parents. He drank heavily—and told his sister about his dark fantasy life.

“I became addicted to watching horror movies, hoping the films would have some gruesome murder of a woman….” Hall wrote in a letter to Keaton, which she includes in her book.

Keaton was deeply upset, but wasn’t worried he’d act on his fantasies.

RELATED VIDEO: Diane Keaton and Brendan Gleeson Star in New Romantic Comedy Hampstead

“There was no indication he would, in anything he’d ever done. He didn’t have that bone in his body,” she says. “He wrote about them and did collages instead.”

By the time he was 65, Hall was showing signs of dementia and had to be institutionalized. But, surprisingly, it wasn’t a purely dark time for him. Before his dementia worsened, he continued to make art.

When his sister visited on Sundays, “We’d get vanilla cones and just walk around discovering things,” she says. “The walls he’d kept people away with were down. It was like he’d been set free—I got to know him in a whole new way.”

For the full Diane Keaton interview and an excerpt from her new memoir, pick up this week’s issue of PEOPLE, on stands Friday.

Brother & Sister will hit bookshelves on Feb. 4.