Peter Sarsgaard Didn't Know Maggie Gyllenhaal Was an Actress When He First Fell for Her (Exclusive)

If you thought Peter Sarsgaard and Maggie Gyllenhaal met on the set of an indie film — or perhaps at a convention for people with double A's in their last names — you'd be wrong.

"I met Maggie at a dinner, and I didn't even know she was an actress," Sarsgaard, 52, admits to PEOPLE of the fateful evening in 2001. "I knew nothing about her, but it was instant," he adds of his attraction.

He says, "Then Secretary came out maybe six months later, and I was like, 'Oh whoa. Okay. She's not like any other actress in the world.' "

Sarsgaard — who recently won the best actor award at the Venice International Film Festival for Memory, in which he plays a man with early onset dementia — says a few years into his relationship with Maggie, he was offered a role in the movie Jarhead with his future brother-in-law Jake Gyllenhaal, 42.

"I think the reason I did it was because I wanted to get to know him better," Sarsgaard says of Jake.

After he wed Maggie, 46, in 2009 (they share daughters Ramona, 17, and Gloria, 12), his former costar became his real-life brother. "And then we got to know each other a little too well!" he says with a laugh.

Over the course of their two-decade relationship, the married duo have also worked together several times, including on Broadway, where Maggie directed Sarsgaard in The Seagull, and in 2020's The Lost Daughter. (For that, she also won an award at the Venice Film Festival, for best director.)

"I love working with her," Sarsgaard says. "She's a great director and I think that stems from being just a great host in general and knowing what people need. Some people need to be told where to go, other people need to be left alone, and she's amazing at sensing that."

He says he also loved working with Jessica Chastain, his costar in Memory. "We didn't hang out afterward, we barely spoke once we were done filming," he says, when asked if they became best buds.

"We were just too busy, and we're both people with multiple children and husbands and wives," he says, noting that he just wanted to get back to his Brooklyn home once they wrapped for the day.

He adds that the project filmed in five weeks, in part to Chastain being an excellent actress: "We barely rehearsed, we'd just shoot the scenes," he says. "She has an enormous amount of film experience and she was an ideal partner. It wouldn't have worked with many other people, I don't think."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Sarsgaard says he's proud of their movie, and even more proud that he and Maggie both won awards for their work at Venice. "What's amazing is they were pulled from semi-obscurity," he says of Memory and The Lost Daughter.

"To me, awards are for highlighting things that otherwise would be hidden, and more important for smaller films. Also, who doesn't like winning something?" he says with a laugh. "It's like your birthday that day! The only thing is you have to learn how to receive something, which isn't most people's thing. It's not mine!"

For more about Peter Sarsgaard, pick up this week's issue of PEOPLE, on newsstands Friday.

Memory is in theaters Dec. 22.