Casey Affleck Describes Significance of Longtime Friends Like Matt Damon: ‘Bread of Life’s Leaven’ (Exclusive)

Casey Affleck is reflecting on his bond with longtime friends like Matt Damon.

The 49-year-old actor spoke with PEOPLE during the Center at West Park’s 2nd Annual Fall Benefit in New York City on Thursday, Nov. 14, where he was asked about the significance of his friendship with Damon, who was an honorary chair at the evening’s event.

“Someone said old friends are the bread of life’s leaven,” Casey tells PEOPLE. “The older I get, the more that means to me.”

“People you’ve known for a long time become more and more important the older you get, and they are for me for sure,” he continues.

As for his presence at the benefit, which celebrates the arts, the Manchester by the Sea star says in part, “[Art] was something that I just loved early on, and I think it does a lot for society.”

“Arts, they entertain us and they keep culture going, and the arts are the mirror that we look into when we want to understand who we are and what we’re doing,” Casey adds.

Casey and Damon, 54, have been friends for more than four decades, along with Casey’s older brother Ben Affleck. The trio have collaborated numerous times, including on 1997’s Good Will Hunting and, more recently, this year’s The Instigators.

“We have this 43-year shared history,” Damon told PEOPLE in August of his longtime bond with the Affleck brothers. “We grew up together.”

The Boston natives’ “shared experience and relationship and trust and love,” as Damon put it, is what makes collaborating such a natural process.

“There’s an underlying understanding of who we are,” he told PEOPLE, adding, “In our business, there’s a whole language that gets created around trying to protect people’s feelings because people’s egos are involved.”

He also said that he and Casey “don’t waste any time on diplomacy, which I love.”

Damon’s memories of pursuing Hollywood dreams alongside Casey and Ben, 52, in the years leading up to the Oscar-winning success of Good Will Hunting included not being “afraid to be vulnerable with each other as young men” in Boston.

As he recalled to PEOPLE, “Part of being an actor and putting yourself in those positions and getting rejected constantly — knowing that somebody else is living that same life and making those same sacrifices, that shared sense of experience, that’s why we would clump up and hang out together and talk about it.”