Jeffrey Dean Morgan Carries One of His Daughter’s Stuffed Animals Everywhere He Goes (Exclusive)

When Jeffrey Dean Morgan is away filming in the zombie apocalypse, his 7-year-old daughter George is never far from his mind.

“On my backpack that I carry everywhere, I have a stuffed animal that she tied on there like three years ago,” The Walking Dead: Dead City star exclusively tells PEOPLE.

The proud girl dad, 59, grins as he shows off more mementos he keeps with him on set. “She makes me bracelets,” he says, holding up his wrists. “I’m always wearing these really colorful kid necklaces.” 

The image of a plushie-toting softie is a far cry from Negan, the foul-mouthed, baseball-bat-wielding menace Morgan has embodied for the past decade on The Walking Dead and, now, its spinoff, Dead City. But the small tokens help Morgan feel close to his daughter and 15-year-old son, Augustus (Gus), whom he shares with his wife, One Tree Hill star Hilarie Burton, 42, while filming away from their 100-acre sanctuary, Mischief Farm, in Rhinebeck, New York.

For more on Jeffrey Dean Morgan, pick up the latest issue of PEOPLE, on newsstands Friday, or subscribe here.

Last year, Morgan also appeared on Amazon’s The Boys and lent his voice to their superhero series Invincible. His new gig as host of NBC’s reality series Destination X has taken him even farther from Rhinebeck to Europe, which, despite stealing some date time away with Burton in France, has made family time tough.

Related Stories

Jim Gaffigan Says Getting Teens Out of Bed on Weekends Is a Game: ‘Do You Blackmail, Motivate or Go to Full Crazy?’ (Exclusive)

Kylie Kelce Struggles with This Parenting Dilemma: ‘Don’t Come for Me’

“I’m not going to lie. It’s been hard,” Morgan says of balancing his career and fatherhood. “My daughter has been a little bit unhappy with me. It was so much different when we just had Gus. He traveled with us when he was young and we didn’t care if he was missing five days of kindergarten. But now that he’s in high school, George is kind of stuck. She wants to be with Dad all the time, and it just hasn’t worked out that way.”

But Morgan says his daughter, whom he describes as “very much a daddy’s girl,” gets it. 

“As long as I tell my daughter, ‘Look, you’re going to get a new dragon out of this,’ or whatever it may be, she knows that Daddy has to go make a couple of dollars to pay for the Amazon account,” Morgan jokes. “So it does ultimately work.”

Recently, they did spend some quality daddy-daughter time together as the Morgans stepped out at Canneseries for the world premiere of Dead City season two in April. 

“I wanted the kids to come because Daddy was getting his hands in the concrete there, and I thought, ‘This may never happen again, and it’ll be here forever,’ ” Morgan says of leaving his handprints on the Cannes Walk of Fame. 

Of having the whole family with him, he adds, “It was really cool. It was George’s first red carpet, and they stole the show. I thought Gus was going to walk away with a three-picture deal.” 

On Morgan’s new show, Destination X, players travel across Europe on a blacked-out bus. Each week, they attempt to guess their location through a series of challenges. The contestant whose guess is farthest from their destination each week gets sent packing.

“With this show, it was a bit harder when you’re across the world. They didn’t come out,” Morgan says of missing his kids on the road, adding, “But our number one priority will always be our kids. I find a way to get home every weekend if I can.” 

Destination X airs Tuesdays at 10 p.m. ET/PT on NBC. Episodes stream the next day on Peacock. TWD: Dead City airs Sunday nights at 9 p.m. ET/PT.