How Outlander handled the arrival of Jamie's [SPOILER!] 

WARNING: This article contains spoilers from “Of Lost Things,” the latest episode of Outlander. Read at your own risk!

writers had a lot to accomplish in penning “Of Lost Things,” the fourth episode of season 3 that introduces — and immediately says goodbye to — several pivotal characters from

the third book from Diana Gabaldon’s series on which this season is based

There was Isobel Dunsany, the woman who falls hard for Lord John Grey, a closeted gay soldier; Lord Ellesmere, a rich elderly man who seeks a young bride; and Geneva Dunsany, the young lass who’s forced to marry him. But none are nearly as intriguing as Willie, the cute consequence of Jamie agreeing to take Geneva’s virginity in exchange for keeping his identity secret from her Jacobite-loathing mom.

Executive producer Matthew B. Roberts talked to EW about the challenges in making the latest episode, along with how easy it was to cast Willie, and why they went with a contemporary tune to end the emotional episode.

ENTERTAINMENT WEEKLY: Before we tackle the plot points, what served as Helwater in Scotland?

Matthew B. Roberts: Helwater is called Gosford House. It’s on the east coast south of Edinburgh. It’s just a big estate with a big beautiful manor house where we used the insides. Gosford was used as Helwater, and then we used the exterior of Hopetoun House [near Edinburgh] as Ellesmere’s Estate.

Scotland has many amazing houses and castles and forests and moorlands. All of those things are great for us. We try to stay as close to the studio as possible. But as we’ve gone three seasons, we started to do what you would call ‘shooting it out.’ You know, we shot this area. We shot that area. We now are moving into our fourth season of shooting. We’re expanding the reach a little bit. It has to not look so much like Scotland anymore.

There was so much story to cover in this episode. Geneva is there…and then gone! So long, Lord Ellesmere! How challenging was it?

That’s normally the biggest challenge of doing any episode. We have to hit those high points. There are so many people who read the books and pick out moments that are special to ‘em. There are a lot of fans who love the smaller moments in the book and we can’t fit all those in and we know that. Even within the writers’ room, we could have five different favorite moments and we have to decide which one is going make it into the show. So, it’s not only the fans who do it, but the writers do it, as well. That’s usually the biggest challenge.

Did it take a long time to find 11-year-old Clark Butler, who played Willie?

Surprisingly it didn’t. He kind of jumped out at us during casting. And the way we cast now isn’t the way we cast, you know, even five years ago. Before, everybody would come in and read. Now much of it is done online. So, [the casting director] gathers them all up and then sends them. We can watch [auditions] on our iPhones if we wanted to. And we were all spread out all over; all over the place when we watched the casting. Clark jumped right out. We all agreed. And we don’t normally always agree.

No. We cast the best person for the role. The best actor. That’s been true with almost every single person we’ve cast. Sam is not really like the way Jamie is described in the book and Claire isn’t described like Caitriona. But now they’re beloved. They’ve taken over what people imagine in the books. I think that’s a testimony to their acting and how they’ve really brought these characters to life. That’s the way we cast now.

“A Hard Rain’s A-Gonna Fall” was such a gorgeous choice for the final montage.

It’s a Bob Dylan song. We were talking about actually getting the Bob Dylan version. And then he won the Nobel Prize so we were like,

. And then we heard this band cover it and kind of fell in love with it. It really sang to us because of Jamie and Claire being separated. That’s why it won out. We very rarely use contemporary songs. In editing, we were trying to figure out the right moment to start it because we don’t do a lot of montages either. But that one really worked out.

With Jamie and Geneva, it’s definitely sex. It’s absolutely, purely sex. It’s certainly clinical for Jamie. But there is a lack of intimacy. He’s just doing it to get out of the predicament that he’s in. But we step up the intimacy between Jamie and Claire. It’s more about showing the smaller, intimate moments between the couple once they get back together. The lack of intimacy between Claire and Frank just helps to enhance the intimacy once Claire and Jamie get back together.

Want to get some scoop about the episode’s big kiss between Brianna and Roger? Click here.

And the discussion about episode 4 isn’t over yet! Tune in at noon ET on EW Radio, Sirius XM 105, to hear me and Amy Wilkinson interview Hannah James, who played Geneva.

1 \'This Is Us\' creator breaks down the biggest clue about Jack\'s death

2 \'Blue Bloods\': Amy Carlson leaves after 7 seasons

3 \'The Simpsons\' urges aid to Puerto Rico at end of season premiere

4 \'Outlander\': Lord Grey actor talks about THAT offer from Jamie

5 Heidi Montag and Spencer Pratt welcome son Gunnar Stone