The importance of the keeper's contribution was swiftly underlined with England doubling their lead in the 59th minute as Jesse Lingard's teasing cross into the box was met by Alli unmarked at the far post to head powerfully past Olsen.
"We don't have renowned world-class players yet", Southgate said, "but lots of good young players who are showing on the world stage that they're prepared to be fearless with the ball, try to play the right way, have shown some mental resilience now".
England take on Sweden in Samara on Saturday with a place in the World Cup semi-finals at stake, and while some may be surprised to see the Scandinavians in the last eight, defender John Stones says they can not be underestimated.
The Croatians lost to France in the semis 20 years ago.
"We were composed and we controlled the game".
"If you look at the games we have played, the opponent has had more ball possession than we have, but we have created more chances, more risky chances", Granqvist said.
He saved superbly from Viktor Claesson before tipping a shot from Berg over the bar and that was enough to ensure Swedish hopes of reaching their first semi-final since 1994 were dashed. At 22, Alli became the second-youngest scorer for England at a World Cup behind Michael Owen. "We worked on that in practice".More news: Deceased Celtics fan is staged playing NBA2K at funeral
More news: Southgate in no mood to change England's winning formula
More news: Jeremy Hunt replaces Boris Johnson as UK Foreign Secretary
They have drawn twice in the group stages of the World Cup since then.
They began solidly against England in a slow and sloppy start to the game that resembled a pre-season friendly at times, only to be outdone by a goal Sweden isn't used to conceding.
"Very happy to be part of this team, I am proud of everyone. This is the way for us to have success, and we're going to continue this way".
Sweden got its best chance two minutes into the second half on Marcus Berg's header, but England goalkeeper Jordan Pickford pushed it out of harm's way. Sterling got hold of the loose ball but his shot was deflected over. He is likely to retain his starting place for the semifinals.
"We were sad and we're disappointed", Andersson said, "but we mustn't forget we put in an excellent performance".
The song was a constant reverberation around stadiums, pubs and homes the last time England reached a major tournament semi-final at Euro 96 and, 22 years on, it is helping to showcase the feel-good factor Southgate has created around the current squad.