England began the shoot-out with a record of six defeats in seven at major tournaments.
Britain's Prince William said: "I couldn't be prouder of @england - a victory in a penalty shootout!"
But Yerry Mina's injury-time header levelled things, putting an exhausted England through half-an-hour of tension in extra-time followed by a nerve-shredding shootout.
The Three Lions have claimed a 4-3 shootout win to advance in Russian Federation.
Police said that an investigation is underway and "as a result of the measures taken by police, the individual who committed this act has been identified and detained". We've ticked one of the boxes so far. It shows our togetherness and character. He is the player that everyone is talking about - wherever you go at the moment, people from all around the world.
The 31-year-old will experience a rivalry most England fans won't encounter during the match, as he lives with his Colombian girlfriend Angelica Pinzon in the country's capital Bogota. It's a big night for everyone.
"There's always going to be a penalty shoot-out in tournaments like this, I know that from watching over the years, so it's just about practising after every session on the training field and we've done that".
Hazim Owens, 25, from Muxton said: "I feel absolutely fantastic. We know obviously in the past England haven't done great and it was great to get that one off our back".
But Southgate said he was already looking forward to Saturday´s quarter-final against Sweden in Samara, with a last-four clash against either Croatia or Russian Federation up for grabs. "I'm just looking forward to the next game".