Fernando Torres scored a dramatic late header to take Liverpool back to the top of the Premier League and leave Portsmouth in relegation trouble.
David Nugent gave the hosts the lead with a low shot before Fabio Aurelio equalised with a powerful free-kick from inside the area.
Hermann Hreidarsson restored Pompey's lead with a stooping header.
But with only five minutes remaining Dirk Kuyt powered in from an acute angle before Torres' late cameo.
With the win in the bag, manager Rafael Benitez will feel vindicated for starting without his star striker who had been rested following his exertions during the FA Cup defeat by Everton in midweek.
But with the scores at 2-2 on 89 minutes, many Liverpool fans would have wondered why Torres had not been brought on earlier.
Benitez also chose to rest Kuyt for much of the match, electing to play what many would consider a weakened XI.
The Reds found it difficult to adapt to playing with each other in Benitez's chosen system, which saw three at the back and Yossi Benayoun and Ryan Babel playing off the lone striker David Ngog.
It was a tough ask for 19-year-old Ngog, who was given his first start, to lead the line against heavyweight centre-backs Sol Campbell and Sylvain Distin.
Unsurprisingly, he did not get a sniff on goal, although Benayoun did in one of the few efforts in the opening 45 minutes, when he found the side netting after being put through on goal.
Moments later, Brazilian Aurelio went even closer when his 25-yard free-kick smacked off David James's right-hand upright.
James, who equalled Gary Speed's all-time Premier League record of 535 appearances, was asked to perform heroics only once when he tipped Javier Mascherano's 30-yard flier round the post.
It was not all Liverpool, however, former Anfield striker Peter Crouch and Nugent giving the visiting backline a decent workout.
But between them they only created one chance of note in the first half when Nugent's lay-off was hit on the half-volley by Crouch over Pepe Reina's crossbar.
Everton fan Nugent was seen taunting the opposition with a '1-0' gesture, which seemed to refer to Liverpool's Cup defeat.
Adams still has faith despite defeat
And the former Preston striker rubbed salt in Liverpool's recent wounds when he picked up Crouch's throughball, sprung the offside trap and placed his effort under Reina's body and into the goal.
With Kuyt already brought on to replace Ngog, Benitez made another change by bringing on playmaker Xabi Alonso.
But it was Aurelio who pulled the scores level with a cracking free-kick, which came about after James was forced to handle Crouch's suicidal backpass with Benayoun bearing down on goal.
Aurelio stepped up to take the kick inside the penalty area and buried it beautifully low through the legs of Crouch and Nico Kranjcar, who was on the line.
Benitez brought on Torres in search for a winner, but it was at the other end where the game took another twist when Hreidarsson headed Nadir Belhadj's free-kick into the ground and past Reina, who seemed confused by the bounce.
It was turning out to be a horrible week for Benitez until his substitutes got him and the side out of jail with two classic strikes.
In the 85th minute, Torres picked out Kuyt inside the area whose neat footwork created enough space to fire in a wicked shot high past James.
Torres then took centre-stage in the dying moments to convert Benayoun's pin-point cross past the reach of the Pompey keeper.
Portsmouth: James, Johnson, Campbell, Distin, Hreidarsson, Basinas, Davis, Mullins (Kranjcar 46), Belhadj, Crouch, Nugent.Subs Not Used: Begovic, Lauren, Mvuemba, Pamarot, Kanu, Gekas.
Goals: Nugent 62, Hreidarsson 78.
Liverpool: Reina, Carragher, Skrtel, Agger, Dossena (Alonso 67), Arbeloa, Mascherano, Aurelio, Babel (Torres 76), Ngog (Kuyt 56), Benayoun.Subs Not Used: Cavalieri, Hyypia, Riera, El Zhar.
Goals: Aurelio 69, Kuyt 85, Torres 90.
Ref: Howard Webb (S Yorkshire).