Leading 3-2 after two goals from Olivier Giroud and one from Aaron Ramsey had seen the Gunners twice come from behind after a brace from Roberto Firmino, they were hit with a late sucker punch from substitute Joe Allen.
With Manchester City being held to a goalless draw at home to Everton, Arsenal missed the chance to open up a five-point gap over their third-placed rivals - Leicester's win at Tottenham ensured they stayed level with the leaders.
"I am frustrated by the result and not disappointed by the performance," said Wenger, who confirmed they have finally concluded the transfer of Basle midfielder Mohamed Elneny and he could be in contention to face Stoke at the weekend.
"We started a bit apprehensive in the first 20 minutes, a bit on the back foot and went 1-0 down, came back convincingly, went 2-1 down, showed character again, made it 2-2 and then 3-2.
"The frustration comes from the fact that at 3-2 we had three situations where we should have made it 4-2 and made bad decisions.
"The frustration then comes from conceding a goal in the last minute.
"We have shown again great character to come back with a point.
"We have to transform the frustration in the dressing room into even more motivation.
"There are a lot of positives
What happened in this game should strengthen our belief."
On Elneny, Wenger said briefly: "He has joined us and we will see if he can qualify for Sunday."
Belief was something Liverpool manager Jurgen Klopp was pleased to see from his side after conceding two soft goals.
"It was a great football game and ultimately we will talk about this game maybe not too long because Sunday is Manchester United but it is not easy to forget," he said.
"The beginning was really good, not perfect but not too far away from it.
"Set-plays we have to defend better
We have to work on it, will work on it
We will analyse it and work it better.
"The second goal, the set-play, he (Simon Mignolet) needs help from all the players.
"At half time we came back in the game and again I think it is brilliant from Giroud but in the beginning we have to defend it better and be more concentrated.
"Then you have low battery and you need a bit of luck and we had it
We changed players and I think it was a well-deserved draw."
Source : PA