A woeful run since Christmas has seen Liverpool win only once in seven games and even that was against a virtual Aston Villa Under-23 team in the FA Cup third round.
Against United they looked far more like their old selves, especially the misfiring forward line, with Mohamed Salah getting back to form with two well-taken goals.
They looked uncomfortable at the back though as United counter-attacked to devastating effect and were knocked out by a superb free kick by Bruno Fernandes late on.
It is one of the toughest periods Klopp has endured at Liverpool for some time but the German was upbeat despite highlighting the decisive mistakes which proved costly.
“It’s not what we wanted, so it’s frustrating,” Klopp said. “We weren’t on our absolute top, but we made a lot of steps in the right direction.
“You could see the boys really wanted to change the fortune, they wanted to score goals and that’s good. We did that, we scored twice, which is good as well. That’s all okay but in the end they scored three, we scored two and they are through.
“But we can take positives out of this game, of course — I saw a lot of steps in the right direction and that’s all good.”
After watching the way his side were undone on the break by United, he said they would need to work on the training ground this week to prepare for the trip to Tottenham Hotspur who also rely heavily on counter-attacking.
“It was good preparation for Tottenham,” he said. “We know exactly what we have to work on. If you are in a situation you don’t like, you want to get out of it.
“We want to get out of it and for that we have to make steps. Tonight we made these steps — not the final ones, but we made steps and that’s for the moment okay.”
Klopp denied his team, who would slip out of the top four in the Premier League if they lose to Tottenham on Thursday, are running low on confidence.
“Tonight I didn’t see any confidence problems. We are on it, believe me, and we will sort that,” he said.