A goal in each half from Terriers midfielder Aaron Mooy sealed the points for the visitors, as Wolves struggled to establish any sort of rhythm to their play.
While Wolves remain in a healthy league position, the loss was their fourth in five games, and Bennett admitted that the manner of the defeat was hard to accept.
"It was tough. I think I'd go as far as to say it's one of the toughest games that I've been involved with since I've been here," the defender said.
"There have been games recently where we've played well and dominated in parts, but not got the result we deserved, but today wasn't one of those. Nothing went for us - our passing wasn't there; nothing we tried to do came off and we were giving the ball away in areas that we can't afford to.
"We didn't have the fluidity we normally do - we didn't deserve anything from the game and that's really tough to take."
In their previous outing, Wolves had given their fans plenty of cause for optimism with a deserved point against Arsenal at the Emirates - the latest in a string of impressive performances and results against the league's traditional super powers.
Bennett - along with the majority of his team-mates - has certainly been more used to days such as that in his Molineux career so far, but admits that, after losing four out of their last five games, Wolves need to go back to the drawing board to play themselves back into form.
"It's the first rocky patch we've had since I've been here," conceded the former Norwich man, who joined Wolves in the summer of 2017.
"We knew it was going to be tough at the start of the season, and now we're at a stage where we need to try to put things right.
"We worked hard during the international break and we'll continue doing that to get things right as quickly as we can. We'll keep working hard and doing what we believe in - that will see us through."
