Every time people lament changes to the lore that amount to "not every member of species X is irredeemably evil" and claim the game is removing villains from it, I think how villains of so-caleld evil species fall into two cathegories: a) bland and boring and b)have something else, unrelated to their species going on for them, that makes them interesting.
Too weak? Nah, that's just bad story telling. Humans aren't weak, even if we'd lose a bare knuckle fight against most other animals. Tools will ALWAYS magnify strength far more than bestial fortitude even can.
At least make their downfall something more believable, like they were too prideful and arrogant to take a serious threat seriously until it was too late. Even if the mechanics of the loss involve humans being physically weaker, it cannot be the overarching "just because".
It's like Batman. Batman can take out most all of the others in the Justice League despite being "just a human", in the right scenarios. Nobody likes a story where Batman is simply crushed by a superior force.
But that's against a creature without sapience. You can't lazily outsmart an enemy as smart as you, can use tools, AND is innately stronger than you. Like, its trivial to come up with a scenario that fits @BudgetBandit's summary:
The human realm is invaded by an alliance of beastkin that are as smart as they are, but stronger and more agile. They attempted to stand against them, but their armies were too numerous and had a superior fighting style. Soon enough, the last standing human Kingdom fell under siege of the combined might of the beastkin and the remaining humans went to ground in order to survive.
That's a combination of their premise and your downfall/underestimation with like 5 min of brainstorming. Don't assume people aren't telling a complex story just because they summarize it in a single sentence.