seems to have the subtext of "Are all superheroes on-topic?"
Is there any superhero that wouldn't count as fantasy if it's not sci-fi?
What do we count as fantasy?
Any superhero that has a "superpower" is going to be either science fiction or fantasy, and thus on-topic for this site. Heroes that do not have superpowers (i.e. Batman) would have to be taken on a case-by-case basis. Batman, as well as many other heroes without powers, are part of universes where there are other heroes and villains who do have powers, or the universe that they live in is clearly "alternate history" and so they are on-topic.
Now as to whether a particular superpower is fantasy or science fiction is a subject of much debate. Many of the folks in the hard SF crowd hold that anything not strictly within the bounds of physics as currently understood is fantasy.
On the other hand, many super powers have scientific, "handwavium", or pseudo-scientific explanations, or are part of the spectrum of abilities categorized as psionics. Any powers meeting these criteria will generally be categorized as science fiction by most people (including me).