(I'm going to create my own examples so as to not make this question an accusation at anybody. Please consider these claims a representation of my own mostly-suppressed quackery, and not representative of any other SE users.)
Suppose someone answers a question about "is there free will" with "there is no free will because aliens extracted it from humanity in 1500 BCE to power their starships".
Now, certainly, I do not believe aliens took free will from us.
But I also might equally disagree that, for instance, we possess a divine soul that enables us to make decisions, or disagree that we are just complicated chemical processes that behave according to the laws of physics. I generally don't downvote answers that rest on assumptions or metaphysics I disagree with, but do often downvote "quackery".
It seems like the things that "quack" answers have in common are:
- They rest on an uncommon belief system.
- They do not specify that belief system in the answer. (For instance, I wouldn't consider an answer like "if one subscribes to Gnosticism, then..." quackery, but I would often consider "Yaldabaoth created this world to imprison our free souls and create the illusion of no free will".)
Because the belief system is uncommon and unspecified, the answer seems unlikely to be useful to the OP or to the broader audience. Am I being fair here? I do not generally downvote e.g. a Christian answer for excluding "from the Christian perspective," at the top. Should I be consistent and downvote all answers that do not try to explain their assumptions and groundings? Should I refrain from downvoting things that seem reasonable if I accept the unstated premises the respondent is working from?
(I'm also not sure that my opinion of these answers would change if, for instance, my "aliens stole free will" answer was formatted like "If you believe my aliens stole free will theory [link], aliens stole free will in 1500 BCE to power their starships." It's certainly better, but.. I'm not sure I could ever see myself upvoting it, at least.)
Related: Should we explain downvotes?, Is it ok to down vote an answer because you disagree with something it said?, and the rest of the voting tag.