As the question says, is there a way to ensure that a user/users are not targeting you for downvotes?
Failing that, what system exists to ensure that this does not happen?
While there's always the possibility that you've been very cleverly targeted, a look at your recent Q+A would strongly suggest that you're not being targeted at all.
You've had 27 downvotes in the last 30 days, which certainly seems an unusually high number until you start to break things down:
4 came from this off-topic (and now closed) question
8 came from this self-answered (and now closed) question and answer set
4 came from this self-answered question about accents
Why does everyone in Game Of Thrones have a British accent?
6 came from this poorly received question about a prop on GoT
Who sent Ser Barristan the pardon for Jorah Mormont?
Of the remaining 5 downvotes, the remainder are single downvotes on questions that have been quite poorly received in terms of the ratio between question visitors and upvoters.
None of the downvotes you've received this month seem in any way remarkable once you accept that self-answered questions invariably attract at least one or two downvotes.
There is already a system to detect malicious voting patterns and correct the reputation accordingly.
If someone serial up-votes and down-votes your answers/questions, you have to wait for at least 24 hours. When the script is run, your reputation will be corrected to pre-malicious-voting level.
Read this page for help.
Quoting from the same page:
When a single user continually votes (up or down) on many of your posts within a short period of time, the system considers these votes to be invalid and removes them.
What to do when you think you are victim of serial voting?
If you see very unusual votes being targeted at your account, don't worry about it. You should wait at least 24 hours after noticing before becoming concerned, as the automated system should detect it and reverse it for you. If, after 24 hours, you do not see any fix to your reputation, please contact the team using the "contact us" form located at the bottom of any page.
Also see: What can I do about revenge downvotes?
What happens to the perpetrator(s):
If such a voting pattern continues to happen between two users mutually or from one user towards another, or otherwise falls outside of normal voting patterns, moderators and/or developers may investigate the matter; intentionally voting merely to reduce or inflate another user's reputation is considered abuse.
The exact nature and rules of the reversal script are kept in secret to avoid potential abusers from finding work-arounds. Relax, your reputation will be back by tomorrow.