Unfortunately for you, there is no way (that I am aware of) within the Stack Exchange framework to block one user from interacting with one other user.
That said, users have been suspended previously for leaving disparaging comments on questions and answers alike. As a general rule we try to resolve such situations amicably, but when that fails, suspension is an option.
As for downvoting, votes are for each user to use as they like. The system will detect and reverse serial voting, where a single user leaves a large number of votes for a single other user, but unless a large percentage of the votes you receive is from this user, or a large percentage of the votes given by this user is on your posts, that heuristic is unlikely to be triggered. (The exact details of how serial voting is detected and what the trigger thresholds are are deliberately vague, to keep people from gaming the system.)
And that's even assuming that the downvotes are from that one user, which you can't know. Even moderators can't see who voted up/down on a particular post. That data is obviously saved, since you can see your own votes and the system can handle them differently based on sequences of events, but it's not accessible.
The general idea is that a single vote shouldn't have a huge impact anyway. Yes, it can sting to get a downvote, but just through a quick glance over your profile, judging by the net votes, your questions seem well received by the community.
The proper response to unfriendly behavior, whether on your own posts or others', is to flag, and do nothing more. Don't respond to it at all. If there's something worthwhile in the response, just phrased harshly, then you might consider incorporating it into your post, but I'd still recommend not responding.
If a user gets a significant number of helpful flags against their contributions, especially the rude or unkind kinds of flags, then this shows up to the moderators for review separate from those flags themselves (which also show up, but only for as long as they are active). The moderator reviewing the situation then chooses what to do, where depending on the situation the immediate options range from doing nothing at all to the user via a user account annotation or moderator private message to issuing a suspension (of varying duration, at the moderator's discretion; there's a suggested progression, but the moderator issuing the suspension can decide on a different suspension length, either shorter or longer than the suggested duration, if they feel that is warranted).
Quite simply, moderators can't manually review everything on the site. Flags are the community's way to raise issues that need handling, but which don't need to be discussed in public on Meta.
${}$
, which is invisible when it renders, to the end of your title.