Harassment generally comes in two forms: harassing comments by another user posted in your own journal, and harassing posts about you in another user's journal. There are different steps to take for each type of harassment.
In either case, in order for the Abuse team to consider taking action, the harassment needs to be one-sided. The Abuse team will not mediate "flame wars" between two parties, and will not take action in cases where both users seem to be equally at fault.
HARASSING COMMENTS: If someone is harassing you in comments to your own journal, the very first thing that you should do is ban them from commenting in your journal. See http://www.insanejournal.com/support/faqbrowse.bml?faqid=112 for information on how to do this. If it's happening in a community, you'll need to contact the community maintainer to ban the user from commenting.
Because there are tools to block users from commenting in your journal, the Abuse team generally won't take action in cases such as these. If you do feel the need to report them, however, be sure that you include links to all of the comments in question, and do not delete the comments until after the Abuse team has reviewed your case.
HARASSING JOURNAL ENTRIES: If a user is posting journal entries that are intended solely to harass you, first ask yourself whether a casual viewer of that journal would have enough information available to identify you. Then, make sure that it's not just an one-time incident.
If the abuse is chronic and you think that the user is violating the Terms of Service, then make a report with the Abuse team. Be sure to include links to all pertinent entries in the user's journal. The Abuse team won't investigate these cases unless reports contain specific links to investigate.