Actually now I think of it, Google takes note of the text in links for making up its rankings, so a link that says "This an untrustworthy site - it's crap, useless" would actually count against a site. A little negative link building isn't a bad thing.