I agree. Removing the downvote option quite literally makes communication harder, despite being a listed goal for the change.
Downvotes also allowed us to indicate when a link was malicious. People are less likely to click "Here is the new update: [Virus]" if there are downvotes.