An agent, such as a virus, toxin, or chemical, that attacks or affects the nervous system preferentially.