Security-wise you want to put secure data in a separate list. SharePoint does not have field-level security, and since surveys are basically a specialized list with a bunch of fields, anyone with read access to the item could also see the 'hidden' fields.
What I would suggest is to have a separate list with a lookup field back to the survey list. Secure the custom list and allow manager to add their comments, reviews, or whatever to the custom list. Note that this does not really scale well since every survey will be available on the list. Further, out-of-the-box your are limited to the ID value in the lookup box, no 'created by', none of the questions, just ID (and version and content type), so you would make sure that you, or the managers, link to the correct survey using only the ID.
Well, actually, the latter s not entirely true. The lookup field has a bit more complexity as to which fields you are allowed to use as lookup fields. If you like you can, for instance, add a 'What is your name' as a text field in the survey and that field may be used as a lookup field in the custom list.
.b
http://furuknap.blogspot.com/My SharePoint blog with articles, reviews, and code samples.
Anonymized questions and answers may be posted to my blog.