An odd but valuable question. Both supervisor and clerical worker are humans, not just "human resources." While much of a clerical worker's "output" can be quantified -- forms processed, files retrieved or filed, hours worked, years employed, punctuality, errors, etc. -- the putatively more important traits -- ability to work with a team, ability to follow instructions, ability to meet deadlines, ability to work under pressure, ability and willingness to follow company rules and protocols, etc. -- are actually judgment calls by the supervisor (and it is essential that the supervisor actually supervised the worker daily or weekly).
All these features are legitimate areas of inquiry. What are not allowed in such an evaluation, however, are personal prejudices; attention to social grouping (gender, sexual preference, race, national origin, age); personal friendship or compatibility with the evaluator (dating, family get-togethers, etc.); unquantifiable traits (voice tone, eating habits, posture, etc.)
Of course there are many areas where the line is unclear: clothing (Do the worker's outfits violate the unwritten dress code?) Is his/her personal hygiene compromising the office's atmosphere? Are outside-of-work living habits (drinking, gang activity, political activism) disrupting the teamwork? Does his/her physical appearance prevent customer relations (beard, tattoos, gross obesity, dyed hair, etc.)
These last features are what make the evaluation task so difficult and where errors in performance appraisals are most likely to happen -- because worker and supervisor are human beings.