Viola pities Olivia because she does not realise that Viola is not really a man, but a woman (compare Viola's soliloquy 'I left no ring with her', where she calls Olivia a 'poor lady', and suggests that she were 'better love a dream').
The simple answer to your question can be found in the lines below: 'I pity you', Viola says, and Olivia responds
That's a degree to love.
No, not a grize; for 'tis a vulgar proof
That very oft we pity enemies.
Olivia seems encouraged by Viola's pity because she believes that pitying her is, at least, one step towards loving her (a 'degree' to loving). Viola replies, rather persuasively (although Olivia doesn't seem to hear!), that pity isn't anything to do with love - because we pity enemies and we don't love them!