In spite of his initial famous rebuff of Elizabeth, when he refuses to dance with her because "she is not beautiful enough" to tempt him, Darcy quickly becomes enamoured of Miss Elizabeth Bennet, in spite of Miss Bingley's constant taunting of him and Elizabeth's own family and the way they expose themselves to ridicule. Note what he says to Miss Bingley in Chapter 6 when she tries to second guess his thoughts as they watch the dancing:
My mind was more agreably engaged. I have been meditating on the very great pleasure which a pair of fine eyes in the face of a pretty woman can bestow.
What draws Darcy to Elizabeth, at least initially, is her "fine eyes" and her physical beauty. This comes to be developed when he discovers more about her character and what a principled, generous and kind woman she is who is not afraid of challenging even such an august person as himself when he is wrong and commits something in error.