The point of view is first person from the perspective of the people who live in the same town as Richard Cory. The point of view is revealed starting in the second line of the poem:
Whenever Richard Cory went down town,
We people on the pavement looked at him ... (1-2)
The people of the town live a life quite different from Richard Cory in that they struggle to make ends meet, and they admire his wealth and assumed success, but from a distance. Having the speaker as a plural "we" highlights Cory's lack of connection with the town. The people class themselves off from Cory (they're on "the pavement," while Cory "glittered"), and the poem gives the impression that Cory does not have a personal connection or a friend in town. The people simply look at him and admire and envy his pristine appearance.
The distance between Cory and the townspeople is further revealed in the third stanza when the speaker reports:
... we thought that he was everything
To make us wish that we were in his place. (11-12)
It is perhaps in part this impersonal relationship with the town-- this disconnect-- that leads Cory to "put a bullet through his head," even as the townspeople labor and wish they had a life like his. No one knows the inner life of Cory, a life that has clearly been troubled; all they know is of his material wealth and his difference from them.