First, I think it is important in poetry NOT to assume the poet is the speaker. Just as in a novel the author is not necessarily the narrator, poetry can be told from any perspective. Perhaps the reason it is more emphasized is due to the brevity of most poems and their lack of typical narrative elements. Even narrative poetry as compared to a novel is going to leave a lot out. Many readers tend to default to the author and speaker being one and the same as it is often the easiest.
Important aspects of the speaking voice: I think it depends on the poem. The basics I like to run most poems through are are male/female, age, cultural background (if present), experience (if present), and overall feeling. Again, beware of putting a picture of the author in any of these. Often there is not one correct answer to any of these criteria, but by choosing, it at least gives you a narrower scope through which to analyze the poem. At the very least, ask how YOU connect with the ideas in the poem, then put the speaker through a comparison of yourself. Do you tend to agree with him or her? Have you had similar thoughts/ideas/feelings/experiences?
It is always interesting when two people read and adequately defend two completely opposite ideas for who the speaker might be. I hope this at least gives you a place to start.
Since poetry is a kind of speech, there must be a speaker in every poem. We generally tend to equate the speaker/the speaking voice with the poet, but the equation is over-simplistic and mostly not correct. The speaker may be just another man or woman; he/she may be young or old; he/she may be a happy, pleasure-loving soul or a grave, contemplative person; he/she may tend to be somewhat idiosyncratic, obsessive, or may be very sane and normal.
Proper understanding of the nature and identity of the speaker is essential for understanding the poem and its perspective. Take, for example, the speaker in Browning's dramatic monologue, 'My Last Duchess'. It is a megalomaniac duke whose reflections on his dead wife are integaral to his character. The speaker in 'The Last Ride Together' or the speaker in 'Porphyria's Lover' is quite different from the character of the duke. The middle-aged Prufrock in Eliot's poem, 'The Love-Song of J.Alfred Prufrock' is a glaring example of how the complexity of the speaking voice may lend linguistic, structural and conceptual coplexities to a poem.