Hmm...this is a bit of a big question that requires some digging. I hate to do all the digging for you (it seems somehow against the spirit of things) so why don't I help you with the first few and leave the last couple for you to solve?
First, let's be clear about what an "epithet" is. In short, it is an adjective or a phrase, often negative, that describes a person or thing's character (such as dirty old man.) Using that definition, we can look around in the first of the books you mention to find more:
Book 10: ‘Odysseus, noble son of Laertes,' 'rugged Ithaca,' 'high Olympus' 'the bold hero Odysseus'
Book 11: "feckless ghosts" "house of Hades" "Ilius, the land of noble steeds" "'My son,’ she answered, ‘most ill-fated of all mankind,"
Book 12: "the child of morning, rosy-fingered Dawn" "the great and cunning goddess Circe" "renowned Odysseus" "dread Charybdis"
These are just some that I found. I am better at finding the ones that relate to people than I am to objects, but I think you get the idea of the kind of things you are looking for. I hope these examples have helped you, and I will leave the last two books for you to uncover the epithets in yourself. Below I have given a pretty good page on epithets in Homer, though they are not arranged by book.
1. grey-eyed Athena
2. Odysseus the great tacticion
3. Zeus who views the wide world
4. Zeus the summoner of the cloud
5. Clear-headed Telemakhos
6. Zeus the lightning's joyous king
7. gently born Eurykleia