Though I cannot find a definition specifically for "blurring literary genres," I can imagine how this would apply to some of the short stories of Edgar Allan Poe.
First it is helpful to understand what a literary genre is, which you probably already know.
A literary genre is a category of literary composition. Genres may be determined by literary technique, tone, content...
The most general genres in literature are...epic, tragedy, comedy, novel, short story, and creative nonfiction. They can all be in the genres prose or poetry, which shows best how loosely genres are defined.
With this in mind, Poe's work represents several genres. Poe wrote not only short stories of horror, but also of detective stories, as well as poetry (such as "The Raven" and "Annabel Lee"). In terms of the "blurring " of genres, I would expect that this means that the distinction of genres as listed above are blended to include the characteristics of more than one genre.
As an example, in Poe's "The Tell-tale Heart," we have a story of horror. The main character kills the old man he lives with and hides his body beneath the floorboards. However, this is not just a story of horror or a thriller, but rather a psychological thriller, a blend of "thriller" and "horror" genres. There are several themes in these kinds of stories. Two deal specifically with reality and perception, which Poe "plays" with in this story. The narrator's sense of reality and perception are skewed, which heightens the suspense and the reader's sense of terror: the narrator is "haunted" by the open and clouded blue eye of the old man, and believes he must kill the older man. His perception is that the eye drives him to commit murder. Poe also uses an altered reality when the old man's "dead" heart begins to beat so loudly that the narrator's fragile grip on reality is destroyed: in this case, his perception and his sense of reality drive him over the edge, and he admits what he has done to the police.
This is, I believe, an example of "blurring" literary genres. Poe learned how to do so exceedingly well, providing dark entertainment for the reader that is the cornerstone of modern horror stories and thrillers; stories that are not easily forgotten.