How do the literary concepts of 'intertextuality' and 'dialogism' contrast or compare to each other?
Considering that these two terms signify the double or multiple voices of every utterance and every written word or text, it is impossible, within the discourse of literary theory, to say one without conjuring the other. Bakhtin's dialogism and Kristeva's intertextuality are very similar. And over the course of the history theory, since Kristeva, the differences between the two terms has become (dialogically) more defined and more blurred. Both terms acknowledge that every word or text is always already double-voiced (or multiple, heteroglossia). You can stretch the definition of dialogism to be identical to intertextuality but that just muddies things up for the purposes of answering your question. And if you were to do so, polylogism is a better approximation of what intertextuality is.
Even though the term intertextuality was coined decades after dialogism, dialogism is a kind of intertextuality. In fact, some have called dialogism a kind of INTRAtextuality in order to distinguish it from intertextuality. For Bakhtin, one of the dialogues always already ongoing in a novel is that between a character and the author. Another is the double-voiced function ongoing within the individual character him/herself. Then there is also the dialogue ongoing between multiple works of the same author, but dialogism can extend to other authors, works and genres. Bakhtin noted dialogism is a function of all language; not just literature or within certain genres.
Bakhtin takes this intra- to more inter- levels using other texts and genres, but Kristeva's use of intertextuality really elaborates and broadens dialogism. She employs not just formal, contextual dialogues (intertextual movements); she also thinks about the semantic, syntactic and phonic deferrals. With the latter, phonic, Kristeva brings a psychoanalytic approach. In other words, a spoken or written word that sounds like another (i.e., violins, violence) opens a dialogue between the two as it relates to a subject, character, text, patient, etc.
Overall, intertextuality is just more broad than dialogism, extending the Bakhtinian double-voiced function to interdisciplinary lengths. And with more recent scholarship on intertextuality, there is more emphasis on the reader's role in producing these intertextual links than the links just 'being there' by themselves. In other words, there is a dialogism in the works of Dostoyevsky and a reader, in any period, expands those dialogues with his/her own interactions. So, there is also a kind of non-linearity that is more characteristic of intertextuality than dialogism. For Bakhtin, dialogism is more logical if it is based within a particular socio-historical context. Intertextuality is a freer, more postmodern version of it.