When asking for clarification as to what Hume's principle of conversion actually means and entails, it's useful to get a larger sense of Hume's discussion in "Of Tragedy." What question is he asking? Who is he in conversation with? What is he trying to explain to begin with? What's the focus of the essay?
With that in mind, Hume begins his essay "Of Tragedy" by noting a phenomenon quite common to dramatic tragedy: often, viewers tend to receive greater degrees of pleasure and enthusiasm from more intense displays of sorrow and suffering. Of course, this raises his question, how is it that these extreme displays of suffering can evoke this response? He cites one Abee Dubos, for one particular explanation, which suggests that the real enemy of human happiness isn't so much unhappiness, but rather apathy, and any kind of emotional response (whether pleasant or unpleasant) is still to be held as better than the lack of one. However, Hume is unconvinced by this explanation, because it does not explain how this negative emotional response is transformed into something positive. Even if it is better to feel sorrow than to feel nothing at all, this still doesn't explain how people can turn sorrow into happiness. Next, he cites one Fontenelle, who argues that emotional responses are ultimately interchangeable, so that one can be transformed into its opposite. As an example, one can turn pleasure into pain and pain into pleasure. One can find joy in sorrow, and one can find sorrow in joy. This logic forms the basis for his theory of conversion, which you ask about above.
From here, he comes to the role of oratory and of artistic expression, which serves as the medium through which skillful writers/poets/artists can manipulate the emotions to achieve a desired effect. As he here writes, "this extraordinary effect proceeds from that very eloquence, with which the melancholy scene is represented." Thus, a great dramatist or artist can use their artistic abilities to create this effect, by which sorrow is transformed in the mind of the viewer.
From here, we come to his examples, which serve as illustrations of his theory. What's interesting is that these examples are meant to illustrate ways in which one emotional reaction can be converted to another. In any case, these examples are ultimately drawn from real lived experience, as examples by which one emotional reaction (usually negative) can be converted into another, more positive emotional experience.
The first example is Novelty, which I would suggest reads a lot like suspense. As Hume himself writes,
Had you any intention to move a person extremely by the narration of any event, the best method of increasing its effect would be to artfully delay informing him of it, and first excite his curiosity and impatience before you let him into the secret.
From here, he draws on the example of Othello and how Iago uses this kind of technique to intensify Othello's jealousy, by playing on impatience. By holding back information, by keeping a listener in suspense, a person can more effectively maintain their interest and create a much stronger emotional reaction in them as a result.
As further examples of this Principle of Conversion, Hume mentions the role by which "difficulties increase passions of every kind; and . . . they produce an emotion, which nourishes the prevailing affection." He gives an example of parents with a sickly child, who could be expected to cherish that child all the more greatly as a result of their uneasiness concerning the child's health. Fear and anxiety are here converted into love.
He mentions the role by which a recent death can result in far warmer sentiments toward the deceased on the part of the survivors. He uses the example of jealousy and separation, which can actually intensify affection and strengthen relationships. Ultimately, through the examples, Hume suggests that his Principle of Conversion is actually based in reality and can be seen as a facet of nature and of human behavior, and if we can see it expressed in the real world, we should not be surprised to find the same principle at play on the stage.
(Do note, Hume's essay does not end here but continues onward, but this does cover much of reasoning which your question asks to clarify. I hope it has proven helpful in this task.)