Victor Frankenstein, in the aptly named novel Frankenstein, states that, after he successfully creates the Monster he did not want to return to his apartment because his "heart palpitated in the sickness of fear". Prior to this statement, Victor had awoke from a nightmare to find his Monster staring at him. Victor, frightened by the figure, fled from his apartment.
The terminology used in this quote show Shelley paid great attention to the crafting of her protagonist's thoughts. For purposes of understanding the two word choices, one must first define them appropriately.
According to the Merriam-Webster Dictionary, palpitated means "to beat rapidly and strongly". Here, this word appropriately fits the description of what Victor is feeling upon contemplation of returning to his apartment where he assumes the Monster is waiting. The level of the word, given not more typical like beat or pounded, heightens the reader's senses as to match what Victor is feeling. (Think about the scary part of a movie- you know something bad is going to happen and your heart beats furiously anyway.)
According to the Merriam-Webster dictionary, sickness means "a disorder, weakened, or unsound condition". This describes Victor's state to a "T". He is horrified by the thoughts of his first view of his Monster. The thought of seeing the Monster again weakens him, causes him to feel unsound. He is obviously disturbed by the thought of what he has done just as much as he is at the thought that he will encounter the Monster again.