Jem cuts the tops off of Mrs. Dubose’s camellia plants because he is angry that she insulted Atticus.
Jem does his best to maintain his dignity in a difficult time, but people are often insulting him and Scout because his father is defending Tom Robinson. This is more than they can accept, because they are racist and Tom Robinson is black. The fact that Atticus Finch, a fine upstanding citizen who is well-respected in the town of Maycomb, would defend a black man accused of attacking a white girl, is unheard of.
Mrs. Dubose is a mean old woman whose sickness makes her meaner. She has a tendency to shout insults at the Finches, and on this particular day the insults are directed at Atticus. When She says something about Atticus, Jem stops. She knows that her insult has hit its mark, and takes pleasure in the fact that she has made him mad.
"Yes indeed, what has this world come to when a Finch goes against his raising? I'll tell you!" .... "Your father's no better than the niggers and trash he works for!" (Ch. 11)
Jem gets angry. In Scout’s words, he “simply went mad.” He takes Scout’s baton, and takes his anger out on Mrs. Dubose’s camellia plants because they are a handy target. She prizes them.
He did not begin to calm down until he had cut the tops off every camellia bush Mrs. Dubose owned, until the ground was littered with green buds and leaves. (Ch. 11)
They seem to be the one bit of beauty in her life, and he wants to take that away from her. He even snaps Scout’s baton in two, yanks her hair, and kicks her.
Atticus responds by having Jem read to Mrs. Dubose, so he can learn what real courage is. He wants Jem to see that Mrs. Dubose is a slave to her disease, because she is addicted to morphine and is slowly meaning herself off of it. By the time she kicks the habit, she herself will be dead. Jem will learn real courage from this though, that real courage is fighting when you know that you can’t win. This is what Atticus is doing in the Tom Robinson case.