This conversation occurs in Chapter 4, and is when Grant is feeling very entrapped and frustrated with his position in life. He feels that he is being forced against his will to spend time with Jefferson thanks to his mother, and as he contemplates his life he thinks that he is lacking the opportunities and chances to get ahead that he deserves and wants. What frustrates him is the commitment he has to his society. He expresses to Vivian his desire to just "pack up and leave" and run away from those commitments and ties, and Vivian responds by challenging him and saying that as teachers they have a "commitment." Note how Grant responds:
You hit the nail on the head there, lady--commitment. Commitment to what--to live and die in this hellhole, when we can leave and live like other people?
This upsets Vivian, and when Grant expresses his anger yet again by swearing, she threatens to leave. What is key to realise in this section of the novel is that Grant is experiencing massive conflict between his own desires and what he wants to do and what he feels his society, and in particular his mother, is forcing him to do. It is this conversation with Vivian where he expresses the massive frustration he feels about this conflict, but it is also this conversation where Vivian helps him to see that he has responsibilities that he cannot run away from as easily as he thinks he can. It is Grant's attitude towards his "commitments" that angers Vivian and leads her to challenge him about what he should be doing rather than thinking about himself all the time.