Pi is relating his story to the "author," the person referred to Pi by Francis Adirubasamy who told him "“I have a story that will make you believe in God." The author is feeling uninspired whilst writing a book, especially as his previous books failed. He has met pi in a restaurant to hear his story.
Pi is now living in Canada and discusses his first visit to a restaurant. Of course, the waiter serving him has no idea of the ordeal Pi has just experienced and he makes a glib comment when Pi uses his fingers to eat. " “Fresh off the boat, are you?” As an Indian, eating with fingers is part of culture and quite normal. Eating with "these utensils" is more unusual than eating with fingers. Pi loses his sense of enjoyment, as his fingers " became dirty under his gaze."
It is very hurtful to Pi as part of the eating process for him is allowing his hands to act as "taste buds savouring the food a little ahead of my mouth." Just as Pi thinks he can exist normally, he is made to feel like an outcast, an uncultured, uneducated person.
The irony is that Pi really has just got off a boat(at the time of the restaurant incident) and the 227 days he spent on the boat would not have met anyone's level of acceptable behavior. Pi is confused about his ordeal and is now embarrassed that his behavior in Canada is viewed as backward or basic and apparently revealing a lack of education and is maybe even questionable. "They (his fingers) froze like criminals caught in the act."