This quote comes from the chapter, "Those Who Don't." The meaning of Esperanza's statement, "All brown, all around, we are safe," bespeaks the reality of racial division and prejudice that exists in the segregated world of contemporary society for Latinos and Latinas living in Chicago in the mid 1980s.
Esperanza has already witnessed the clashes of racial identity in previous chapters. She never truly feels safe and comfortable unless she is among her own race. She has felt the discomfort of being in an integrated school where she is perceived as the "other" ("Our Good Day"), has seen her cousin's friend Louie arrested for being in the wrong place at the wrong time ("Louie, His Cousin, and His Other Cousin"), and other injustices that have nothing to do with character but everything to do with prejudice. This is why Esperanza says when able to relax in her own environment,
"All around brown, we are safe. But watch us drive into a neighborhood of another color and our knees go shakity-shake and our car windows get rolled up tight and our eyes look straight. Yeah. That's how it goes."
Other neighborhoods have proved dangerous, causing the fear. Their car is secured to keep the friends safe. Their eyes look forward to keep from inviting trouble.