The suggestion here is that the prison seems to be built to keep people out instead of being built to keep people in.
Most jails and prisons are meant to keep prinsoners from moving, escaping or generally being capable of committing crimes again. They are places of captivity, not places of defense.
In this description, McCarthy says that the prison is oriented in an unusual way. It looks like it is meant to act as a keep, a castle, or a defensive structure to repel attacking enemy forces.
This description also suggests that the building is archaic, a suggestion which evokes images of dungeons and darkness from a time in history that no longer exists. The prison is therefore a bit "fantastic" and strange. It doesn't belong to the world that the boys live in.