What two structures are present in Chapter One and what do they indicate about Puritan society?
The King's Chapel being discussed so very little displays the Puritan's desire to separate themselves from England and establish their own identity, but still have some reverence for it in appearances.
The prison has great features that show potential symbols for your future reading. There are iron spikes and weather-stained wood indicative of a troubled prisoner within. Also, just outside the prison door there is a blooming rosebush with an incredible fragrance and perfect buds right now because it is June in chapter one.
This rosebush reminds me of the perfect socitey the Puritans longed for.
The prison, as mentioned above, indeed represents their necessity to punish crime. They took the justice their God was to deliver into their own hands and punished crimes harshly.
The only two structures present in Chapter 1 are the prison and the King's Chapel. The prison is discussed at length, the chapel is only mentioned in passing.
To me, this indicates that Puritan society was (or at least Hawthorne thought it was) as concerned with punishing the people who broke the rules as it was with saving souls. Perhaps even more so, given how much the prison is discussed.
By discussing prison so much in the very first chapter, the author is showing us his belief that Puritan society was very punitive.