One of Shakespeare's principal themes in As You Like It is the difference between the court and country or, more precisely, the court from which the rightful Duke has been ejected and the forest of Arden where the Duke and his followers seek refuge.
Under normal circumstances, the forest represents a sinister place where outlaws go in order to escape punishment. In As You Like It, where the court represents treachery, disloyalty, and corruption, the Forest of Arden represents safety and natural goodness even though Nature can be harsh. For example, Duke Senior and his followers are living "like the old Robin Hood of England," a reference to a very positive cultural figure in medieval England, a victim of the tyrannical exercise of wrongful laws.
Even more to the point, Charles describes the Duke's new way of life as living "as they did in the golden world," which refers essentially to the world before it was corrupted by sin--an earthly paradise. This "golden world," although it can cause great discomfort (cold, for example, which several characters suffer from), everyone in the forest in "more free from peril than the envious court."
The contrast, then, is between envy and treachery in court life or the freedom of life in a natural paradise, which has some physical drawbacks, but is infinitely preferable to the corrupt nature of the court and its politics.