The most important reason for the decrease in the rural, non-agricultural population was the growth of towns and cities. These new urban areas became the focus of most of the non-agricultural economic activity.
Before 1500, it made sense that there would be people living in rural areas without farming. These would be people who were smiths or millers or cobblers. There would be a need for them in every area of England and the difficulty of travel would have meant that they would need to live near the people with whom they would trade.
Later, this proximity became much less important. Eventually, greater security and better roads (though these did not come for quite some time) made it easier for people to travel. Food production increased and towns could arise. Now, the economic activities that had been done by these people in rural areas could be done in more centralized locations. What this meant was that people could live and work in towns and sell what they made both in the towns and to people in the countryside. This was more lucrative for them than living in rural areas and only being able to sell to people very near to them.