It is important for Willie to appear to be "making it." The tape recorder was probably prohibitively expense (I actually remember when they were uncommon and very expensive) and certainly nothing that Willie needed. It also suggests that it was important for Willie "reenforce" Howard since he was totally dependent on him for any future job security.
On another level, it symbolically showcases the difference between Willie's world and Howard's. Willie's world is better symbolized by the organic, the garden that he wants to grow that the modern world and its large apartment buildings will never allow. The tape recorder, the mechanical, non-organic device, is symbolic of the future/present world, the world that Willie can never belong to. He may want to, but it can never happen.
During their conversation, Howard is immensely distracted by the tape recorder. Howard is not interested in Willy at all and already knows that Willy is going to be released from his position with the company. Willy tells Howard that he is also going to buy a tape recorder because Willy wants Howard see that he is modern and cutting edge. Willy wants Howard to see that he too is successful enought to get the new technology. Willy is above all trying to please Howard.