Biff knew that his father would question whether he had actually gone to see Bill Oliver to ask for a loan. His impulse to take the expensive pen may have been motivated consciously or unconsciously by a desire to have tangible proof that he was in Oliver's private office. But after he had stolen the pen he quickly realized it didn't prove anything positive but only that he was a thief. 

Another possibility is that the pen seemed like a trophy. Biff's best years were when he was a star athlete in high school. He was always bringing home trophies, and Willy displayed them all with pride. Biff may have had an impulse to bring his father the pen as another trophy. But it would have been a poor substitute for the ten- or fifteen-thousand dollars that Willy expected his son to borrow from Bill Oliver to start a sporting-goods business. The pen was not something to be proud of but, on the contrary, something to be ashamed of. He couldn't give it to his father. He couldn't keep it himself because it would remind him of his humiliation. He couldn't throw it away. And he could hardly return it to Bill Oliver, who would already be missing his expensive pen and realizing what must have happened to it. 

The pen makes Biff realize his mediocrity. He doesn't deserve to own such a pen. He regrets taking it, just as he regrets trying to be something he cannot be. 

Biff Loman resembles Christian Darling, the focal character of Irwin Shaw's story "The Eighty-Yard Run" (1955). See reference link below.

Biff stole while growing up and he steals as an adult. He takes what he wants without hesitation. He blames his father for this habit because Willy did not punish him for stealing when he was young and instill in him a sense of integrity.

Biff had gone to see Bill Oliver to pitch a deal and borrow money. Oliver keeps him waiting for hours, which Biff surely resented, and then dismisses him quickly. Biff, left alone, goes into Oliver's office, where he certainly does not belong, and takes the expensive pen. In one regard, sneaking into Oliver's office and stealing his pen is a petty, angry reaction to having been kept waiting and then rejected by Oliver. Biff tells his brother Happy, "[H]e gave me one look and--I realized what a ridiculous lie my whole life has been!" Oliver's success reminds Biff of his own life's failure. Taking the pen was also an impulse, a crime of opportunity in which Biff's basic character asserts itself.

Always one to avoid responsibility, Biff enlists Happy's aid in lying to Willy about stealing the pen. Without any hesitation, Happy covers for Biff with their father, making up a story to make Biff seem innocent of any misdeed.