It is interesting that you asked the question the way you did: "If I steal someone's pen and replace it with another exact same pen, is it stealing?" You did not say if I take someone's pen, you used the word steal, and this implies that you know it is wrong to take something that is not yours, even if you replace it with an exact replica.
How we deal with these situations in our lives are determined by our ethics, the set of internal rules which govern our behavior. Assuming we have been parented even reasonably well, we understand right from wrong at an early age; even without proper parenting, there comes a time in our lives when we know for ourselves what is right and what is wrong in most situations. When things get fuzzy for us and we are not absolutely certain what is right and wrong, it is often because we want to do what benefits us rather than what is right; then we try to justify our choice by making certain claims. Here are the primary arguments a person might make for and against the idea that taking a pen and replacing it with an exact replica is stealing.
First, he would claim that no damage has been done and no one has been hurt so it is not stealing--since we all know that stealing hurts people, right? Perhaps no measurable damage has been done, but that cannot be assumed from the statement. If the pen were replaced immediately, perhaps the owner would never notice and therefore not feel any damage; if it were not replaced immediately, possible "damages" could occur, such as not having a pen available when he needs it. If the pen is replaced immediately, then what's the point of taking/stealing it at all? Unless there is a valid reason for taking the pen, this is a clear case of purposeless stealing.
Second, he would say that his intent was not to steal, so it cannot be considered stealing. In this case, the better word might be borrowing; however, borrowing is using the pen and then returning it, not replacing it with another pen, even an exact replica. So why did he take one pen and replace it with another? If it were done to play some kind of a joke, perhaps the act could be justified and considered something other than stealing; if not, he is again guilty of purposeless stealing.
Finally, he would argue that since he did not benefit from the "exchange," it was not stealing--because of course people always steal to gain something for themselves, right? First, this is not true, as we have evidence that some people steal simply for the rush it gives them.
Why would celebrities, who have fame, fortune and a reputation to protect choose to shoplift? UCLA psychiatrist Dr. Heather Krell said the drive to shoplift is similar to the drive someone has to become a star.... "You have to believe in yourself and that you're going to prevail and you have to get caught up in the moment. And it certainly seems that that's part of shoplifting," she said.
Second, there had to be a personal motive of some kind for him to do anything at all with this pen; if not, he would have left the pen alone. In some way, he benefited from this exchange and it therefore must be considered stealing; if not, he is still guilty of purposeless stealing.
"Stealing is taking another person's property without permission," according to the excellent eNotes article (linked below). The person who stole this pen and then replaced it with an exact replica may try to justify his act as mere taking; however, by definition it is stealing.