Here can mean in this place or in this spot or more generally, Present – presently in this spot or presently being considered or talked about. Here is wherever the speaker literally is or whatever the speaker is doing at the present time or about to do. “Here we go.” Since here refers more to the vicinity of the speaker, it does make more sense to say here when the speaker has it in his hand and to use there when it is no longer in his hand.
But, here and there can be interchanged; they are both used to convey giving something to someone regardless of whose hand it is in at the time. It is idiosyncratic and derives from different dialects and probably could be considered bad grammar but it has just become accepted. I will say that here are there are vague concepts that denote calling attention to an object, idea or direction to an object or idea. Here and there are not defined by their relation to people. They are words that direct the attention of the listener to an INDICATED place. If you are standing within two feet of another person and hand them a present, you can say “here you are” as the present is in your hand and you indicate this. You could also say there you are as you mean for it to leave your and go to their hands. Or, saying “there you are” indicates that “there” is a place that is extended from the speaker’s body as in moving away from the general or even essential “here” of the speaker. In the end, I agree with your assessment. Using here and there is based on direction, presentation and indication of the speaker. The closer the proximity to the speaker’s body, state or essential being would use here and if the speaker is indicating something apart from himself or giving something away, there makes more sense. But you will find this general rule is bendable and that’s why you’ll see them used interchangeably especially when here or there indicates something relatively close to the speaker.