In a sense, when O'Brien says that each soldier carries "ghosts," he is not referring to any specific and identifiable thing or person.  He's talking about the war itself, which has become a kind of ghost.   For example, in the episode in which a Long Range Recon Patrol goes into the mountains to listen for any activity, the narrator tells Sanders

. . . it's spooky.  This is the mountains.  You don't know spooky till you been there. . . .Serious spooky.  You just go with the vapors--the fog sort of takes you in. . . .

The listeners begin to hear music--chamber music, cocktail party music, the Haiphong Boys Choir, "and a barbershop quartet"--and, eventually, the only way to stop the music is to call in artillery and air strikes on the jungle.  Later, the narrator confesses to Sanders that all of the details about the music are untrue but says, "listen, it's still true."  When Sanders asks him what the moral is, his answer is perfectly logical in the context of a ghostly experience: "'Hear that quiet, man?'" he said. "'There's your moral."  In other words, the war--which has no connection to normal human experience--becomes a ghost, something they feel, and sometimes hear, but not a thing that they can identify or, more correct, objectify.  They know the war exists, but it's not tangible, touchable, because it's not subject to logic or reason.  

After the killing of the baby buffalo by Rat Kiley, Mitchell Saunders observes

For the common soldier, at least, war has the feel--the spiritual texture--of a great ghostly fog, thick and permanent.  There is no clarity. Everything swirls.  The old rules are no longer binding, the old truths no longer true. . . . You can't tell where you are, or why you're there, and the only certainty is absolute ambiguity.

The men, at least in O'Brien's story, become part of that "ghostly fog" and can be said to carry the fog with them every step they take, which blankets them with a moral, physical, and intellectual ambiguity that they feel, but can't act upon and resolve.  When, at the end of the "How To Tell A True War Story" section, O'Brien says that their experience in Vietnam "wasn't a war story.  It was a love story.  It was a ghost story," he is explaining the Vietnam experience as a kind of ghost that every soldier lives with for his entire tour, and even after his tour and return to "the World," that ghost is still with him.

This quote appears abruptly in the middle of a paragraph describing a variety of personal items that imply certain personality traits;

Until he was shot, Ted Lavender carried the starlight scope, which weighed 6.3 pounds with its aluminum carrying case. Henry Dobbins carried his girlfriend's pantyhose wrapped around his neck as a comforter. They all carried ghosts.

By carrying a scope, Lavender might be implied to be a good shot, or someone who prioritizes necessity over luxury. In contrast, Dobbins is implied to be superstitious and perhaps a bit nonsensical. This comparison is wrapped up into a cohesive idea through the use of the word "ghosts".

By ghosts, the author means unseen forces, such as memories, particularly ones that harken back to the soldier's lives in America, as well as the things they have done in Vietnam. Since ghosts are typically associated with haunting people, this implies that the soldiers are silently influenced and distinguished by these ghosts, and are at least partially responsible for the different choices the soldiers make in terms of their behavior and the things they choose to carry. Ghosts might include sentimentality for a loved one, like Dobbins and his girlfriend, or fear of the unknown, or hatred of communists, or guilt for a thing done or not done.