Your question is a little confusing, as Cousin Joshua St. Claire is a nominal character in To Kill a Mockingbird by Harper Lee, and he and Aunt Alexandra never speak in the novel because he is long dead. He is Aunt Alexandra's and Atticus's cousin, not a contemporary of Jem and Scout.
We meet Cousin Joshua in chapter 13 of the novel, and of course Aunt Alexandra is the one who brings him up. We know that she is the one who is terribly (overly) concerned about family heritage, both her own family's and others'. She is very quick to condemn an entire limb of a person's family tree simply because of one "bad apple"; however, just listening to her one would never guess that any of the Finches displayed anything but exemplary behavior. Fortunately, we have Atticus and his children to tell us the truth.
When Aunt Alexandra arrives in Maycomb, Jem and Scout are dismayed, but Atticus knows that he is going to need her here as the trial is approaching and trouble is brewing. She insists on talking to (lecturing) the children about their extraordinary heritage, and to prove how wonderful the family tree is, she brings out a
purple-covered book on which Meditations of Joshua S. St. Clair was stamped in gold. “Your cousin wrote this,” said Aunt Alexandra. “He was a beautiful character.”
Jem examined the small volume. “Is this the Cousin Joshua who was locked up for so long?”
Aunt Alexandra said, “How did you know that?”
“Why, Atticus said he went round the bend at the University. Said he tried to shoot the president. Said Cousin Joshua said he wasn’t anything but a sewer-inspector and tried to shoot him with an old flintlock pistol, only it just blew up in his hand. Atticus said it cost the family five hundred dollars to get him out of that one—”
Aunt Alexandra was standing stiff as a stork. “That’s all,” she said. “We’ll see about this.”
Obviously this is not at all how Aunt Alexandra intended the conversation to go. In her mind, Cousin Joshua was a commendable person based on the book he wrote and which she now counts as one of her prized possessions (which we know is true because she brought it with her on this trip to Maycomb).
The reality, of course, is that Cousin Joshua was, shall we say, an eccentric man who did at least one outrageous, crazy thing--and probably more, since Atticus says it cost them a bundle to "get him out of that one" (my emphasis).
So, I'm not sure about any memory Cousin Joshua shared with Aunt Alexandria, which is what your question asks; however, it is clear that her memory of him is a lot more angelic and glowing than he deserves.
For more analysis and insights on this classic novel and others, be sure to take a look at the excellent eNotes sites linked below.