This is the burning question for the characters in the play, especially for Catherine and her sister Claire. One of the first lines Catherine speaks is "I'M not crazy," directed to Hal when they discuss Robert's notebooks at the beginning of Act I.
Auburn presents differing attitudes to the question of what Catherine has inherited. Claire is almost dead certain that Catherine has inherited their father's mental instability (if not the graphomania yet). Hal is convinced that Catherine has inherited her father's genius but not the mental instability nor the graphomania (obsession with writing that comprises nonsense).
Catherine herself believes both things. She sees the things about herself, like her hyper-emotionalism and her excessive drive to solve an obtuse mathematical problem, as signs of her father's illness ... plus that she talks to her father after his death ... none of which help her know what to think about her inheritance: gift or curse.
Finally though, we discover Catherine herself has written the coveted mathematical proof:
HAL. You wrote this?
CATHERINE. Yes ... I started after I quit school. I finished a few months before Dad died.
Hal's persistence and patience wins out and the audience believes, even if Claire and Catherine don't, that, at least for the present, Catherine has inherited Robert's genius and that within that stable genius there is presently no room for mental instability.