It's also important to note the Brutus doesn't quite understand the ghost's message to him. When the ghost says it will speak to Brutus again at Phillipi, Brutus interprets this to mean that there will be another appearance of the ghost once they reach Phillipi. What Brutus doesn't understand, however, is the other way you get to see ghosts again; the path that he will take by the end of Act 5!
The ghost of Julius Caesar arrives and pays a visit to Brutus in his tent, warning Brutus that he will see him again at the battle of Philippi. When Caesar's ghost first comes into his tent, Brutus reacts by saying,
"I think it is the weakness of mine eyes
That shapes this monstruous apparition.
It comes upon me.--Art thou any thing?
Art thou some god, some angel, or some devil,
That mak'st my blood cold and my hair to stare?"
So apparently Brutus was quite afraid when he saw Caesar's ghost (his blood ran cold and his hair stood up on end). But after the ghost leaves, Brutus calms down, saying that he would have spoken with him more had he stuck around long enough.
Check the links below for more information about both Brutus and this scene - Good luck! :)