So, the answer here can be yes or no? I'd have to go with no.
My three reasons would be as follows:
1) Madison did not think a bill of rights was necessary. He did propose it once discussion was underway, but he did not in general think it needed.
2) When people called him the father of the Constitution, he himself claimed that he was not responsible for it, but that many people wrote it. I say take him at his word; it was the work of many.
3) The bill of rights was strongly influenced by previously existing documents, including the Virginia Bill of Rights, which Madison did not write.