DNA consists of polymeric chains of nucleotides. A nucleotide consists of three parts, a phosphate group, a deoxyribose sugar ring, and a nitrogenous base. The phosphate and ribose are really present simply to act as a backbone for the double helix. So a covalent bond is really important here for purposes of rigidity and strength. It's the bases that really form the heart of the hereditary information contained in the DNA. G and C are complementary bases, and T and A are the other complementary pair. These base pairs are complementary because their structures fit together in a hydrogen bonding sense like jigsaw puzzle pieces. C and T are pyrimidine structures while G and A are purine structures. The reason that complementary base pairs across the double helix bind to each other with hydrogen bonds as opposed to covalent bonds is so that the double helix can separate when necessary for things like protein synthesis. This way the DNA can unwind and separate temporarily and then reform the double helix when mRNA formation is complete. If there were covalent bonds, this would not be possible.