All developing embryos start off as more or less female. Genetically they are male. There is the presence of the Y chromosome right from conception, but it doesn't do anything right away. That means for the first several weeks of any pregnancy, the developing embryo essentially follows a "female blueprint." Nipples are included in that blueprint.
Sometime after week four of development, the Y chromosome starts kicking in. After about 8 weeks, testosterone production begins, which starts creating basic male sex characteristics. Other hormones, along with testosterone, act together to inhibit further development of female ducts and structures. But by the time all of that happens, men's nipples have been in place for quite some time. Hence, as your original question stated, dudes have nipples.