The mixture of coffee and milk would constitute a homogeneous mixture. This is because when the two substances blend together, the mixture itself takes one "same" (homogeneous) form. When the milk and the coffee mix, the resulting substance is a one-form color brown that takes the characteristics of the substance of which you have added the most. In other words, both substances blend in together to form a complete combination of the two. 

Contrastingly, a heterogeneous mix is obvious to the human eye because its components often remain separate and you can see each one individually while they are still part of a mix. For example, oil and water would be heterogeneous because they are substances that, upon mixing, will immediately separate and remain individual substances. Another example of a heterogeneous mix would be cereal and milk. Unless the cereal is meant to be soluble as in some cases, what we often see in a bowl of cereal is a mix of a solid (cereal) and a liquid (milk) each adhering to their original forms without turning into one same thing. An even more simple example of heterogeneous mixtures would be a bowl of Skittles, where each small piece of candy has its own color and does not mix with the others. 

Is milk from the store a homogeneous mixture?

There are types of milks based on how it is produced and thus they may exist in different kinds of solutions. The milk that is drawn directly from the cow is not yet processed so you can see the fats floating and other substances. In this case the milk is a heterogeneous mixture. If you process the milk, the fat is whipped that it forms into tiny particles thus making the milk homogenous. The milk that is placed in the carton for direct consumption is called "processed milk" or "homogenized milk."