The primary taste in a coffee or tea is tannin. Tannin is an acid found in many parts of trees and other plants, which has a strong, bitter flavor. While it is an acid, many acids have different flavors; while lemons and limes taste 'acidic', hot chili peppers and spices are also acids. Tannin is just another acid along this line.
In addition, the tannin in coffee also acts as a mild astringent, constricting and shrinking tissue that it comes in contact with. This can change the texture of the tongue while drinking coffee, and contributes further to the taste.
Tannin is mainly found in tree nuts, leaves and bark. Coffee, tea, and wine all contain high concentrations. Tannin is used outside of the food industry in tanning leather, which is why some leathers can taste bitter if you accidentally get them in your mouth.