There are two main groups who would have cause to object to a subsidy for sugar.
First, anyone who competes with the sugar industry will object to such a subsidy. Subsidies lower the cost of producing sugar and increase its supply. This makes sugar cheaper and makes it harder for those who make competing products (non-sugar sweeteners, for example) to compete with sugar.
Second, all taxpayers would have cause to object to the subsidy. When taxpayer money is used to subsidize sugar, all taxpayers are basically being forced to transfer some of their wealth to sugar producers. They must do so regardless of whether they would voluntarily consume much sugar. This is unfair to taxpayers and they would be justified in objecting.