Well, let me start by stating that I don't think we should. However, using Mill's reasoning, there are certain solid reasons we should. First, doing so provides a universal binary scale: you can appraise just two factors, pleasure and pain, rather than duty, religious imperative, etc. This provides a strong universal reference. Second, this unified scale would seem to escape the conflicting claims that so trouble people: what happens when two duties clash? When legal duty and religion clash? This scale dissolves those. Third, it allows for a kind of universal calculus of ethics—a hope of objectivity.