This is a good question. Let me make a number of points on this distinction.
First, when we are dealing with words that are similar, then there is a lot of overlap in meaning and usage. So, I would say that there will be more similarities than differences between the usage of "only a few" and "a few." In other words, these words can be interchanged often. So, it would be correct to say, "I only want a few." And this would essentially mean the same as, " I want a few."
Second, there is a slight difference in nuance. When someone says "only a few," the word "only" limits the word, "few" in a rather definite way. We can say that the word, "only" draws attention to the word, "few."
Third, without the word, "only." Things are less definite. The words, "a few" still limits the number, but it is not as definite.