What does the phrase 'the significance of a study' refer to?
The qualitative significance of a study is as above in readerofbooks' reply. This is to say that the results of a particular study are of more practical purpose compared to other studies.
However, the quantitative significance of a study is an entirely different concept, and is a technical term in the paradigm of statistics that may be used when describing the results of social surveys for example.
In statistics, the term 'significance' refers to how likely the study results are under null conditions. In any rigorous scientific study, one has a null hypothesis as a starting point. The null hypothesis can only be rejected if the results significantly contradict the null model. In statistics, the null model might be that the data (or the data mapped to a more convenient scale) follow a Normal or Gaussian distribution. If the results of the experiment, or in the case of social statistics perhaps a survey, are particularly unlikely under the assumption that the null model is true, then the result of the study is said to be significant in that the observed data are significantly far from typical results that would be seen if the null were true. For example, the mean of the data may be somewhat higher or lower than that specified under the null model, or the spread (or variance) may be significantly larger or smaller than that under the null. To make a clear and objective decision as to whether the result of a study can be described as significant, one needs to choose the size of the test (or type I error =alpha). Typically, as suggested by R.A. Fischer, and then adopted forever more, we choose alpha=0.05 or 0.01 and if the test statistic lies in the 100alpha% tail of the null distribution (which may be Gaussian for example), we say the result is significant at the 100alpha% level, eg 5% level.
Importantly, the result of a quantitative study may be statistically significant, but if the practical difference between the null model and that indicated by the data is negligible in real-world terms, that is to say we are quite confident it is numerically different, but the numerical difference is too small a change to care about pragmatically, then the result may be statistically significant, but not of practical or qualitative significance. On the other hand however, a result cannot be of qualitative significance and not of statistical significance, because lack of statistical significance suggests no change at all. This might be counter-intuitive in scenarios say where the mean of the observed data is practically larger than that in the null model, but in statistical terms there simply aren't enough data points to convincingly prove that the null model has been rejected. This idea may be contentious in real-world scenarios where for example a cluster of deaths from operations occurs, but there are not enough events to say that they weren't by accident and emanate from a sinister cause. In reality, clusters of unwanted events can occur by chance alone, and it is universal chance that the paradigm of statistics works around and attempts to account for. It is the story of 'the boy who cried wolf' - if too many accusations are made that are not warranted, then onlookers, or society, end up ignoring those who make the accusations as the accusations aren't specific enough. Where concerning or interesting results are seen in a study that indicate there may be significant change or evidence against the null model, these results are termed 'hypothesis-generating'. The study is not the last word, but funding bodies or other parties might be persuaded to invest resources into further studies in the area of research.
This is a good question. The significance of a study can be measured by how important a study is. To put it another way, some studies are more important than other studies. To be able to grasp fully the significance of any study, or anything for that matter, it will require time. Only history can determine the significance of anything.
With this stated, the significance of a study is usually tied to how other scholars view the work in view. More specifically, does a study produce something in the future? Does it produce a shift in paradigm? If it does, then we can say that the study is significant. Let me give you an example. In the field of sociology, Foucault work on sexuality was very significant, because it created a new way to look at sexuality in the west.