Validating survey with Plausibility report

[Todd WhiteParticipant]

To what extent should the results of the ENA for SMART Plausibility Report be used to reject my survey?

[SMARTKeymaster]

The results of the ENA Plausibility Report assesses the overall quality of the data collected from your anthropometric survey. The Plausibility Report is a standard tool that ensures comparability between surveys and provides managers an easy tool for evaluating the data. It is important to run this tool when analyzing your survey data. That said, the overall score is not meant to be used as a definitive tool for validation, but rather a prompt to highlight key issues for concern. A qualified survey manager should read each test and use it to assess the quality of the survey data. The tests are a tool to identify both selection and measurement bias. The tests may also highlight field realities.
For example, if your survey identified more younger children and is; therefore, penalized for age distribution (and has a highly significant p-value), the survey manager should evaluate whether there truly are more younger children in the target population or whether there is an issue of selection bias where younger children were more likely to be surveyed than older children in the population. To determine this, survey managers should first discuss the field realities with survey teams and cross-check with previous surveys in that region. If there is reason to believe the sample may not be representative of the population in terms of age, consider the penalty points. If not, these penalty points can thus be ‘ignored’ but the age distribution should be discussed in the discussion section of the survey report.

[NarendraParticipant]

Can someone tell me how digit preference score has been calculated in the plausibility report? What is the statistical tests were used to analyse the DPS?

[SMARTKeymaster]

Weight and height measurements are taken to one decimal point, and MUAC measurements are taken to the nearest millimetre. A random distribution is expected for the first decimal of weight and height measurements (kg and cm, respectively), and the last whole number of MUAC measurements (mm). In order to identify any rounding of measurements, the digit preference test is performed on each basic measurement. The test used is derived from the WHO[1] MONICA study of blood pressure. The formula de-sensitises the test to allow minor degrees of digit preference which are not sufficient to alter the results of the survey. If the numbers are truly random then each terminal digit (from 0 to 9) should occur in approximately one tenth of the observations. In the plausibility test output each hash tag represents the proportion of measurements recorded with that particular number as the terminal digit. If there has been no rounding, then there should be a similar number of hash tags for each of the ten digits (0 through 9) since the probability of having each of the last (terminal) digits is equal. A simple Chi-squared test of the observed frequencies against the expected frequencies is performed to evaluate this criterion, and a Digit Preference Score (DPS) is then computed using the following formula:

DPS = 100 * (χ2 / (df * N))1/2

Where N is the number of observations (subjects in the survey), χ2 is the chi-square statistic for the test of homogeneity of the terminal digits, and df are the degrees of freedom (i.e., df = 9 because there are 10 possible terminal digits). The DPS ranges from 0 to 100. It is low when there is no digit preference and high when the digit preference becomes large enough to affect the result of the survey6. Proper training in anthropometric measurements is essential in eliminating high digit preference scores.

[1]http://www.ktl.fi/publications/monica/bp/bpqa.htm

[NarendraParticipant]

Thank you for your response.

The link you shared at the bottom of your response is not working. Could you please send me the document explaining DPS in more detail or the publication you referred in the link?

[Blessing MureverwiParticipant]

Try: http://www.thl.fi/publications/monica/bp/bpqa.htm

[SMARTKeymaster]

Thank you Blessing! The correct link is http://www.thl.fi/publications/monica/bp/bpqa.htm . Alternatively, the study can be found by performing a google search: WHO MONICA study of blood pressure

[Author]

Posts