Hi Faham,
Our apologies for the late response. We again did not want to risk other errors that the update may have caused in the last few days of the study.
Participants do not respond as quickly on weekenddays. The study has stopped now.
We do want to follow up with you about some of the issues we found.
We have detected the following issues:
- Participants who have succesfully filled out surveys (that we indeed received) and disappeared days later
- Participants whose surveys remain “grey” and turn to “expired” days later
- Participants who have filled out surveys that we never received (with evidence of submission)
The following are concrete examples, matched per issue above. We think the same issues may aply to multiple participants for other days as well.
- We compared data files we downloaded from several dates with one another: later dates with previous dates. We were not able to check all files for all days, so these are also just specific cases we found.
17281: on December 12th surveys of December 10th, 11th and 12th are now marked as expired that were successfully submitted previously. We also see that the device ID of this participant changed on December 10th. The same is true for 16225. The surveys of this participant that were once submitted were marked expired from November 30th onward. This change happened on Deccmber 11th or 12th. - This change in device ID we see with other participants as well, if surveys changed status after a few days it happened from the date of device ID change. Most of these turned from “grey” to “expired”. We don’t know if these could have in fact have been submitted (see also our next point). We do find participants for which this is the case (grey --> green).
- 16293: submitted a survey on December 11th on 12:18 (see screenshot of last page of survey). In Ethica it is marked as expired on 12:31.
Ethica dashboard as well as Kibana do not show submitted surveys (only expired).
We have the following questions, again organized per issue:
1a. Can we retreive all lost data for these participants?
1b. And is there a way to detect all other participants this happened to?
We cannot compare all possible combinations of files; and we only have saved downloads for some days and surveys have also been lost within a day (we saw this happen).
2a. How can we know for certain that these questionnaires were not submitted and then lost?
2b. How can we know the cause for “grey” surveys: because they were never sent out/received or simply expired?
Eg participants with broken phones also have “grey” surveys. Should these not be marked as blocked? And some questionnaires are still marked as “grey” while Ethica apps have been synced.
3a. Can we retrieve all lost data for this participants? And detect participants with similar issues? We have had more participants with similar issues, but no screenshots.
Or, is it possible to simply receive all datafiles for each survey for each participant?
Thanks,
Tim, Teun & Irene
