I’m having some problems with understanding our survey data. Study #3911, survey #21937, 21938 and 21939.
We’re using a study which, after a practice survey, sends a daily survey 7 days in a row. These surveys are triggered at a time chosen by the participant and expire after 3 hours. Participants also fill in a debriefing questionnaire in the week after these 7 days. When installing the app together with our participants, we ask participants to fill in 2 surveys: 1 practice survey and 1 survey to choose their preferred time to receive the 7 surveys (out of 6 timing options - 11 am, 1, 3, 5, 7 or 9 pm).
We just started including participants and are having two issues so far in understanding our data:
One participant joined our study January 24. He has 2 unanswered surveys and we don’t understand what this means. The Avicenna FAQ seems to tell me that it means his phone hasn’t updated yet, but after noticing the unanswered surveys, we saw him completing still another one. He also told us he filled in 2 out of 7 daily questionnaires while we only have the results of one.
Also, is there any way we can instruct our participants to prevent receiving unanswered surveys?
We use a one-question survey for participants to choose which time they want to receive their daily surveys. We configured some extensive triggering logics for the triggering to work. What I don’t understand in our data is that our participants have a lot of ‘extra’ unanswered surveys, but they seem to be related to the other time options than their chosen ones. So for example, one patient chose the surveys to be sent at 5 pm. Our activity responses on January 26 show that he completed the 5 pm survey. But it also shows unanswered surveys for the same participant at (some, but not all of) the other timing options (1, 3, 7 and 9 pm). We included 4 participants so far. While these unanswered surveys at the not-chosen time option don’t show for our first included participant, it does for all other 3 participants.
I’m wondering what this means. I assume our participants aren’t receiving the same survey 6 times a day (I haven’t checked but I can ask them; however this wasn’t the case when pilot testing our study)? But why are these extra surveys still in our data?
Can someone please clarify these two issues for me? Thanks in advance!
Hi Fay @fbanffe
This is Amin from Avicenna product team, and I’ll be happy to assist you with this.
One participant joined our study January 24. He has 2 unanswered surveys and we don’t understand what this means
While my guess is that the initial 2 surveys (the practice survey and the time slot selection) are most probably the unanswered ones, I highly recommend you go to the Activity Session tab from the left sidebar, then select the participant in question and all the activities to see the unanswered sessions along with all the details (when the survey was prompted, the session ID, etc). You might need to change the view from Day view to week or month view to see the unanswered sessions. For more info please read here: Activity Sessions | Avicenna Learn
… What I don’t understand in our data is that our participants have a lot of ‘extra’ unanswered surveys, but they seem to be related to the other time options than their chosen ones. …
If you have checked the Activity Sessions and the sessions are created and tagged as unanswered, then my guess is that probably there’s an issue with the way you setup your criteria, but not so sure about that. I think you set 6 triggering logics for the daily activity (for the 6 time slots) and set a criterion for each one like this: if Qi == n where i determines the i-th Question in your study and n determines the n-th time slot. At times this might result in invalid criteria if the participant fails to respond to the initial survey. If it’s possible, please add me as a researcher to your study so that I can find the root cause and update you. My researcher email is: amin.ghaffari@avicennaresearch.com.
Best,
Amin
While my guess is that the initial 2 surveys (the practice survey and the time slot selection) are most probably the unanswered ones, I highly recommend you go to the Activity Session tab from the left sidebar, then select the participant in question and all the activities to see the unanswered sessions along with all the details (when the survey was prompted, the session ID, etc). You might need to change the view from Day view to week or month view to see the unanswered sessions. For more info please read here: Activity Sessions | Avicenna Learn
The initial 2 surveys aren’t the ones that are unanswered. We fill these in together with our participants so this cannot be the case.
Thank you for your advice on checking the Activity sessions! I can see that the unanswered surveys are the final 7th survey and the following debriefing questionnaire. This makes a bit more sense. Should I check if our participant has deleted the app or could there be another explanation why the final 2 surveys (from which the first one should already have been expired) are still unanswered?
If you have checked the Activity Sessions and the sessions are created and tagged as unanswered , then my guess is that probably there’s an issue with the way you setup your criteria, but not so sure about that. I think you set 6 triggering logics for the daily activity (for the 6 time slots) and set a criterion for each one like this: if Qi == n where i determines the i-th Question in your study and n determines the n-th time slot. At times this might result in invalid criteria if the participant fails to respond to the initial survey.
I think I set the triggering logics up in a different way, actually I made it far more complicated by using 42 triggering logics (7 x 6 with different messages for each day). I think this went alright, but I added you to our study to check just in case. The ‘extra’ unanswered surveys (related to the non-selected timing options) which I keep seeing in our Activity Responses, actually don’t show in the Activity Sessions. So I think there isn’t any problem with the triggering logics, but can you maybe explain this difference? We don’t want all these unsent/unanswered surveys in our data…
Hi Fay @fbanffe
Sorry or the delay, somehow I wasn’t notified of your message,
I will check your study and get back to you.
We had an issue with our notifications (we have fixed that and the latest version should be available to download by this Monday (Feb. 17th, 2025). If there was an issue with the notifications, that explains why, other than that, there’s a 99.9% chance they have simply forgot to respond to the survey.
As for 42 TLs, I remember that, I gave you the instructions to use Button Caption placeholders
Best,
Amin
Hi Amin,
Thanks for your reply.
I hope your update resolves any issues our participants been having.
Can you maybe explain to me how a survey that should already have been expired for quite some time could still remain unanswered?
And yes that’s true about the TLs.
Let me know when you’ve been able to check our study.
Hi Fay @fbanffe
I went through all the sessions for the three activities listed in your study and saw absolutely no issues when checking for the following participants:
***155
***694
***065
***457
***896
***982
***556
***643
Moreover, the app update is released so any rare notification issues should be rectified.
Please let me know if I overlooked anything and I’ll be more than happy to double check and get back to you.
