English

日本語

PSYCHEXP
TUTORIAL

(ESM) Experience Sampling Method No coding skills required.
overview 1

OVERVIEW

In this tutorial we will be taking a look at how to create an experience sampling method project in psychexp.

The Psychexp Experience Sampling Method (ESM) Survey platform combines both a convenient and intuitive interface for the researcher together with an ease of use experience for the respondent. The ESM project utilizes a progressive web application (PWA) to send notifications to inform respondents when it is time to answer a survey, and the respondents respond on their device's web browser, ensuring a reliable reach and extremely high response rate. The Psychexp ESM platform is compatible with numerous types of ESM research and is highly customizable.

overview 1
What is a progressive web app (PWA)? 1
1 - What is a progressive web app (PWA)?

An ESM project requires some form of signalling system to inform the respondents that it is time to participate in a survey. The Psychexp ESM platform utilizes a progressive web app (PWA) to deliver such a signal to the respondents mobile device.

A PWA is a very small javascript program that is downloaded from a link and installed like an addon to the device's browser. Once installed to the browser it looks and feels like any other native app on a mobile device, however in actual fact it is just a cleverly disguised web browser window that can receive a notification (signal) and can also display the notification to inform the respondent when it is time to complete a survey.

Unlike a native app that can require hundreds of MEGABYTES of space, the Psychexp PWA only requires a few hundred KILOBYTES of space to install the javascript to the browser. Additionally, unlike a native app the psychexp PWA does not need permission to access any part of the respondent's device and only requires permission to receive notifications to the device's browser. A PWA can be installed on either an IOS (safari) or ANDROID (chrome) device. PWA technology and the ability to send notifications to a PWA has been available on the ANDROID platform for many years, however sending notifications to a PWA on IOS is relatively new and as such requires IOS 16.4 or higher to operate correctly.

Installation of the PWA is quick and easy and once installed the respondent needs only to enter a 4 digit code to register for your ESM project. (An email address will also need to be entered if you select to have your ESM project require email registration) The 4 digit code and the PWA download link are both displayed on the registration page so you only need to supply your resondents with the project's registration link or QR-code in order to have them sign up for your project.

Once installed the PWA is accessed from an icon that is automatically displayed on the device's home screen, or when a notification arrives to the device the PWA is automatically opened when the respondent clicks on the notification. Once the PWA is opened all current and past signals are displayed and clicking on a signal directly opens the ESM survey page for that signal in a browser window.

Experience Sampling Method If you would like to install the Psychexp PWA and send a test notification to your device, firstly open this page on your mobile device and then click the following link to install the pwa.
Psychexp PWA

 
2 - What type of ESM can I create?

The Psychexp ESM offers the researcher a very large degree of design freedom. You can create multiple sets of questions and assign a different question set to each signal that is sent to the respondent. An ESM question set can utilize the same functionality as any other type of Psychexp project, including the ability to insert an experiment into the question flow.

For example, let's say you wish to create a survey schedule consisting of one pre-survey, then one ESM: morning and evening signal x 4 days, and one post-survey. With the Psychexp ESM, you can create the pre-survey, ESM, and post-survey question sets within a single project, which means your respondent only needs to register once to participate in all surveys. It is also possible to create different sets of questions for the morning and evening surveys, or to display a different set of questions for every signal sent.

Add New Project 3
3 - Add New Project

Creating an esm is as simple as clicking "Add New Project", entering a name for your project and selecting "ESM" from the project type list.

Once you have created your project you can edit it in the same way as you would any other type of Psychexp project. For full details of how to edit a project please refer to the "Getting Started" tutorial.

Question Set
4 - Question Set

When creating your question sets it is very important to plan them in advance.

  1. First, carefully plan the ESM survey schedule.
    • For example, a pre-survey on DAY 1, ESM survey 3 times a day on DAYs 2-5 (4 days), then a post-survey on DAY 6, etc.
  2. Next, decide on how many question sets you are going to need.
    • For example, three question sets are needed, one for the pre-survey, one for the ESM survey, and another for the post-survey, etc.
  3. Then, check to see if there is partial overlap between multiple question sets.
    • For example, the question set for the pre-survey and post-survey overlap for the most part, etc.
  4. Finally, consider in what order you would like to arrange the responses on the data set.
    • In the ESM, the answers to all question sets are combined into a single data set (CSV file).
    • On each dataset row, the answers in the same order from top to bottom as is shown on the project edit screen will be shown for each consecutive question set in order 1,2,3...
    • In each row, only the cells relevant to the question set the respondent answered in the survey will be populated with answer values, and the columns for the other question sets will be blank cells.
    • For example, in a pre-survey→ESM survey→post-survey design, we recommend assigning Question Set 1 to the ESM survey and Question Sets 2 and 3 to the pre-survey and post-survey, respectively. In the dataset, responses to Question Set 1 (ESM survey) will follow after the start date, signal order, etc., making it easier to check the status of ESM responses and process the data for multi-level analysis.

After planning out your ESM survey, proceed with the following steps to create each question set.
At the top of the edit project page, you will find the [Question set] section.

Directly under the [Question set] section, the edit screen is displayed containing the components in [Question set 1].
Here you will find a [Default block] that contains one [Single choice] question. You can edit this question and also add more blocks, questions or experiments to complete your [Question set 1].

  • In a pre-survey→ESM survey→post-survey design, we recommend creating the ESM survey in [Question Set 1].

Clicking the [ADD QUESTION SET] button will create the new [Question set 2].
Create the required blocks, questions and experiments in [Question set 2] in the same way as the previous step.

  • In a pre-survey→ESM survey→post-survey design, we recommend creating the pre-survey in [Question Set 2].

Clicking on the [DUPLICATE THIS QUESTION SET] button will copy the contents of the question set currently displayed into a new question set.
For example, duplicating [Question set 2] will create [Question set 3] with exactly the same content.

  • In a pre-survey→ESM survey→post-survey design, If there is any overlap in the content of the pre-survey and post-survey, then duplicating [Question set 2] to create [Question set 3] will save you a lot of time.

Clicking the [DELETE THIS QUESTION SET] button will delete the entire question set that is currently displayed.
Once a question set has been deleted it cannot be restored. If there is currently only one question set, or if the question set you are trying to delete is assigned to a signal on the settings page, then an error message will be displayed and the question set is unable to be deleted.

Signal Settings
5 - Signal Settings

Once you have created your question sets, you can then create your ESM signals on the PROJECT SETTINGS page. The method of setting the signals varies depending on the type of ESM.
The following is a step-by-step explanation.

SET SIGNALS BY DAY NUMBER

In the [SET SIGNALS BY DAY NUMBER] ESM, for each signal, the date (DAY#), send time, and question set number needs to be set.

Before setting the signals, carefully plan a research schedule.
Referring to the chart below, make a diagram that reflects the research schedule that you have planned.

Tips for creating your research schedule.

  • The signal numbers (signal 1, 2, 3...) are arranged in the same order as they are sent.
  • Set the date (DAY#), send time, and question set number for each signal.
  • It is acceptable to have DAY# with no signal. Refer to: [DAY 3] in the diagram below.
  • The send time (Time 1, 2, 3...) does not need to be in the same order as they will be sent.
  • You can set signals to be sent at fixed times. (Refer to: [Time 1], [Time 3] shown in red on the diagram below.)
  • Multiple signals set to be sent at random times can be set up within the same day. (Refer to: [Time 2] shown in blue on the diagram below.)

SET SIGNALS BY DAY NUMBER

Please pay particular attention to the timing of when the first signal will be sent.

  • If the SIGNAL START DATE is set, the start date for sending the signals is calculated from DAY 1; if a signal is set for DAY 1, all respondents will receive their first signal on the SIGNAL START DATE.
  • If the SIGNAL START DATE is not set, DAY 1 is the day after the respondent has completed their registration. If a signal is set for DAY 1, each respondent will receive their first signal the day after they complete their registration.
  • If no signal is set for DAY 1 and the first signal is set for DAY 2 or later, the day after the SIGNAL START DATE / REGISTRATION COMPLETED DATE is counted as DAY 1, and the first signal will be sent when the first set DAY number is reached.

    Example: Signal #1 set to DAY 1
    If SIGNAL START DATE is set to 1st of January, the first signal will be sent on the 1st of January.
    If SIGNAL START DATE is not set, and respondent completed registration on 1st of January, the first signal would be sent the day after registration on the 2nd of January.

Once you have finished planning your research schedule, you can begin to set the signals in accordance with the plan.

  1. Firstly, create the sending times.

    Create the required number of sending times in accordance with your research plan.
    The diagram below shows the send times created in accordance with the schedule in the [SET SIGNALS BY DAY NUMBER] ESM example diagram shown above.

    • A FIXED TIME signal is created by only setting a time value in the FROM TIME field (UNTIL TIME: 00:00). Refer to: [Time 1] and [Time 3] in the diagram below.
    • Setting both a FROM TIME and an UNTIL TIME value will create a RANDOM TIME signal between the FROM TIME and UNTIL TIME. Refer to: [Time 2] in the diagram below.
    • If multiple signals are set to use the same RANDOM TIME setting, the INTERVAL value represents the minimum time that must elapse between any two times randomly created (between the FROM TIME and UNTIL TIME). The INTERVAL time is ignored if only one signal is set to use the RANDOM TIME, so in this case for clarity setting the INTERVAL to zero is advised.
    • The REMINDER and EXPIRE field can be set for both a FIXED TIME and RANDOM TIME.
    • REMINDER: A second signal can be sent to respondents who have not yet accessed the survey URL a designated amount of time after the first signal is sent. This is called a REMINDER. The content of this REMINDER message is exactly the same as the first signal sent. THE REMINDER setting can be used to prevent respondents from forgetting to respond, and can effectively increase the response rate.
    • EXPIRE: Access to the survey questions shown when a respondent clicks the signal URL can be blocked after a set period of time from when the first signal was sent (not the REMINDER signal). For example, if you set the EXPIRE time to 30 and the first signal is sent at 9:05, the respondent will have until 9:35 to access the URL in the message and start answering. After that time, it will expire and the survey questions will not be displayed. Instead the text set in the BASIC SETTINGS section EXPIRED SURVEY field will be displayed. (example: Sorry, this survey link has expired).
    SET SIGNALS BY DAY NUMBER
  2. Next, create the signals.

    Create and edit the settings for the required number of signals in accordance with your research plan.

    • Prior to creating these signals, the required number of question sets need to be created on the [Edit Project] page.
    • It is recommended to complete the settings in each signal before creating the next one. For example, complete the settings of #1 as in the below diagram and then after clicking the [SAVE CHANGES] button, create #2 signal by clicking the [ADD SIGNAL] button. Keep adding signals in the order of ADD - EDIT SETTINGS - SAVE. If you just add all the signals first and then return to #1 to edit the settings, error messages will begin to appear in relation to the signal time being earlier than the previous signal and this will cause the order of the signal numbers to be wrong.
    • If other errors occcur, the signal that is causing the error will be displayed in one of the following colors. RED: The timing settings are too strict. You can fix this by decreasing the number of signals, or by decreasing the interval between signals. You can also adjust the time window in order to increase the length of time between the from time and until time. BLUE: The random time span overlaps with another random time span on the same day. GREEN: There are two identical fixed times on the same day.
    • When you are adding and editing your signals it is recommended that you set a [SIGNAL START DATE] in the Advanced Settings section. By doing this, the date the signal will be sent is displayed, which makes it easier to check if the timing that the signal should be sent is correct or not. If you do not require a [SIGNAL START DATE] for your project, then delete the [SIGNAL START DATE] once you have finished creating all of your signals.
    SET SIGNALS BY DAY NUMBER
  3. Finally, click the TEST SIGNALS button to check your signals.

    Once you have set all your signals, it is time to check if they match the research plan you created.
    Click the TEST SIGNALS button located directly beneath the section where you made your times and signals. Signal times for each day will be displayed in accordance with the settings you created.

    • Signals shown in red are fixed time signals.
    • Signals shown in blue are random time signals. The time of the random signals will change each time you click the TEST SIGNALS button. Press the button several times to check that the random distribution is as you expected.
    SET SIGNALS BY DAY NUMBER

SET SIGNALS BY WEEKDAY

In the [SET SIGNALS BY WEEKDAY] ESM, for each signal, the day of the week, send time, and question set number needs to be set.

Before setting the signals, carefully plan a research schedule.
Referring to the chart below, make a diagram that reflects the research schedule that you have planned.

Tips for creating your research schedule.

  • Make a schedule diagram starting from Sunday when using [SET SIGNALS BY WEEKDAY]. This is because in the signal setting section, the system is designed so that the week starts from Sunday.
  • Even if you want to start sending signals from a day other than Sunday (example: Sending in the following order, Mon, Tues, Wed, Thur, Fri, Sat, Sun), you set a weeks worth of signals in the signal setting section starting from Sunday. Then if you set the [SIGNAL START DATE] to a date that is a Monday, a week worth of signals will be sent starting from Monday (Send order: Mon, Tues, Wed, Thur, Fri, Sat, Sun).
  • Other scheduling tips are the same as those described in the [SET SIGNALS BY DAY NUMBER] section.

SET SIGNALS BY WEEKDAY

Please pay particular attention to the timing of when the first signal will be sent.

  • If the [SIGNAL START DATE] is set, all respondents will receive their first signal on the first weekday that has a signal set on or after the [SIGNAL START DATE].
  • If the [SIGNAL START DATE] is not set, each respondent will receive their first signal on the first weekday that has a signal set on or after the day after they complete their registration.

    Example: Signal #1 set to Sunday
    If the [SIGNAL START DATE] is set to 1st of January 2024, the first signal will be sent on Monday the 1st of January and the next signal will be sent on Wednesday the 3rd of January. If the [SIGNAL START DATE] is not set and the respondent registers on Monday the 1st of January 2024, the first signal will be sent on the first weekday that has a signal set on or after the 2nd of January 2024 (Wednesday 3rd of January).

Once you have finished planning your research schedule, you can begin to set the signals in accordance with the plan.

  1. Firstly, create the sending times.

    Create the required number of sending times in accordance with your research plan.

    • For an explanation on how to do this please refer to the [SET SIGNALS BY DAY NUMBER] section.
    SET SIGNALS BY WEEKDAY
  2. Next, create the signals.

    The diagram below shows the send times created in accordance with the schedule in the [SET SIGNALS BY WEEKDAY] ESM example diagram shown above.

    • Question set 1 is intended for weekday use, question set 2 for weeknight use and question set 3 for Saturday evening.
    • #1, 2 and 3 are assigned on Monday, #4, 5 and 6 on Wednesday, #7, 8 and 9 on Friday, and #10 on Saturday.
    • This represents the case where the [SIGNAL START DATE] is set for Monday, January 1, 2024. To the right of the # number, the date and time the signal will be sent is displayed in green letters.
    SET SIGNALS BY WEEKDAY
  3. Finally, click the TEST SIGNALS button to check your signals.

    Once you have set all your signals, it is time to check if they match the research plan you created.

    • For an explanation on how to do this please refer to the [SET SIGNALS BY DAY NUMBER] section.
    SET SIGNALS BY WEEKDAY

SET SIGNALS BY DAY OF THE MONTH

In the [SET SIGNALS BY DAY OF THE MONTH] ESM, for each signal, the day of the week, send time, and question set number needs to be set.

Before setting the signals, carefully plan a research schedule.
The procedure for making a schedule diagram is mostly the same as for [SET SIGNALS BY WEEKDAY] however, instead of a weekly cycle a monthly cycle is made.

Tips for creating your research schedule.

  • Make a schedule diagram starting from the 1st when using [SET SIGNALS BY DAY OF THE MONTH]. This is because in the signal setting section, the system is designed to start from the 1st of the month.
  • Even if you want to start sending signals on a date other than the 1st of the month (Example: First signal sent on the 15th and then signals after that are sent on the 1st and 15th each month), in the signal settings you set the first signal date to the 1st of the month. Then set the [SIGNAL START DATE] to the date you want the signals to start being sent from (Example: 15th).
  • Other scheduling tips are the same as those described in the [SET SIGNALS BY DAY NUMBER] section.
Finally, click the TEST SIGNALS button to check your signals.

Once you have set all your signals, it is time to check if they match the research plan you created.

  • For an explanation on how to do this please refer to the [SET SIGNALS BY DAY NUMBER] section.
SET SIGNALS BY WEEKDAY

SET SIGNALS EVERY [N] DAYS

In the [SET SIGNALS EVERY [N] DAYS] ESM, for each signal, the number of days cycle to wait in between signals, send time, and question set number needs to be set.

Before setting the signals, carefully plan a research schedule.
Referring to the chart below, make a diagram that reflects the research schedule that you have planned.

Tips for creating your research schedule.

  • Every signal has the same day number [N] in a [SET SIGNALS EVERY [N] DAYS] ESM. You are unable to send signals on days between the number of days [N] you set. If you set the day number to 1, then the signals you set will be sent every day.
  • Set the NUMBER OF CYCLES setting in the advanced settings section to [2] or more. (If set to [0], the cycle repeats indefinitely.)
  • In the [SET SIGNALS EVERY [N] DAYS] ESM, a signal is sent on the [SIGNAL START DATE] and then the following signals are sent on a cycle that counts [n] days from the day after each signal is sent.
  • If you do not want a signal to be sent on the [SIGNAL START DATE] of your schedule, then use [SET SIGNALS BY DAY NUMBER] instead and do not set a signal to be sent on the first day.
  • Other scheduling tips are the same as those described in the [SET SIGNALS BY DAY NUMBER] section.

SET SIGNALS EVERY [N] DAYS

Please pay particular attention to the timing of when the first signal will be sent.

  • If the [SIGNAL START DATE] is set, all respondents will receive the cycle 1 signal on the [SIGNAL START DATE]. The count for the next cycle begins on the next day and the signal is sent [N] days later. This repeats for as many cycles as is set.
  • If the [SIGNAL START DATE] is not set, each respondent will receive the cycle 1 signal on the day after they complete registration. The count for the next cycle begins on the next day and the signal is sent [N] days later. This repeats for as many cycles as is set.

    Example: If set to N=3
    If the [SIGNAL START DATE] is set to 1st of January 2024, the first set of signals (one or more signals sent during the same day) will be sent on January 1st. The next set of signals will be sent three days later, on January 4th.
    If the [SIGNAL START DATE] is not set and the respondent completes registration on 1st of January 2024, the first set of signals will be sent on January 2nd. The next set of signals will be sent three days later, on January 5th.

Once you have finished planning your research schedule, you can begin to set the signals in accordance with the plan.

  1. Firstly, create the sending times.

    Create the required number of sending times in accordance with your research plan.

    • For an explanation on how to do this please refer to the [SET SIGNALS BY DAY NUMBER] section.
  2. Next, create the signals.

    The diagram below shows the send times created in accordance with the schedule in the [SET SIGNALS EVERY [N] DAYS] ESM example diagram shown above.

    • Question set 1 is used for the morning ESM and question set 2 for the day ESM.
    • Signals 1, 2, and 3 are assigned to DAY 3.
    • This represents the case where the [SIGNAL START DATE] is set to January 1, 2024. To the right of the # number, the date the signal is sent is displayed in green letters.
    SET SIGNALS BY WEEKDAY
  3. Finally, click the TEST SIGNALS button to check your signals.

    Once you have set all your signals, it is time to check if they match the research plan you created.

    • For an explanation on how to do this please refer to the [SET SIGNALS BY DAY NUMBER] section.
    SET SIGNALS BY WEEKDAY

Editable project
6 - Editable project

With an editable project you can firstly get your respondents to sign up to the project, and then after that, at any time, as often as you wish, you can create a new survey (question set) and send all or individual respondents a signal to inform them it is time to do the survey.

Firstly let us look at the difference between an ESM and an EDITABLE RROJECT. With an esm, once a respondent registers for the project the project locks and no changes can be made to any question sets. With an EDITABLE PROJECT the project does not lock when respondents register. Instead, each question set locks when you send a signal containing that question set to any respondents.

The following example shows just one scenario of how an EDITABLE PROJECT could be used.

  1. Question set 1 is automatically created when you create your EDITABLE PROJECT.
  2. Edit question set 1 by adding more blocks, questions or experiments.
  3. Have your respondents register for the project.
  4. Send all or individual respondents a signal that stipulates question set 1 as the survey content. (question set 1 is now locked and cannot be edited) You can send all or individual respondents another signal stipulating question set 1 as the survey content at anytime if you want the respondents to respond to question set 1 multiple times.
  5. Create question set 2 by adding a new question set or by duplicating question set 1 and editing it.
  6. Send all or individual respondents a signal that stipulates question set 2 as the survey content. (question set 2 is also now locked and cannot be edited)

Once you have created your question set you can send your respondents a signal by going to the SEND SIGNAL page.

  1. Select the respondents you want to send a signal to.
  2. Select the question set that you want your respondents to complete from the drop down list.
  3. Set the date and time you want the signal to be sent. If you leave this field as YYYY-MM-DD HH:mm the signal will be sent immediately. (If you schedule a time to send the signal any respondents living in a timezone where the stipulated time has already passed will receive the signal immediately)
  4. Set a reminder value if required. (The reminder is only sent to respondents that have not clicked the original signal) Set the reminder to zero if you do not require a reminder to be sent. Caution: Any reminder signals that are sent are included in the calculation of your monthly signal send limit.
  5. Set an expire value if required. (Details about the expire value can be found above in section 5. Signal Settings > [SET SIGNALS BY DAY NUMBER])
  6. Clicking the SEND SIGNAL button will display a confirmation screen where you can check the details of the signal prior to sending.

Multiple signals can be scheduled to be sent to all or individual respondents. However, please note caution is required as once a signal has been registered it cannot be deleted and will be sent to the selected respondents.

PRO TIP: If you need to stop signals being sent to a respondent then the only way to stop them is to delete the respondent from the project on the RESPONDENTS page. If you restore the respondent any signals with a time greater than the restore time will be sent when that time is reached. (signals with a time that elapsed during the time the respondent was deleted will not be sent)

For an editable project each question set has an individual dataset. To download the data go to the DATASET page and select the question set you require from the dropdown menu, then enter your column names and save the changes before clicking the download button.