| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Single Case Experimental Design (SCED) Frequently Asked Questions for ACP Assignment | |||||||||||||||||||||||||
2 | ||||||||||||||||||||||||||
3 | Instructions | The aim of this google sheet is to provide an overview of guidance for FAQs about SCEDs, to provide an interactive resource for trainees to get answers to queries and be able to learn from previous questions that have been answered. The hope is that this document will answer most of the typical queries that arise during a SCED. It is an interactive document so new questions can be added if they are not currently included. I will recieve a notification when a new question is added to the sheet and I will aim to answer it when I can or provide a response at the next SCED research clinic. Please note that we are unable to provide responses relating to marking and whether an assignment will pass - any information provided here is offered as advice and is separate from the marking process. It is a trainees responsibility to understand and be able to explain why they have made decisions related to completing the ACP. | ||||||||||||||||||||||||
4 | ||||||||||||||||||||||||||
5 | Reminder of resources available | |||||||||||||||||||||||||
6 | Lecture slides & recordings | Available on Blackboard - please refer to these if you need a recap of design features or how to conduct the analysis | ||||||||||||||||||||||||
7 | SCED for Clinicians website | https://sced-for-clinicians.netlify.app/ | ||||||||||||||||||||||||
8 | The website contains lots of resources to support the design, conduct, analysis and reporting of SCEDs, including; | |||||||||||||||||||||||||
9 | Step-by-step analysis guides for different software | https://sced-for-clinicians.netlify.app/practice/ | ||||||||||||||||||||||||
10 | Dummy data templates and example results | https://sced-for-clinicians.netlify.app/practice/ | ||||||||||||||||||||||||
11 | Reliable change calculator & guide | https://sced-for-clinicians.netlify.app/resources/reliable-change/ | ||||||||||||||||||||||||
12 | Autocorrelation guides and SPSS templates | https://sced-for-clinicians.netlify.app/resources/auto-correlation/ | ||||||||||||||||||||||||
13 | Example results template | https://sced-for-clinicians.netlify.app/practice/ab-design/files/AB_example_results.pdf | ||||||||||||||||||||||||
14 | Reporting guidelines | https://sced-for-clinicians.netlify.app/resources/reporting-guidelines/ | ||||||||||||||||||||||||
15 | Gloassary of terms | https://sced-for-clinicians.netlify.app/resources/glossary-terms/ | ||||||||||||||||||||||||
16 | Guidance papers | https://sced-for-clinicians.netlify.app/resources/guidance-papers-on-single-case-experimental-design./ | ||||||||||||||||||||||||
17 | Published SCED examples | https://sced-for-clinicians.netlify.app/resources/published-papers-on-single-case-experimental-design./ | ||||||||||||||||||||||||
18 | *If there are any broken links or missing documents on the website, please let Mel Simmonds-Buckley & Chris Gaskell know so we can fix it | |||||||||||||||||||||||||
19 | ||||||||||||||||||||||||||
20 | # | Category | Query/question | Guidance | Resources | |||||||||||||||||||||
21 | 1 | Idiographic Measures | Do the idiographic measures need to include one of each type of measurement - frequency, intensity and duration? | No - you do not need to have one of each type of measurement. It is fine to have three intensity measures for example. Try to develop idiographic measures that capture several different areas a client wants to change, rather than have 3 measures that assess slighlty different versions of the same thing (e.g., an intensity measure and a frequency measure of low mood) to get a broader picture of change in an individuals life (unless there is a specific reason to do so). Consider which type of measurement - an intensity, frequency or duration measure - is the best way to capture change for that domain. | See lecture slides on developing idiographic measures. Refer to published SCED papers for examples of different types of idiographic measures | |||||||||||||||||||||
22 | 2 | Missing Data | How much is an acceptable amount of missing data? | It is hard to put a number on how much missing data is acceptable as there are several influencing factors, including the amount, timing and grouping of missing data points. You need to consider; i) How much is missing as a percentage of the total number of data points - A 6-month intervention with 4 weeks missing is less of an issue than an 8-week intervention with 4 weeks missing. Evaluation of missing data rates in published SCED studies found the highest reported rate was 24% - missingness greater than this value will cause considerable difficulties for analysis and make interpretation problematic. ii) Is the data missing in consecutive chunks or scattered across the time series? Individual missing data points are likely to have less of an impact as trends can still be assessed using the data from the previous and following days (and could be imputed using the average of the two adjacent data points). Chunks of missing data cause more problems as they inhibit assessment of temporal change and are harder to impute. iii) At what stage/study phase is the missing data? Data that is missing from close to the phase change points (i.e., the switch from baseline to intervention or from intervention to follow-up) has more of an impact as it makes it difficult to assess immediate changes in level of scores at the transition points and will hamper meaningful interpretation. As the baseline phase acts as a control to compare to, considerable missing data from the typically shorter baseline phase will make comparisons with the intervention phase and interpretation of intervention effectiveness challenging. | https://doi.org/10.3390/educsci11020076 See Peng & Shen (2021) article for a review of different methods for dealing with missing data in SCEDs, lists strengths and weaknesses and finishes by providing some recommendations for the most appropriate method to use, including imputation approaches depending on the amount and timing of missing data. https://doi.org/10.1177/01454455241226879 See Aydin (2024) paper for guidance on simple imputation methods for missing SCED data. | |||||||||||||||||||||
23 | 3 | Shiny apps | I am getting an error when I try to upload my data into the shiny app? | If you are getting an error when you try to upload you data, try troubleshooting the following; 1) First, check your data is formatted correctly. The shiny apps provide instructions for how the data needs to be formatted (different shiny apps have different formatting requirements). The practice packs on the website include dummy data formatted correctly for each shiny app that can be used as a template and the analysis guides also describe how to format the data correctly. 2) Second, some of the shiny apps are not equipped to handle missing data and will produce an error if there are any data points missing. You will need to consider imputation approaches (e.g., simple mean imputation) or use Excel or one of the other shiny apps to produce your visual plots if you have missing data. 3) Finally, the shiny apps have user capacities so if the app is overloaded with users it can produce an error. If you have checked points 1 and 2 and are still getting an error, leave it a while and try again to see if it works when the server is less busy. | See comment above for guidance on imputation techniques. | |||||||||||||||||||||
24 | 4 | Analysis | How can I produce the time series plots in Excel? | An Excel analysis guide is provided on the website with instructions on how to produce plots with phases, trend lines and manually edit the graphs. The advantage of Excel is that it allows the most flexibility and editing of the plots to suit your requirements. For your assignment, it does not matter whether you use Excel or the Shiny apps to produce the plots - it is up to you. | https://sced-for-clinicians.netlify.app/resources/excel-guide/ See website for Excel guide and the Dixon et al (2009) paper in the Guidance papers for instructions | |||||||||||||||||||||
25 | 5 | Nomothetic Measures | What is an appropriate nomothetic measure and how many do I need? | Ensure you select a nomothetic measure that has been validated and has published psychometric properties - such as validity and reliability indices and published norms or available information about the standard deviation of the mean baseline severity for a clinical sample. This will enable you to analyse whether reliable change criteria are met (see query below). Use of a measure that has not been validated or have sufficient psychometric properties are unreliable and will be open to criticism. You need at least one nomothetic measure completed at the start of baseline (pre) and at the end of the intervention (post) as a minimum. It can be completed more regularly. Select a measure that maps onto the client presentation. In addition, it can also be good practice to consider more general measures which assess more general functioning/distress or quality of life. Your choice of measure may be guided by service requirements for measuring outcomes. | See lecture slides on nomothetic measures. Search for an available psychometric validation paper for your selected measure to ensure the nomothetic analyses will be possible. | |||||||||||||||||||||
26 | 6 | Nomothetic Analysis | How do I work out the whether there has been reliable change on the nomothetic measure if there is not a reported reliable change index (RCI) value? | If a nomothetic measure does not have an established RCI value that indicates the degree of change (pre-post) that exceeds measurement error (called reliable change), you can calulate this value using the psychometric properties of the measure. You can use a Cronbach's alpha value for internal consistency of the measure and the standard deviation of the mean baseline severity for a clinical sample/published norms to calculate the RCI value using the provided calculators (see resources box). These values should be available in the development paper for the measure used or a psychometric validation paper that has validated the reliability and validity of the measure. Clinically significant change can be assessed by evaluating whether scores have moved from above to below a reported clinical cut-off or threshold. | https://www.psyctc.org/stats/rcsc1.htm Online calculator for reliable change criterion. https://sced-for-clinicians.netlify.app/resources/reliable-change/ Or see RCI excel calculator and accomanying guide on the website. | |||||||||||||||||||||
27 | 7 | Design | The client had a break from the intervention (e.g. might be due to holiday/therapist absence/childcare issues or another reason) and they continued to collect the idiographic measures during this period. Would this count as an ABAB design as there was a treatment removal component? | For there to be a phase that is considered a treatment removal component, then the withdrawal phase needs to have been agreed in advance (e.g., that a defined period of time without treatment would occur before it happened, rather than applying it retrospectively). If you and the client both agreed that treatment was going to be put on hold for a set timeframe and planned to continue collecting measures during that period with an agreed time to reintroduce the intervention, then you could build a rationale for the design being an ABAB. However, if it was determined on a week-by-week basis that the client was unable to make the session and the break in treatment ended up being a period of time that neither you or the client were planning on having as a break from treatment, then it is not a design feature and it would have to be considered a gap in sessions during the treatment phase. If a client has to have a break from treatment (e.g. for a holiday), try to encourage them to continue completing the measures if they are able. | See published example of an ABAB design for an example of how to report and analyse a multi-phase SCED design (see query below). | |||||||||||||||||||||
28 | 8 | Baseline trend analysis | Why are there two Tau values and how are they different? How do I know if I need to adjust for a baseline trend when reporting Tau? | Analsysis using Tau has 2 phases - 1) test whether there is a significant baseline trend by analysing the Baseline Tau value for the baseline data points only and 2) then compute an intervention effect size to show the comparison of data in the baseline phase with the intervention phase (how big is the difference). There are two versions of the Tau intervention effect size - one that corrects for a baseline trend and one that does not make any correction - which one you report will be based on whether the Tau statistic in the first phase of analysis was significant and indicated a baseline trend that needed to be corrected for. The test to assess whether there is a significant trend in the baseline phase (using Baseline Tau and the associated p value) only uses the data in the baseline - report the Tau value and the associated p value (e.g., 0.625, p=.004) as the test of whether there was a significant baseline trend. If there is a significant baseline trend (p<.05), then you will need to use the baseline corrected Tau effect size for the assessment of the intervention effect - the comparison of data in the baseline versus intervention phase(s). The larger the Tau effect size for the intervention effect (based on comparison of the baseline and intervention data), the bigger the difference in scores in the two phases. The p value for the comparison Tau effect size is a test of whether the Tau intervention effect size is significantly different from zero (i.e. no difference between phases). | Use the online Tau calculator to compute Baseline Tau and the Intervention Tau effect size. https://ktarlow.com/stats/tau/ See lecture slides on Baseline corrected Tau and paper by Tarlow (2017) for more information. https://doi-org.sheffield.idm.oclc.org/10.1177/0145445516676750 | |||||||||||||||||||||
29 | 9 | Analysis | If I have more than two phases (e.g., any design other than an AB) how do approach the calculation of Tau to assess the baseline trend and which phases should I compare using the non-overlap effect sizes? | When there are multiple phases there are more potential comparisons that could be done, which can become overwhelming, so you don't necessarily need to compare every phase to all other phases. It is recommended to select comparisons which relate to the hypotheses you want to test - what are the research questions that relate to each phase? i.e., expect improvement in symptoms during the B1 treatment phase compared to A1 baseline phase. Think about what you are expecting to happen in the second treatment phase (i.e., continued improvement in symptoms) or follow-up phase (i.e., maintenance of effect)and what phase it makes most sense to compare that to and make an appropriate hypothesis. It would be helpful in your method analysis section to explain which phase comparisons you have done and the rationale why. The test to assess whether there is a significant trend in the baseline phase (using Tau and the associated p value) only uses the data in the baseline. So if you have more than two phases you only have to assess the baseline trend once - regardless of which other phase you are comparing the baseline to as it will always be the same (as it will be based on the baseline data only). If there is a significant baseline trend, then all the tau effect sizes for the phase comparisons comparing the baseline to one of the other phases will need to use the baseline adjusted tau effect size (the second tau test for the magnitude of the difference in data points in one phase compared to another phase). | See published example of an ABAB design for an example of how to report and analyse a multi-phase SCED design (see query below). | |||||||||||||||||||||
30 | 10 | Analysis | How do I analyse data from an idiographic measure that is based on week days only (i.e., when the response is only possible on a week day - did they attend work/school)? | When the measure relates to a response that is tied to a particular day of the week such as a work or school day, the data from weekends/non-work days is forced to be a particular response - as the person cannot attend work/school at the weekends. This will mean for non-overlap effect sizes, the amount of overlap between data points in the baseline and intervention phases will be inflated because you will have 2 points out of 7 that will always overlap (as they will all be the same response). As the reason for these data points being rated as they are is not related to the treatment and cannot be changed by the intervention, they should not be considered in analyses used to evaluate the intervention. Only include data for work/school days in these analyses and (i.e., don't include weekends so there are only data points for the days where there is potential for different scores). Explain the justification for this approach in the method and explain the data isn't missing data, rather uncollected data. For the time series plots, you can either take the same approach (this will mean you will have a different number of data points compared to other idiographic measures that use the full 7 days) or leave the weekends as blank in the plot (you can do this in Excel) and explain in a figure footnote that the blank sections are the weekends. The second approach will allow you to look at the trend over each week with the weekends being visible to see if there are any overall changes or trends during the intervention. | https://sced-for-clinicians.netlify.app/resources/excel-guide/ See website for Excel guide and the Dixon et al (2009) paper in the Guidance papers for instructions | |||||||||||||||||||||
31 | 11 | missing data | If a client takes a break from the intervention to go away (3 weeks) and does not complete the ideographics whilst on holiday does that mean that the data can not be used? or is it okay to have a break? | It doesn’t mean the data definitely can’t be used, as it will depend on the missing data considerations outlined in the answer to question 2 (above). The assignment guidance states that the intervention phase needs to be at least 28 data points, so if three weeks of that was missing in a chunk that would make any meaningful interpretation impossible and the data would not be usable. However, if the intervention is phase is longer then the impact becomes less – consider the points outlined in the response to question #2 in terms of the amount of missingness overall and the timing of the missing data. If the data is used with the chunk missing, it is also worth considering that there will need to be enough time series data points after the missing chunk to identify a trend in scores and extrapolate from the trends from before to after the missing chunk of data. Relatively few data points after the missing chunk will reduce the confidence of any observations as it will be difficult to assess whether there has been a consistent pattern of scores. If you do use the data, you will need to appropriately acknowledge the limitation and the gap and provide an appropriate rationale/justification for this, so the marker understands. | See response to question 2 for more information about dealing with missing data. | |||||||||||||||||||||
32 | 12 | Missing data | When using Excel do I just leave the missing data points blank or do I have to somehow let Excel know that they are missing? Similarly, in the TAU calculator for AB designs (ktarlow.com) do I just enter a gap or do I somehow mark the missing data point with an "x" or "n/a"? Thank you! | How you format missing data will depend on how you plan to handle it and which software you are using. Missing data points will be blank for that time point in excel, but for Shiny apps that cannot handle missing data you may need to consider an imputation approach to fill in these gaps. If you play around with the different calculators trying inputs with and without missing data points, you will be able to establish how it handles missing data points and what input is required. Does it return an error? - if so then imputation or removal of blanks may be required. If you try inputting the data into the KTarlow baseline Tau calculator with gaps for missing data and without gaps - it returns the same output so you can establish that inclusion of blanks does not affect the input. | ||||||||||||||||||||||
33 | 13 | Idiographic measures | With reference to the first question on this list: It states it is fine to have three measurements of one type, e.g. intensity. However the handbook says: "Idiographic measures need to contain a blend of intensity, duration and frequency measures that assess the clinical concerns of the case". Please can I just confirm that it is okay to have only one type for the assignment (that type made most sense to the client)? | Idiographic measures should consider containing a blend of intensity, duration and frequency measures that assess the clinical concerns of the case - they should be co-developed with the client to measure the areas most important to the client. SCEDs which are able to get a blend of these types of measures will enable more rich data to be collected. However, the type of measure (intensity, duration, freq) will depend on the best or most appropriate way to measure that construct, so it is okay to have the type that makes sense to the client. It does not have to be one intensity, one duration and one frequency measure. There does need to be at least three idiographic measures that are heterogeneous though (assess different targets) - rather than the same construct measured with intensity, duration and frequency versions. | ||||||||||||||||||||||
34 | 14 | Wording | As AB designs are not true SCED can I is still refer to that I done a SCED in the write up or should I say that it is an empirical case-study? | AB designs are referred to as quasi-experimental SCEDs - see teaching slides section on designs for more information. | ||||||||||||||||||||||
35 | 15 | Baseline data collection | If in a service which sees clients weekly, could I use session 2 for continued assessment (i.e., seeing clients on day 7 of the baseline period), and start intervention during session 3 (therefore allowing a 14 day baseline prior to intervention officially starting)? I'm aware that this isn't ideal, but wonder whether it would be feasible as long as I address it as a potential limitation in the discussion section of my SCED write-up? Thank you! | Yes, that is fine. If you have a 14 day baseline period, with any sessions with the client during that time focused on assessment and introduce the intervention at the 3rd session that is a typical approach seen in psychotherapy SCEDs. As with most psychotherapy SCEDs, the limitation is whether you can truly have a neutral baseline and whether just contact with a clinician for assessment, even if they are not introducing the change focused element of the intervention, could have a beneficial effect. | ||||||||||||||||||||||
36 | 16 | Idiographic measures which can't be collected everyday | With reference to question 10 above, does a measure which can't be collected everyday still count as one of the three idiographic measures, or would it have to be in addition to three idiographic measures that can be completed everyday? | It can be considered one of the idiographic measures, but be mindful of how days where you are unable to collect data might affect the minimum number of data points you are able to collect for the baseline period. If in doubt, you could collect it as an additional measure to 3 other idiographic measures. | ||||||||||||||||||||||
37 | 17 | Autocorrelation | When we assess autocorrelation, am I correct in thinking we do this for each ID measure and do this for baseline and intervention for each? Also, when report these do we just report the first lag or do we report all lags? In the latest powerpoint it seems to imply just the first lag but I am not sure. If we are expected to report all lags then do we report all these with the autocorrelation coefficent and pvalue? Thank you | Autocorrelation should be assessed in each idiographic measure. This should be assessed separately for each phase (baseline, intervention and any additional phase). The instructions provided for assessing autocorrelation in SPSS explains how to do this and split across phases. At the very least autocorrelation for the 1st lag should be reported. There are no set recommendations about how many lags should be reported - SPSS analysis settings assess 16 lags as the default but this is an arbitary number. The number of lags that are relevant will be related to how many data points are included in each phase - more lags may be more applicable to longer timeseries. For example assessing 16 lags will mean 2 lags are redundant in a 14 day baseline (as there are only 14 lags available). In the text in the results section of the write up, you only need to report where there are significant (p<0.05) lags (which measure, which phase) and how many (number of lags), but you do not need to report the coefficients or exact p value - you could include the SPSS output containing this information in the appendicies. | ||||||||||||||||||||||
38 | 18 | Tau | I understand if there is no baseline trend you don't report the corrected Tau (Tau-U). You just report the Tau A vs B (the one which isn't corrected). However, does this tau still hold more weight over the other non overlap effect sizes, although it has not needed to account for baseline trend as there wasnt one. Just thinking in the context of wording in the assignment and strenghts and weaknesses of the analyses. If there is no baseline trend for an ID measure and Tau Avs B is non-significant showing no effect of intervention whereas the other non-overlap effect sizes are showing some varied effectiveness, would it be correct to still say Tau is more robust? Thank you | The other non-overlap effect sizes are influenced by different factors including the length of the baseline, length of the intervention phase etc. so the strenghts and weaknesses of different non-overlap statistics will need to be considered in the context of your design. Consult the lecture slides and the referenced papers to learn more about which non-overlap effect size may be best for you context. However, the family of Tau effect sizes are generally considered more robust as 1) they are less influenced than the other non-overlaps by the presence of autocorrelation, 2) are less impacted by number of data points in the phases and 3) are able to adjust for baseline trends when they are present. | See Manolov & Solanas (2017), Manolov & Moeyaert (2017) and Soloman (2014) for recommendations for selecting analytical method and the advantages/disadvantages of different approaches | |||||||||||||||||||||
39 | 19 | Tau | On the example table 2 statical analysis in the PowerPoint / teaching, it mentioned 1If baseline trend is not significant, Tau between phase effect size is reported (τAvsB). 2If baseline trend is not significant, Tau-U between phase effect size is reported (τAvsB-trendA). Which on is right? | Tau-U or Baseline-corrected Tau effect size for the A vs B phase comparison is reported if the baseline trend is significant. | ||||||||||||||||||||||
40 | 20 | Method | Do we need to incude an analysis section under the method section? For example Autocorrelation was used to.. Tau was used to.. and do on? | It is recommended to include a section on the Data Analysis approach used in the Method. Consult the SCRIBE reporting guidelines for detailed information on how to report the study and what sections to include. | See SCRIBE reporting guidelines. https://sced-for-clinicians.netlify.app/resources/reporting-guidelines/ | |||||||||||||||||||||
41 | 21 | Figure | For the visual graphs for the ideographic measures, can I use all three graphs as one figure (e.g., Figure 1) or do the need to be separated figures (e.g., Figure 1, Figure 2, Figure 3)? | It is helpful to separate them out into individual figures for each idiographic measure so you can explcitiy refer to each one in the text and it is clear which measure you are referring to. | See published examples of SCEDs for examples of how to report results. | |||||||||||||||||||||
42 | 22 | Autocorrelation | For autocorrelation if all lags are significant for one ideographic measure, do I need to report all autocorrelation coefficient and p values in the text (e.g., significant first order (lag 1) autocorrelation present (autocorrelation coefficient = .365, p=.014)? For one of my idiographic measure all 16 lags are sig. Can I just refer to a screen shot of the SPSS output in the appendix? Or should I create a table and refer to that? | You don't need to include all the autocorrelation coefficients and p values for all lags in the main text write-up. In the results section you can just report how many and which lags were significant (p<.05) in each phase. You can include a copy of the SPSS output for the autocorrelation results in the appendix with the exact autocorrelation coefficients and significance values. | ||||||||||||||||||||||
43 | 23 | Control measures | In the handbook it says we can use a control idiographic measure to show that the intervention is working, do we need to include a control measure or is this optional? | Use of a control measure is not a requirement, it is an optional addition. | ||||||||||||||||||||||
44 | 24 | Baseline data collection | I'm aware of Q15, but as I see clients biweekly and sessions can be slow, it will likely be the 3rd session when we have everything set up for the SCED and intervention. I am quite worried that it won't be much of a neutral baseline, as in the current setting just being able to air out worries and have normalising for what someone is going through can have quite an impact. Would I just be able to reflect on this when writing up the SCED or would you advise not carrying on with it? I'm also aware that we may not finish before the 4 week gap, so there would be a break in treatment. Would it be okay to plan this in and have a ABAB design? | You need to have a discernable change in session content from the baseline phase to the intervention phase to be able to test out your research question - i.e., are you are able to separate out and indicate a shift in the focus of your work to more direct change work when you start the intervention. Ifwhen you analyse the data and it appears that there started to be improvement in the baseline before the intervention work started, you can consider explanations for this in the discussion where you could discuss whether activites/interactions that took place as part of the baseline work may have had a therapeutic benefit and produced improvements. You could also compare baseline improvents across the idiogrpahic measures - improvement in all measures may suggest there was a general impact of contact with a professional, whereas improing baseline trends in some but not all idiographic measures could encourage reflections about why or what was happening in those baseline sessions that contributed to specifc improvements. Regarding the second question about an ABAB design, I am not sure what the finishing before the 4 week gap refers to so it is hard to answer. General advice without knowing the specifics would be if you plan in a period without the intervention and the client is aware of this in advance and the context and rationale for doing this then this could be set up as an ABAB design. You would need to ensure the client continues to collect the idiographic measures during the withdrawal/2nd A phase. | ||||||||||||||||||||||
45 | 25 | Treatment phase observations | In the handbook, it says the intervention phase needs to contain at least 28 observations. Can I check, does this refer to the number of data points rather than the number of days the intervention spans? So for example, if the intervention phase lasted 28 days but there is missing data (meaning there are around 20 data points for example), the data wouldn't be usable for the purposes of the SCED assignment? | You need 28 data points, i.e. a minimum period of time over which data was attempted to be collected during the intervention phase. If there is a small amount of missing data this could include imputed data points (using either a simple or more complex imputation method). When there is missing data for a short intervention period I would strongly consider using an imputation approach. However, it is worth considering the impact of increasing amounts of missing data as this will have a bigger impact when the intervention period is short - consider answer to Q2 for considerations about how much missing data is viable. | ||||||||||||||||||||||
46 | 26 | Idiographic measures | I have a client with depression and we identified engagement with number of self-care tasks and then score (/10) of mood and score (/10) with confidence - do these sound feasible to be written up as SCED? | If these map onto the core issues for the client and they feel they are able to be measure them using the scales you have defined then they sound approproate. It is not clear how you plan to measure engagement with a number of self-care tasks. Measures that are binary (e.g. only 2 possible responses such as yes or no) are more challenging to analyse so bear this is mind. | ||||||||||||||||||||||
47 | 27 | Ideographic Measures | I have a 6 year old patient and I plan to set up ideographic measures for parents to answer daily. But I was wondering if I could get the patient to answer a weekly ideographic measure within our weekly sessions e.g. a smily face scale for mood. Would I be able to use this within the SCED analysis as it isn't a daily measure? | You could still do this to include the clients voice alongside the parents daily measures - it would be a nice addition. But it would not meet the requirements for one of the 3 idiographic daily measures so set it up as an additional measure you collect and ensure the parents complete 3 daily measures to meet the requirements of the assignment. The childs weekly measure could be plotted weekly to assess their reported change alongside the full daily measures. | ||||||||||||||||||||||
48 | 28 | Ideographic Measures | I set up 3 ideographic measures at assessment, and they have completed them however they have misunderstood one of the measures and this has led to me having to change the scale of one of the measures for them to understand it fully and be able to complete it. Am i able to still use the data points already collected and just reflect on the fact that I had to alter one of the measures because of the clients ability/understanding? As I wont be able to go back and collect the assessment period again. Hope this makes sense. | If the client misunderstood the planned scale, but completed the data using a consistent scale based on their interpetation throughout (i.e., the data collected during the baseline and intervention periods are on the same scale and are therefore comparable), then I think this could still be usable with appropirate discussion of the limiations of that measure. However, if the scale of the measure has been changed part way through so that it is different between baseline and intervention and means you are unable to directly and reliably compare the data across phases then this measure will not be usable (e.g., if it has changed from a 10 point scale to a 5 point scale a score of 5 is not directly comparable and means different things depending on what scale it was completed for). | ||||||||||||||||||||||
49 | 29 | Tau | When considering the strength of the effect size of Tau do we base it on Cohen's d as it is a correlation coefficient or do we base it on Scruggs & Mastroppieri's interpretations like with the PEM, NAP and PND? | If you have used the newer version of Tau/Tau-baseline corrected as caluclated by this calculator http://ktarlow.com/stats/tau/ then it is calculated using Kendall's rank correlation so is constrained to be between -1 and +1 (like a traditional correlation value) and can be interpreted like a correlation. Note that interpretation of correlations are based on Cohen's (1988, 1992) criteria for Pearson's r effect sizes, NOT Cohen's D effect sizes which are not correlations - small, medium, and large effects are .10, .30, and .50, respectively, for Pearson's r. PLEASE NOTE: This interpretation does not apply to the older version of Tau/Tau-U as calculated here http://singlecaseresearch.org/calculators/Tau-U Tau/Tau-U variant is not constrained to be within -1 and +1 so cannot be interpreted like a correlation. The baseline corrected version of Tau was developed as an updated version of Tau to solve this problem of Tau/Tau-U. Scruggs & Mastrippieri's criteria apply to the non-overlap effect sizes that are expressed as percentages. | ||||||||||||||||||||||
50 | 30 | visual analysis | I have got my visual graphs with trend lines for each phase but I wanted to create the trended range lines due to variability - I can find the split middle method but I can't see how this works for trended range? Do you have any guides for this? | Trended range lines in each phase visually depict the range of scores in each phase while taking into account trends. The SCDA shiny app has this option in the visual analysis tab (https://tamalkd.shinyapps.io/scda/ Plot estimate of variability -> select 'Trended range' as the measure of variability). Alternatively, it is relatively simple to add manually to an Excel plot - see this paper (https://doi.org/10.1027/1614-2241/a000042) for how the trended lines can be drawn by plotting 4 points in each phase (pg 108). You can then insert a line to the excel plot and position it at the calculated points to create the trended range lines. See Box 1 in the guide in the adjacent cell for how to create plots in Excel and add lines to plots. Also, if you are concerned that high variability may be having an impact on linear trend lines, then you could use trend line stability or SD bands as a way of assessing the variability of the data and an empirical approach to determining how well trend lines represent the data. It doesn't necessarily have to be included as a visual aid on a plot (it may make the plot too messy if you try to add too many visual aids), you could report the percentage of data in each phase that falls within a trend stability envelope band. The manolov overlap shiny app has options to calculate this - https://manolov.shinyapps.io/Overlap/. The WWC visual two phases tab has a trend stability envelope plot and you can use the settings on the left hand side to determine within what percentage of the median data points need to be (i.e. where the envelope bands are drawn using the sliding scale option) - the plot then reports the percentage of data within the bands in each phase below it. | https://sced-for-clinicians.netlify.app/resources/excel-guide/ See website for Excel guide and the Dixon et al (2009) paper in the Guidance papers for instructions | |||||||||||||||||||||
51 | 31 | Tau | In my data analysis section I have described that I would be calculating Tau and Tau-U using Tarlow's Baseline Corrected Calculator - firstly can I check Tau-U is something to be calculated from this calculator? Secondly, my output on the calculator said to use the baseline corrected effect size - so does this mean I am now using Baseline Corrected Tau instead of Tau-U? and therefore should I change my data analysis section to 'baseline corrected tau' instead of 'tau-u'? | Tau/Tau-U (where Tau-U is the version that removes effects of baseline trends if they are present) is the older version of Tau is calculated using this calculator (http://singlecaseresearch.org/calculators/Tau-U) or the Tau-U effect size option in this calculator (https://jepusto.shinyapps.io/SCD-effect-sizes/). Tau/Tau-Baseline corrected (where Tau-BC is the version that removes effects of baseline trends if they are present) is a newer version of Tau that solves some of the limitations of Tau/Tau-U. It is calculated using this calculator http://ktarlow.com/stats/tau/ or the Tau-BC effect size option in this calculator (https://jepusto.shinyapps.io/SCD-effect-sizes/). | ||||||||||||||||||||||
52 | 32 | Design | Is ABC (no F/U) a true SCED or quasi-experimental? | ABC design where A is baseline, B is Treatment 1 and C is a (different) Treatment 2 is an alternating treatment design. As the design has two manipulations (i. change from baseline to treatment 1 and ii. change from treatment 1 to treatment 2) it is one of the designs that is considered a true experimental design. | ||||||||||||||||||||||
53 | 33 | Statistical analysis | I'm struggling to interpret my results. Tau suggests a significant deterioration (which supports the visual analysis) and PEM is 25% but PAND is coming out at 78%. I have ensured the direction is correct and it doesn't seem to change the statistic on the shiny app whether I set it to 'increase' or 'decrease'. Logically, the 78% seems to be an inverse (i.e. should be 22%) but I'm not sure if I can just flip it round or how I can justify/explain this in my write up. | Without seeing the data and the plots it is tricky to make exact recommendations. But it is worth noting that when there are lots of data points there are often multiple solutions to PAND so it is tricky to know which data points and from which phase the shiny app has removed to compute the value. If the baseline has all the same values this may explain why it doesn't change whether you set the direction of improvement as increase or decrease as the same number of data points would need to removed to avoid any overlap regardless of direction. Make sure that the comparisons you are using to compute PAND are based on 2 phases only. If you have more than 2 phases, you need to compare two of the phases at a time (i.e., A versus B, B versus C etc) in separate comparisons. The analysis software will not be set up to handle 3 phases, it will assume there are two phases so you will need only include the data for two phases at a time. The https://tamalkd.shinyapps.io/scda/ shiny app effect sizes will be wrong if there are more than 2 phases (see teaching slides - practical exercise for statistical analysis that refers to this). The simplest solution if you are unsure and can't resolve it is to just report a different non-overlap statistic - you need to report 3 so if you have Tau, you could just pick tow of the others to report instead. | ||||||||||||||||||||||
54 | 34 | Autocorrelation | I am really struggling to understand the implications of autocorrelation - all of my baselines are not significant at any lag. However 2 of my idiographic measures in the intervention phase are significant across all lags. I know I need to state this, but I do not really understand what the implications of serial dependency means. Does this impact other analyses i choose when looking at effect size? I think I need a dummies guide to explain autocorrelation/serial dependency and what it means when im interpretting my results (other than be cautious). Please help me understand this. UPDATE: Thank you for the response. It was really helpful. | The simplest way to view it is that when there is autocorrelation present in the data the common assumptions of analysis tests are violated. This means the results from those tests might be biased or inaccurate and lead to inaccurate conclusions about the effectiveness of the intervention. Some of the statistical analysis effect sizes/tests are more influenced by the presence of autocorrelation than others - see indicated paper for an overiew. Tau for example is considered more robust to the presence of autocorrelation than many of the non-overlap effect sizes. Higher levels of autocorrelation (measured on a -1 to +1 scale) between datapoints means it is more likely that treatment effect sizes will be overestimated. If autocorrelation is present in your data, your interpretations of the findings need to consider if any of the effect sizes you have used are susceptible to bias as a result. You can highlight caution when interpretating findings from effect sizes that are known to be more affected by autocorrelation and place more weight in findings from tests that are more robust to autocorrelation (e.g. Tau). This is most important when different effect sizes have different findings regarding the effectiveness of the intervention - influence of autocorrelation may be one reason why they have found different results. | Paper outlining the impact of autocorrelation on different SCED effect sizes. https://doi.org/10.1002/bin.1783 | |||||||||||||||||||||
55 | 35 | Consent | Where are the example consent forms that can be used for assignments? | Consent form templates can be found on Blackboard - in Organisations -> PSYR09 Clinical Psychology -> Trainee Information Pack. | https://vle.shef.ac.uk/ultra/organizations/_13914_1/cl/outline | |||||||||||||||||||||
56 | 36 | Write Up | How detailed should the write-up be for the intervention procedure section? The published examples are usually around a paragraph and some include a table detailing the sessions. Would following this example be OK, or should it be more detailed, similar to the way we are encouraged to write clinical practice reports? (Sorry if this is a silly question, but generally speaking, is the style of writing for a SCED quite different from writing a Clinical Practise Report?) | You can use the published papers as an example. You need to include a breif description of the intervention (e.g. number/duration of sessions and the protocol used- if available) so the reader can understand what was done. Incluiding a table detailing each session is a useful way of illustrating the work you did with the client. The SCED needs to be written in the style of an academic paper (like if you were to submit it for publication) so needs to include an introduction, method, results and discussion sections, follow APA formatting and be written in more formal, academic language with critical evaluation and reference to the evidence base and appropriate literature. The teaching slides include detailed description of what you need to include in each of these sections and use the published examples as templates for writing in an academic style. | ||||||||||||||||||||||
57 | 37 | Generalisation Measure | I have finished collecting data for my SCED. For various reasons, the client completed 4 nomothetic measurements (of which have good psychometric properties). Unfortunately, I have only just seen the guidance regarding generalisations measures. I did not use these, and wanted to check if this case was still okay to use as write up? Is the generalisation measure more a recommendation rather than necessity? Thanks | A generalisation measure is not a requirement. It is an additional option you could consider, alongside a disorder specific or general distress nomothetic measure. As stated in the coursework guidelines 'At least one valid and reliable nomothetic measure needs to be used' and this should be matched to the clients presenting problem or assess general distress. See the teaching slides for more details on the types of nomothetic measures you could use. | ||||||||||||||||||||||
58 | 38 | Ideographic Measures | One of my ideographic measures was binary / an event occurance measure. Is the visual and statistical analysis for this the same as for the others, or is there another process I should be following for binary results? | You need to complete the visual and statistical analysis for all idiographic measures as part of the assignment requirements. However, binary measures make the analysis/interpretation slightly trickier so we advise that you don't rely on them as a primary outcome, rather use them as supplementary outcomes for interpretation of non-overlap effect sizes. The time series plot will look odd as only two values were possible and binary outcomes are more likely to be influenced by floor/ceiling effects and will make non-overlap effect sizes unreliable - e.g., if the event occurs and also does not occur on days in the baseline phase, then the intervention phase will have 100% overlap (as only two scores are possible and both occured in the baseline). You need to include the 3 non-overlap effect sizes for the assignment, but for binary outcomes you could also include additional analysis that computes the proportion of days that the event occurred in the baseline compared to the intervention phase to see if there has been a change (i.e., extinction of behaviour if decrease is the desired outcome or engagement with a behaviour if increase is the desired outcome). This will better capture the impact of the intervention on a binary measure than the non-overlaps. You are more likely to also find significant autocorrelation with binary outcomes. Due to these limitations, you will need to interpret the results taking these into consideration and interpret the findings in a balanced way (i.e. don't put too much weight on the non-overlap effect sizes for that measure) and don't overemphasise conclusions based on those effect sizes. That will demonstrate appropriate knowledge and understanding of the methods you used. | ||||||||||||||||||||||
59 | 39 | Nomothetic measures | Do the clinical and non-clinical norms for the outcome measures (e.g. PHQ9 and GAD7) need to reflect the demographics of the sample (e.g. older adults)? | Ideally they need to represent the population of the client you are working with to make the most accurate comparisons. However, these may not be available depending on what that population is. If that is the case then then closest available norms will suffice and you can comment on/critique the relevance to your client in your discussion. | ||||||||||||||||||||||
60 | 40 | Protocol | Do we need to follow a specific protocol for the SCED, or will a generalised CBT approach, with other components (such as chairwork etc), be acceptable as the intervention method? | This will depend on the client you are working with and their presentation. You need to be able to outline the rationale for the intervention/protocol you have used with reference to the literature and build the argument for why you decided to use it with your particular client and test its effectiveness with a SCED design. The approach you use should therefore be guided by the literature and you should be able to back up why you selected that approach (i.e. with reference to clinical guidelines, theory of why it might target change relevant to your client, evidence of potential adaptations that might be required or evidence of the intervention with other/similar populations). | ||||||||||||||||||||||
61 | 41 | Abstract | Do we need to write an abstract for the report? | If it is not outlined in the assignment guidelines then it is not required. | ||||||||||||||||||||||
62 | 42 | Appendix | Do all tables and graphs need to go in the main text body or can they go into the appendix to save words? | Information that is solely presented in the appendices is not marked. Ensure all results required as part of the assignment guidelines are provided in the main body of the report. Information included in the appendices should be supplementary to provide more information or expand on what is included in the main report. Important information should never be solely provided in the appendices. Information can be summarised in the main text and then direct the reader to the appendix for more detailed inforamtion if they need it (e.g., sentence summarising the of amount of significant autocorrelation lags (<.05) in each phase and then signpost the reader to the appendix for the full statistics of the lags tested.) | ||||||||||||||||||||||
63 | 43 | Methodology | I have read in the handbook that data can be collected from staff, family members or the client for richness. If I am working in inpatient, is it possible that all measures are collected from staff and none from the client? The intervention was going to be a PBS plan to reduce restraint and IM medication. Or, for university requirements, does the client have to be involved at some point in the process to make it suitable for assignment purposes. | Data can be collected by staff, so it is okay if all measures are provided by staff. Ensure you provide a rationale for who collected the measures and make sure your intrerpretations are made in the context of the data provided by staff (e.g., cannot make assumptions about how the client feels if that was not measured directly). Also ensure all other assignment guidelines are met, particularly around needing to include a formulation and description of the intervention that can be mapped onto the time series data,. | ||||||||||||||||||||||
64 | 44 | Effect sizes | My three idiogprahic measures all have the same PAND effect size - it feels like something has gone wrong somewhere however I have ran the analysis multiple times and get the same result (the other tests do vary). Is there a likely way round this/ should I choose another test? | Without seeing the data it is hard to advise, but it would seem suprising that PAND would have the same effect size for all three measures as it often has multiple solutions to remove the least number of data points to remove all overlap between phases. The simplest solution would be to select another non-overlap effect size instead if you are unsure. The Tau comparison between phases counts as one non-overlap effect size, so you only need two others (these could be PEM, NAP etc.). | ||||||||||||||||||||||
65 | 45 | Outliers | What do we do with outliers in the idiographic data? | You do not have to do anything to statistically deal with outliers in the idiographic data. You would need a strong rationale for removing them. However, certain non-overlap effect sizes are more susceptible to influence from outliers (e.g. PND) so I would advise you to consider and understand which would be the best ones to select. Effect sizes that compare pairs of data points (such as NAP) and TAU are likely to be more robust. | ||||||||||||||||||||||
66 | 46 | Method | A client collecting idiographic measures wanted a treatment break during ramadan which we agreed at the start of our work in December. They have continued to collect data during this treatment break. We have agreed to meet in early April to review the progress made and repeat nomothetic outcomes. We agreed to use this check-in session to assess whether further treatment would be necessary. Would this be an AB-FU sced? Thank you | If treatment ended before the break and you are meeting to review and check in rather than resume treatment then this gap could be viewed as a follow-up phase. Think about how you can map your phases on to a research question - the baseline provides a control to compare to before treatment starts, the intervention phase is the delivery of the treatment to see if it improves outcomes and any follow-up phase is to see if any treatment benefits have been maintained/are durable. Can also see response to question 7 for additonal response around managing treatment breaks. | ||||||||||||||||||||||
67 | 47 | Assessment period | I began a sced with a client, however after the initial two week baseline period, there was a gap of three weeks before commencing the intervention due to client illness and then leave over the holiday period in december. The client contimued to collect data during this period. What effect will this have on the sced data analysis and write up? | If you have not started the treatment during this gap then it can be considered part of the baseline. You will just have an extra long baseline to compare to. | ||||||||||||||||||||||
68 | 48 | Data points | My client completed idiographic measures twice daily (morning and evening) for the three seperate measures. Should the morning and evening scores have seperate lines on the graph or be complied into one line to represent flucuations in that one measure? | If they are measuring the same concept, just twice a day rather than once a day, then you should plot it all as one line. If there is a reason the morning and evening measurements would be capturing different concepts that makes up part of the design and development of the measure, then you could do them as two different plots, but you would need to provide a clear rationale for this and still ensure there were 14 data points in the baseline phase for both plots. | ||||||||||||||||||||||
69 | Missing data | The SCED data ranges between 0 and 10 and there are a few missing data points. I have calculated the mean to input the missing data but this is 8.88 - do I round up to 9 so it looks like a feasible data point or should i keep it at 8.88? | Yes, you can round up to 9 so it is a data point on your scale. | |||||||||||||||||||||||
70 | Write up | I worked with my client for two assessment sessions, prior to starting to collect the idiographic and nomothetic measures, do I include information from these in the write up/treatment adherence table? | ||||||||||||||||||||||||
71 | Missing data | The client did not attend sessions for two weeks and did not complete measures during this time, so there are 15 data points missing. The time period for the missing data was not during a phase-change period. The intervention is also ongoing, so there will hopefully be a good amount of data collected after the missing 15 data points. Will I still be able to use this data for the SCED assignment, as long as the missing data is under the 24%? | ||||||||||||||||||||||||
72 | Missing data | I have two data sets which both have around 24% (or slightly more) missing data. For each data set there is missing data where the baseline moves to the intervention. Would this mean that I can not use either data set for the SCED, or with a data imputation method would it be okay? | ||||||||||||||||||||||||
73 | ||||||||||||||||||||||||||
74 | ||||||||||||||||||||||||||
75 | ||||||||||||||||||||||||||
76 | ||||||||||||||||||||||||||
77 | ||||||||||||||||||||||||||
78 | ||||||||||||||||||||||||||
79 | ||||||||||||||||||||||||||
80 | ||||||||||||||||||||||||||
81 | ||||||||||||||||||||||||||
82 | ||||||||||||||||||||||||||
83 | ||||||||||||||||||||||||||
84 | ||||||||||||||||||||||||||
85 | ||||||||||||||||||||||||||
86 | ||||||||||||||||||||||||||
87 | ||||||||||||||||||||||||||
88 | ||||||||||||||||||||||||||
89 | ||||||||||||||||||||||||||
90 | ||||||||||||||||||||||||||
91 | ||||||||||||||||||||||||||
92 | ||||||||||||||||||||||||||
93 | ||||||||||||||||||||||||||
94 | ||||||||||||||||||||||||||
95 | ||||||||||||||||||||||||||
96 | ||||||||||||||||||||||||||
97 | ||||||||||||||||||||||||||
98 | ||||||||||||||||||||||||||
99 | ||||||||||||||||||||||||||
100 | ||||||||||||||||||||||||||