Testing a Computerised Systematic Observation Tool in the Clinical Setting
This study found that the SMILI was feasible and acceptable for use in the intrapartum setting. The SMILI had been found to have good face and content validity and reliability when tested in a non-clinical setting, this study aimed to identify whether it was reliable, valid and feasible for use in the real world of care of women in labour.
The study was successful in recruiting participants across the four participant sites and achieved the maximum goal for numbers with forty nine complete observations analysed. Overall the great majority of women, birth partners and midwives approached gave consent to participate.
Logistical issues, relating to researcher travel and the smaller numbers of prospective participants meant that recruitment at the two smaller participating units was lower than the larger units.
The study compares favourably in size with the most comparable studies undertaken in North America. The Miltner study undertook 150 hours of observations of 24 nurses caring for 75 women, the Barnett study undertook 75 hours of observations of 17 nurses caring for 30 women.
The study demonstrated that the direct observational approach was understandable and acceptable to the majority of women, their birth partners and midwives in the clinical setting. The SMILI worked well in the clinical setting with loss of data in only three cases due to user error. The SMILI worked from an inexpensive memory stick on standard personal computer laptops without any special adaptation and so did not represent a costly method of collecting data yet it was successful in obtaining a large amount of comprehensible and usable data relating to intrapartum midwifery care.
The study further tested the SMILI for face, content and construct validity and internal reliability and found the instrument to be both valid and reliable for use in intrapartum care.
The observers rated face and content validity of the SMILI high in the clinical setting. Following some minor amendments to content after the early observations, content validity of the SMILI was found to be high by the observers using the instrument in the setting for which it was designed. The observers felt that the programme enabled them to record the support seen.
The SMILI was found to have good levels of construct validity through the identification of significant correlations between the care recorded and women's feelings expressed in the postnatal questionnaire (SCIB). Women rated the support they received more highly when they received higher quantities of emotional, informational, tangible and partner support and when the midwife was in the room more. A larger sample of observations would enable further testing of the correlations between both negative and positive behaviours and women's views. Emotional support behaviours had the strongest links with women's views, which confirms the findings of earlier studies with women which identify emotional support behaviours as the most important category of support. Construct validity of the SMILI was further supported by the findings showing no correlation between the amount of non-support behaviours and non-supportive care including assessment and women's views of the support received with women's views in the postnatal questionnaire.
Good levels of internal reliability of the SMILI were found. Different aspects of 'attitude' correlated highly with each other for the woman, her birth partner and the midwife. Where a midwife remained in close proximity to the woman, she was also likely to demonstrate a positive demeanour, use a positive vocal tone and display a positive facial expression. Where a midwife displayed a neutral or negative demeanour she was more likely to use a neutral or negative vocal tone and display a negative facial expression.
The veridicality of the methodology was demonstrated through the identified correlation between women's views of the support received and the observers' overall assessments of the support they had observed. For the most part, the women and midwives appeared to share their definition of the construct of support. The high level of correlation between the women's and observers' assessments further supports the validity of the systematic observational approach to measure the quantity and quality of intrapartum support.
Testing of the inter-rater reliability of the SMILI in the clinical setting was not undertaken. This was due to the judgement that asking women and midwives to consent to have two observers present in the labour room would present more ethical issues in terms of being considerably more invasive of such a private experience.
The SMILI was tested for key features of validity and reliability using a number of approaches and was found to demonstrate good levels of validity and reliability. However, it is recognised that the validity and reliability of any new instrument is an ongoing process, not completed in one study. Further testing of the validity and reliability of the SMILI in other settings will be required to ensure transferability of the instrument to settings outside the specific circumstances of these studies.
The study found that the SMILI and other outcome measures enabled measurement of key aspects of the quantity and quality of midwifery intrapartum support.
The data relating to the quantity of behaviours show considerable variations between midwives and identify some patterns of behaviour for particular midwives. For example, two midwives were found to have shown below the study average of all neutral and negative behaviours and above the study average for all positive support behaviours.
Measuring the quantity of behaviours facilitates the measurement of the quality of the support. The behaviours chosen to be measured using the SMILI were based on the theoretical definitions of high quality intrapartum support and the large body of research undertaken with women that identified the key behaviours they experienced as being supportive and unsupportive. The SMILI demonstrated good sensitivity to differences in behaviour by midwives that resulted in different assessments of support by women and observers. It is therefore possible to begin to build a clearer picture of the details of what may be described as high quality midwifery support.
The widely recognised Institute of Medicine definition of quality in healthcare is care that is person-centred, safe, effective, efficient, equitable and timely. Through the measurement of the amount of time that the midwife is present in the labour room the SMILI is able to contribute to the assessment of the quality of the support through recording the extent to which the support is safe, equitable and timely. Through the measurement of the frequency that negative and positive behaviours are displayed the SMILI is able to assist in the assessment of how person-centred and safe the care is. Through the exploration of correlations between the behaviour data recorded and key clinical outcomes the SMILI can contribute to the assessment of whether the support is safe, effective and efficient.
All of the observations were undertaken in Scotland in NHS hospital settings which may limit the generalisability of findings. The practice of midwifery in the NHS in Scotland may differ in some key respects from other parts of the UK and internationally. While midwives in this study were able to provide one to one support during labour this may not reflect the situation for all midwives at all times providing care in other parts of the NHS in Scotland or elsewhere. However, the aim of this study was to test the validity and reliability of the SMILI in measuring the quantity and quality of midwifery support in labour. The instrument could then be used in a wider range of practice settings to describe routine midwifery support across the UK. While the woman and midwife participants in the study provide a reasonably representative sample of the UK population in terms of risk factors, parity, age, use of pain relief, type of birth, type of midwifery training and years of experience, they do not comprise a representative sample of the UK population in terms of ethnic diversity.
A further potential limitation of the clinical study is the unknown impact of the presence of an observer in the labour room on the quantity and quality of the care provided. Though careful precautions were taken in this study to ensure that the observer was as unobtrusive as possible, the midwives being observed had been made aware of the overall purpose of the research and that the observer would be recording the supportive elements of care. The effect of an observer or researcher on participants cannot be measured. It is possible that the quality of care provided by the observed midwives was on occasion of a higher level than may be expected with no observer present. However, the identification of substantial variations in the behaviours of the midwives and the presence of some negative behaviours suggest that the observers were successful in being unobtrusive and limiting their impact on the care provided.
Finally, accurate consent rate numbers and reasons for non-participation were not collected in the clinical study and would be required in a larger study. Consent was not sought from all women, birth partners and midwives potentially available to be observed on any shift and therefore a true denominator for a consent rate would have been difficult to identify. As midwives who were potential participants were also on occasion seeking the consent of the woman and her birth partner, they operated in some respects as a gatekeeper for entry into the study. The researcher was not present during this information and consent discussion. Any data about rates and reasons for women and birth partners declining participation were therefore 'filtered' via the midwives. A data collection procedure for the collection of the number of women, birth partners and midwives approached, number who declined and reasons given for declining would be included in a future study.
Discussion
This study found that the SMILI was feasible and acceptable for use in the intrapartum setting. The SMILI had been found to have good face and content validity and reliability when tested in a non-clinical setting, this study aimed to identify whether it was reliable, valid and feasible for use in the real world of care of women in labour.
The study was successful in recruiting participants across the four participant sites and achieved the maximum goal for numbers with forty nine complete observations analysed. Overall the great majority of women, birth partners and midwives approached gave consent to participate.
Logistical issues, relating to researcher travel and the smaller numbers of prospective participants meant that recruitment at the two smaller participating units was lower than the larger units.
The study compares favourably in size with the most comparable studies undertaken in North America. The Miltner study undertook 150 hours of observations of 24 nurses caring for 75 women, the Barnett study undertook 75 hours of observations of 17 nurses caring for 30 women.
The study demonstrated that the direct observational approach was understandable and acceptable to the majority of women, their birth partners and midwives in the clinical setting. The SMILI worked well in the clinical setting with loss of data in only three cases due to user error. The SMILI worked from an inexpensive memory stick on standard personal computer laptops without any special adaptation and so did not represent a costly method of collecting data yet it was successful in obtaining a large amount of comprehensible and usable data relating to intrapartum midwifery care.
The study further tested the SMILI for face, content and construct validity and internal reliability and found the instrument to be both valid and reliable for use in intrapartum care.
The observers rated face and content validity of the SMILI high in the clinical setting. Following some minor amendments to content after the early observations, content validity of the SMILI was found to be high by the observers using the instrument in the setting for which it was designed. The observers felt that the programme enabled them to record the support seen.
The SMILI was found to have good levels of construct validity through the identification of significant correlations between the care recorded and women's feelings expressed in the postnatal questionnaire (SCIB). Women rated the support they received more highly when they received higher quantities of emotional, informational, tangible and partner support and when the midwife was in the room more. A larger sample of observations would enable further testing of the correlations between both negative and positive behaviours and women's views. Emotional support behaviours had the strongest links with women's views, which confirms the findings of earlier studies with women which identify emotional support behaviours as the most important category of support. Construct validity of the SMILI was further supported by the findings showing no correlation between the amount of non-support behaviours and non-supportive care including assessment and women's views of the support received with women's views in the postnatal questionnaire.
Good levels of internal reliability of the SMILI were found. Different aspects of 'attitude' correlated highly with each other for the woman, her birth partner and the midwife. Where a midwife remained in close proximity to the woman, she was also likely to demonstrate a positive demeanour, use a positive vocal tone and display a positive facial expression. Where a midwife displayed a neutral or negative demeanour she was more likely to use a neutral or negative vocal tone and display a negative facial expression.
The veridicality of the methodology was demonstrated through the identified correlation between women's views of the support received and the observers' overall assessments of the support they had observed. For the most part, the women and midwives appeared to share their definition of the construct of support. The high level of correlation between the women's and observers' assessments further supports the validity of the systematic observational approach to measure the quantity and quality of intrapartum support.
Testing of the inter-rater reliability of the SMILI in the clinical setting was not undertaken. This was due to the judgement that asking women and midwives to consent to have two observers present in the labour room would present more ethical issues in terms of being considerably more invasive of such a private experience.
The SMILI was tested for key features of validity and reliability using a number of approaches and was found to demonstrate good levels of validity and reliability. However, it is recognised that the validity and reliability of any new instrument is an ongoing process, not completed in one study. Further testing of the validity and reliability of the SMILI in other settings will be required to ensure transferability of the instrument to settings outside the specific circumstances of these studies.
The study found that the SMILI and other outcome measures enabled measurement of key aspects of the quantity and quality of midwifery intrapartum support.
The data relating to the quantity of behaviours show considerable variations between midwives and identify some patterns of behaviour for particular midwives. For example, two midwives were found to have shown below the study average of all neutral and negative behaviours and above the study average for all positive support behaviours.
Measuring the quantity of behaviours facilitates the measurement of the quality of the support. The behaviours chosen to be measured using the SMILI were based on the theoretical definitions of high quality intrapartum support and the large body of research undertaken with women that identified the key behaviours they experienced as being supportive and unsupportive. The SMILI demonstrated good sensitivity to differences in behaviour by midwives that resulted in different assessments of support by women and observers. It is therefore possible to begin to build a clearer picture of the details of what may be described as high quality midwifery support.
The widely recognised Institute of Medicine definition of quality in healthcare is care that is person-centred, safe, effective, efficient, equitable and timely. Through the measurement of the amount of time that the midwife is present in the labour room the SMILI is able to contribute to the assessment of the quality of the support through recording the extent to which the support is safe, equitable and timely. Through the measurement of the frequency that negative and positive behaviours are displayed the SMILI is able to assist in the assessment of how person-centred and safe the care is. Through the exploration of correlations between the behaviour data recorded and key clinical outcomes the SMILI can contribute to the assessment of whether the support is safe, effective and efficient.
Limitations of the Research
All of the observations were undertaken in Scotland in NHS hospital settings which may limit the generalisability of findings. The practice of midwifery in the NHS in Scotland may differ in some key respects from other parts of the UK and internationally. While midwives in this study were able to provide one to one support during labour this may not reflect the situation for all midwives at all times providing care in other parts of the NHS in Scotland or elsewhere. However, the aim of this study was to test the validity and reliability of the SMILI in measuring the quantity and quality of midwifery support in labour. The instrument could then be used in a wider range of practice settings to describe routine midwifery support across the UK. While the woman and midwife participants in the study provide a reasonably representative sample of the UK population in terms of risk factors, parity, age, use of pain relief, type of birth, type of midwifery training and years of experience, they do not comprise a representative sample of the UK population in terms of ethnic diversity.
A further potential limitation of the clinical study is the unknown impact of the presence of an observer in the labour room on the quantity and quality of the care provided. Though careful precautions were taken in this study to ensure that the observer was as unobtrusive as possible, the midwives being observed had been made aware of the overall purpose of the research and that the observer would be recording the supportive elements of care. The effect of an observer or researcher on participants cannot be measured. It is possible that the quality of care provided by the observed midwives was on occasion of a higher level than may be expected with no observer present. However, the identification of substantial variations in the behaviours of the midwives and the presence of some negative behaviours suggest that the observers were successful in being unobtrusive and limiting their impact on the care provided.
Finally, accurate consent rate numbers and reasons for non-participation were not collected in the clinical study and would be required in a larger study. Consent was not sought from all women, birth partners and midwives potentially available to be observed on any shift and therefore a true denominator for a consent rate would have been difficult to identify. As midwives who were potential participants were also on occasion seeking the consent of the woman and her birth partner, they operated in some respects as a gatekeeper for entry into the study. The researcher was not present during this information and consent discussion. Any data about rates and reasons for women and birth partners declining participation were therefore 'filtered' via the midwives. A data collection procedure for the collection of the number of women, birth partners and midwives approached, number who declined and reasons given for declining would be included in a future study.
SHARE
