GRAPHICAL ABSTRACT

INTRODUCTION

An estimated 44 million people are employed in organised sector workplaces in India (Census 2011) [1]. Within the organised sector, information technology (IT) and IT-enabled service sector employees form a key workforce from an economic perspective. The IT sector’s contribution to India’s gross domestic product (GDP) increased from 1.2% in 1998 to 7.4% in 2022, and it is expected to contribute 10% to India’s GDP by 2025 [2]. India’s IT sector is the leading sourcing destination globally, accounting for approximately 55% of the global market share of the services sourcing business (year 2019–2020). During the fiscal year 2021–2022, the IT industry employed 5.1 million people in India [3], and the good health of these employees is vital for sustained economic growth and development [4].

With rapidly evolving work ecosystems, the health profile of relatively young employees is changing. Interactions with medical officers indicate that obesity, hypertension, impaired glucose tolerance, mental disorders, substance use, and repetitive stress injuries are commonly observed in the IT workforce. Work-related stress is often reported to be closely associated with many health and productivity issues in IT employees [5].

Work stress was observed in an estimated 51% of IT professionals [6] and in 55% of managerial employees [7]. Despite methodological limitations and variations, it is generally accepted by stakeholders that work stress is widely prevalent and is an important issue to be addressed because the Indian workforce is huge and consists of the young, productive segment of the population. Work-related stress is a response people may have when presented with work demands and pressures that are not matched to their knowledge and abilities and that challenge their ability to cope [8]. Low job security, excessive work demands, lack of job control, monotonous work, lack of organisational support, adverse physical working conditions, inflexible work hours, relationships at work, role conflict and ambiguity, and work-life imbalance are known work stressors [9].

Stress in harmful proportions increases the risk of non-communicable diseases (NCDs) and subsequent complications, which may increase healthcare costs and decrease work productivity. It is essential that harmful levels of work-related stress should be detected early and managed for optimum health and work efficiency. However, regular screening for work stress is non-existent in most workplaces because it is not a mandated part of the periodic medical examination process. In addition, few tools or questionnaires to screen for harmful work stress are available, and they are lengthy, not validated for Indian workplaces, or limited in scope, without comprehensively covering the experience of health-related symptoms due to harmful stress levels. Copyright and the costs of study instruments limit their application in low-resource settings and by researchers.

There is an unmet need for a short screening tool to identify harmful work stress and its related symptoms and to evaluate them amid the regular examination of workers. Towards this goal, Tool to Assess and Classify Work-related Stress (TAWS-16) was developed by the Centre for Public Health, National Institute of Mental Health and Neurosciences (NIMHANS) and used in many workplaces. The overview, content, criteria, and construct validity of TAWS-16 have been published [5], and the current paper presents the results for the instrument’s reliability and internal consistency.

METHODS

This reliability assessment study is part of a larger validation study of TAWS-16 conducted on 356 managerial-supervisory staff affiliated with select IT companies in Bengaluru, Karnataka, India, from October 2020 to December 2020. Reliability assessments were conducted on a random sample of 62 study subjects drawn from the set of 356 IT staff. Sampling details are available at https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0280189 . The sample size for the reliability assessment was estimated based on an expected intra-class correlation coefficient (ICC) of 0.8 [10], a significance level of 0.05, a test power of 80%, and a 20% non-response rate. The final sample size was calculated to be 62 subjects for the assessment of test-retest, inter-method reliability, and internal consistency.

TAWS-16 consists of 2 sections, A and B, with 16 items each. Items in Section A investigate the work-stressor experience of employees over the past 6 months of employment and categorize their work-stress levels. Table 1 presents these items and their domains of assessment. Details regarding items, responses, scoring for each item, and cut-off scores for categorization of work-stress levels are available in the Instruction Manual of TAWS-16 [11]. Employees reporting experience of work stressors affirmatively in the last 6 months (for each item in Section A) are asked further regarding their ability to cope or efficiently manage each reported stressor. Subjects scoring >48 points in Section A are classified as having work stress, with different cut-off scores for mild, moderate, and severe stress. Items in Section B investigate commonly reported psychosomatic symptoms suggestive of work stress. Employees experiencing symptoms are asked further about their frequency. A cross-tabulation of different categories of work-stress levels and symptom levels is used to provide a color code and intervention guide for occupational health managers, from an NCD risk-reduction perspective [5].

Data Collection

Data collection was initially planned to occur through face-to-face interviews with the study participants, but we changed the approach due to the spread of coronavirus disease 2019 (COVID-19) in Bengaluru. A web application was developed for TAWS-16, following the complete life cycle of software development. Participants from consenting IT companies in Bengaluru were contacted, and a web link ( https://app.esamiksha.in/stress_assessment) was provided to gather their responses. We received a total of 356 responses.

Assessing inter-method reliability

A simple random sample of 62 respondents was contacted by telephone and interviewed 1 week later, with the interview completed within 7 days of contact, to assess reliability between the self-administered web method and the telephone interview method. Inter-method reliability was considered acceptable if the ICC >0.7.

Test-retest reliability

Test-retest reliability provides an indication of stability over time. A TAWS-16 link was provided to participants on 2 separate occasions within 1 week for self-administration and reporting. Test-retest reliability was considered acceptable if the ICC >0.7.

Internal consistency

Internal consistency concerns the extent to which items on the test or instrument measure the same thing. It was assessed by computing Cronbach’s alpha for each item, each sub-scale, and the entire study instrument after data collection from all study subjects, with 0.8 considered minimally acceptable.

Data from the application was provided to the investigator as a comma-separated values file, which was converted to Excel format (Microsoft, Redmond, WA, USA) and checked for duplicate values, outlier issues, and consistency in responses. Coding and data were analyzed using SPSS version 25 (IBM Corp., Armonk, NY, USA). Data privacy and confidentiality were ensured because the web application captured anonymous, de-linked data, with each subject creating his/her own user ID and password. Identifiers such as personal name, phone number, company name, and residence details were not collected.

Statistical Analysis

Test-retest and inter-method reliability were assessed for agreement beyond chance using unweighted percentage agreement (for 2×2 contingency tables) and kappa statistics [12]. Both ICCs and Spearman’s rank correlation coefficients [13] were computed as a measure of test-retest reliability and inter-method reliability. ICCs were classified as follows: “excellent” (≥0.81), “good” (0.61–0.80), “moderate” (0.41–0.60), and “poor” (≤0.40). The values of kappa were characterized as follows: “slight” (0.0–0.2), “moderate” (0.2–0.4), “fair” (0.4–0.6), “substantial” (0.6–0.8), “almost perfect” (0.8–1.0) [10]. Cronbach’s alpha was computed to assess internal consistency for both inter-method and test-retest reliability, with 0.8 [14] considered minimally acceptable.

Ethics Statement

Scientific and ethical clearance was obtained by NIMHANS Ethics Committee No. NIMH/DO/IEC (BS & NS DIV) 2020–21. Informed consent (verbal and through web-link) was obtained from each study participant. Participants were informed that they could withdraw from the study at any time.

RESULTS

We studied 62 participants, of whom 50 respondents (80.6%) did not reveal any work stress in either method (M1 and M2). Statistically significant inter-method reliability was observed between the self-administered and telephone interview methods for assessing work-related stress using TAWS-16 (kappa= 0.525, p<0.001; Table 2). Statistically significant test-retest reliability was observed for TAWS-16 as well (kappa=0.475, p<0.001).

For assessing psychosomatic symptoms suggestive of work-stress using TAWS-16, both inter-method and test-retest reliability assessments were conducted. Of the 62 participants, 35 (56.5%) did not reveal any symptoms suggestive of stress in either the self-administered or telephone interview methods. A statistically significant agreement (p<0.001) was observed for the ability of TAWS-16 to assess psychosomatic symptoms suggestive of work stress in both the self-administered and telephone interview methods (kappa=0.572, p<0.001; Table 3). Similar results were observed for the test-retest reliability assessment.

The reliability of TAWS-16 was assessed through test-retest and inter-method reliability tests. Table 4 shows the ICC values for test-retest (0.96) and inter-method reliability (0.88), indicating excellent reliability. Variation due to systematic errors in the inter-method assessment of TAWS-16 was around 12%, and that of the test-retest assessment was 3.6%, as per ICC values. The highest ICC value was observed for test-retest reliability (0.96) regarding work-stress assessment. All other ICC values showed excellent scores. Significant positive correlations were observed for both test-retest and inter-method reliability. Spearman’s rho ranged from 0.88 to 0.95, with the highest value (0.95) found for test-retest reliability for work-stress assessment.

Internal Consistency

Cronbach’s alpha for both test-retest and inter-method reliability showed good internal consistency. For the work-stress assessment items, the values of alpha for test-retest and inter-method reliability were 0.983 and 0.941, respectively. Similarly, for the items of symptoms suggestive of work-stress, the values of alpha for test-retest and inter-method reliability were 0.918 and 0.924, respectively (Table 4).

DISCUSSION

The reliability of TAWS-16 was investigated with a sample of 62 IT employees in Bengaluru, extracted from our overall validation study. A strength of the study was that it tested the reliability of a much-needed work-stress assessment tool, with a focus on increasing the scope of work-stress screening by assessing reliability between the self-administered and interview-administered methods. The study was completed despite the COVID-19 pandemic among IT professionals, who are a key working population in India. The web application ensured accuracy and completeness in data collection and has the potential to be customized for different workplaces and industries. The COVID situation provoked various aspects of work stress that invite further research and interventions.

TAWS-16 not only assesses and quantifies work stress, but also measures coping abilities and the experience of psychosomatic symptoms suggestive of work stress. TAWS-16 is brief, easy to administer, and easily understood by respondents. This tool has already been used in various industries, and implementing it has proven to be feasible.

The study results indicate that TAWS-16 has acceptable to good test-retest reliability. Because of methodological differences, it would not be epidemiologically correct to make direct comparisons with the findings of other studies, but the reliability characteristics of TAWS-16 appear to support its suitability for use.

For both test-retest and inter-method reliability, the agreement for work-related factors and symptoms suggestive of work stress exceeded 80%, and all kappa values were 0.40 or higher, suggesting acceptable test-retest reliability. The results for both test-retest and inter-method reliability were quite consistent. Similarly, almost all values of the ICCs and R_s regarding reliability exceeded 0.85 for both work-related factors and symptoms suggestive of work stress.

It is entirely plausible that work-related factors causing stress and the physical symptoms suggestive of work stress are consistent over the period of time studied, and hence our findings revealed a similar level of agreement. An underlying assumption of studies of test-retest reliability is that the object of measurement is very stable and is unlikely to change between measurements. It should also be noted that categorical measures (e.g., measures describing respondents as experiencing or not experiencing stress) will lower the values of the reliability statistics of those measures.

Though we have not undertaken a formal comparison in the study, a review of reliability measures indicates that TAWS-16 is as reliable as existing tools in terms of psychometric reliability scores, such as the Occupational Stress Scale [15], Job Description Index [16], Work Role Inventory [17], Work Ability Index and Work Ability Score [18], and Job Content Questionnaire [19].

The study’s limitations were closely related to the ongoing COVID-19 situation, since we were unable to conduct an inter-observer reliability measurement. Working from home might produce intermittent variations in stress responses for IT employees, which might interfere with test-retest and inter-method reliability as well. The questionnaire is presently available in English. For wider application in informal sectors, it would be necessary to translate TAWS-16 into other Indian languages and conduct cross-linguistic validation studies as well.

Since publicly available tools specific to measuring and classifying work stress in Indian work settings are limited, TAWS-16 will help to bridge this unmet need in occupational health services in India. The validity of TAWS-16 has been established and published. The findings of this study indicate that TAWS-16 has acceptable to excellent inter-method and test-retest reliability and internal consistency. TAWS-16 can be used for both self-report and interview-based screening for work stress. Although further assessment using other methods and in other demographic groups is required, TAWS-16 shows promise as a useful research and evaluation tool for work-stress assessment and classification.

Notes

CONFLICT OF INTEREST

The authors have no conflicts of interest associated with the material presented in this paper.

FUNDING

None.

AUTHOR CONTRIBUTIONS

Conceptualization: Gururaj G, Sukumar GM. Data curation: Roy R. Formal analysis: Roy R, Philip M. Funding acquisition: None. Methodology: Roy R, Sukumar GM, Philip M. Project administration: Roy R, Sukumar GM. Visualization: Roy R. Writing – original draft: Roy R. Writing – review & editing: Roy R, Sukumar GM, Philip M, Gururaj G.

ACKNOWLEDGEMENTS

REFERENCES

• 1. Ministry of Home Affairs, Government of India. Census tables: worker population enumeration; 2011 [cited 2023 May 6]. Available from: https://censusindia.gov.in/census.website/data/census-tables#
• 2. India Brand Equity Foundation. IT & BPM idustry in India; 2023 [cited 2023 May 6]. Available from: https://www.ibef.org/industry/information-technology-india
• 3. Ministry of Electronics and Information Technology, Government of India. Employment generation; 2022 [cited 2023 May 6]. Available from: https://www.meity.gov.in/content/employment
• 4. Directorate General, Factory Advice Service and Labour Institutes (DGFASLI), Ministry of Labour & Employment of India. Chapter 8: statistics on status of compliance with statutory provisions based on data collected by DGFASLI; 2011 [cited 2023 May 6]. Available from: https://dgfasli.gov.in/sites/default/files/safety_document/factories_2011.pdf
• 5. Roy R, Sukumar GM, Philip M, Gopalakrishna G. Face, content, criterion and construct validity assessment of a newly developed tool to assess and classify work-related stress (TAWS-16). PLoS One 2023;18(1):e0280189ArticlePubMedPMC
• 6. Darshan MS, Raman R, Rao TS, Ram D, Annigeri B. A study on professional stress, depression and alcohol use among Indian IT professionals. Indian J Psychiatry 2013;55(1):63-69ArticlePubMedPMC
• 7. Anbazhagan A, Rajan LS, Ravichandran A. Work stress of hotel industry employees in Puducherry. Asia Pac J Mark Manag Rev 2013;2(5):85-101
• 8. World Health Organization. Occupational health: stress at the workplace; 2020 [cited 2020 Dec 6]. Available from: https://www.who.int/news-room/q-a-detail/ccupational-health-stress-at-the-workplace
• 9. Rathore M, Kaushik V. Stress impact on industrial area of Indian managers. J Hum Ecol 2009;25(1):37-40Article
• 10. Fleiss JL. The design and analysis of clinical experiments. New York: Wiley; 1986. p. 17-28
• 11. Gautham MS, Roy R, Philip M, Guruaj G. TAWS-16: tool to assess and classify work-related stress. Bengaluru: Department of Epidemiology, Centre for Public Health; 2021. p. 7-16
• 12. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159-174ArticlePubMed
• 13. Spearman C. Demonstration of formulae for true measurement of correlation. Am J Psychol 1907;18(2):161-169Article
• 14. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16(3):297-334ArticlePDF
• 15. Chen WQ, Wong TW, Yu TS. Reliability and validity of the Occupational Stress Scale for Chinese off-shore oil installation workers. Stress Health 2001;17(3):175-183Article
• 16. Leong FT, Vaux A. Job descriptive index. In: Keyser DJ, Sweetland RC, editors. Test ritiques. Austin: Pro-Ed; 1992. p. 319-334
• 17. Miller JO, Carey SJ. Work role inventory: a guide to job satisfaction. Nurs Manage 1993;24(1):54-56. 60. 62
• 18. El Fassi M, Bocquet V, Majery N, Lair ML, Couffignal S, Mairiaux P. Work ability assessment in a worker population: comparison and determinants of Work Ability Index and Work Ability score. BMC Public Health 2013;13: 305ArticlePubMedPMCPDF
• 19. Karasek R, Brisson C, Kawakami N, Houtman I, Bongers P, Amick B. The Job Content Questionnaire (JCQ): an instrument for internationally comparative assessments of psychosocial job characteristics. J Occup Health Psychol 1998;3(4):322-355ArticlePubMed

Figure & Data

References

Citations

Citations to this article as recorded by