INTRODUCTION

The novel coronavirus disease 2019 (COVID-19) was initially discovered in Wuhan, China in December 2019, and has since spread around the globe [1]. The first patient with COVID-19 in the United States was diagnosed on January 20, 2020 [2]. Since then, the infection has progressed into a worldwide pandemic, with the United States having the greatest number of cases and deaths in the world as of May 24, 2020 [3]. There is currently significant global interest in understanding how the infection is spreading, how to contain the spread of the infection, and developing a vaccine against the virus.

Many nations had opted to secure their borders and implement social distancing measures to mitigate the spread of the virus. While state mandated social distancing measures, which will henceforth be called as the lockdown measures, in the Unites States were in effect in most states, the degree of implementation varied between states and even counties. Most states allowed their residents to leave their homes to get groceries, to exercise outdoors, and to get medical care, while closing non-essential services and mandating social distancing while allowing people associated with essential services to continue working [4]. Several states never formally implemented any social distancing, and at the time of the data analysis of this study, many had already started relaxing these measures. There has been a lot of debate about the timing and necessity of state mandated lockdown orders as the economy of the states and the country have slowed down due to these steps. At the time of the preparation of this manuscript, there was significant skepticism regarding withdrawal of state mandated lockdown orders.

Social determinants of health may lead to healthcare disparities [5], and have been observed to be associated with an increased risk of infection during prior viral respiratory pandemics [6]. Socioeconomic status, racial/ethnic minority status, household composition, and environmental factors, were found to be significantly associated with COVID-19 incidence, severity, and mortality [7,8]. A similar study conducted in Northern Italy during the same period found that specific socioeconomic factors (unemployment and public transportation usage) and the proportion of private healthcare facilities played an important role in the spread of the infection [9]. Another European study found that total population, poverty, and income are the primary factors regulating death from COVID-19 in the European region [10].

An understanding of these factors may aid the medical and scientific communities in identifying populations at greater risk for acquiring COVID-19. Therefore, our aim was to determine the association between the timing of lockdown at the county-level and the spread of COVID-19 after adjusting for the effects of covariates such as demographic profile, geographic, socioeconomic, and other known risk factors of COVID-19.

METHODS

This cross-sectional ecological study was conducted from publicly available county-level data. No individual patient protected health information was utilized. A list of all the Unite States counties and county-equivalent entities was obtained from the United States Census Bureau, with each county identified by its unique Federal Information Processing System (FIPS) code [11]. For each variable, the FIPS code was matched to ensure all variables are obtained for the same census designed area. The estimated 2019 population of each county was used, as extrapolated by the United States Census Bureau using its own 2010 data [11]. Data on the COVID-19 cases were obtained from various sources as mentioned below. All confirmed cases starting from the first case until May 24, 2020 were included in the study, the cut-off date for the data analysis in this study. The cut-off date was chosen because states had started withdrawing the lockdowns after this date. The inclusion criteria were then applied to generate the sample.

Inclusion Criteria

(1) The United States counties with greater than 100 000 population; (2) The United States counties with confirmed cases of at least 50 per 100 000 people.

The independent variable

The number of days to reach the target daily case rate (DCR; the day of confirmed cases at 50/100 000 in that county) from the day of official announcement of lockdown order.

Other covariates

Total population, population density, percentages of unemployment, poverty, population greater than 65 years, education level below high school level, female sex, African-American, Hispanic, rural, poor health, smoking, obesity, number of people per primary care physician (PCP), received influenza vaccine, social association, housing problem, diabetes mellitus, food insecurity, and uninsured.

Dependent variable (outcome)

DCR per 100 000: The rate of daily increase in cases recorded from the first case to the 50/100 000 cases. Calculation was performed by dividing 50 (numerator) by the number of days (denominator) in that county from the first recorded case to the day of 50/100 000 cases. An average rate of increase of cases per day in the county from the first detected case until the endpoint (50/100 000) showed the magnitude of spread of the virus in the community.

Definitions and Sources

The daily number of cases of COVID-19 was obtained from USA Facts [12]. Population density was computed by dividing the population of each county by the number of square miles per county [13]. Data for high school education, unemployment, and poverty were obtained from the Economic Research Service of the US Department of Agriculture as of 2018 [14]. For education, individuals of age 25 or older who had not obtained a high school diploma were included. The threshold for poverty was defined per United States Census Bureau, based on family size and income. All remaining data for the following variables were obtained from the county health rankings website [15]. Adults are defined as individuals greater than or equal to 20 years of age. Demographic data collected at the county-level included percentage of females, African Americans, and Hispanics as of 2018. Health-related variables examined included percentage of adults currently smoking tobacco (2017), percentage of adults with obesity defined as body mass index greater than or equal to 30 kg/m² (2016), PCP ratio was defined as the number of people served by one PCP (2017), percentage of Medicare enrollees who received the seasonal influenza vaccine (2017), percentage of adults with the diagnosis of diabetes mellitus (2016), and percentage of uninsured adults age 20 to 64 (2017). Further socioeconomic variables examined included percent rural population (2010), rate of social associations (2017), percentage of households with housing problem (2016), and percentage of people with food insecurity (2017). The rate of social associations was measured by the number of social membership associations per 10 000 people. Housing problem was measured by presence of overcrowding, high housing costs, and lack of kitchen or plumbing facilities. Food insecurity was measured by the percentage of population that indicated a lack of adequate access to food.

The date of lockdown order for each county was obtained from Institute of Health Matrix and Evaluation website (http://www.healthdata.org/) and was measured by the number of days from the state mandated lockdown order to the day of confirmed cases at 50/100 000 in that county, henceforth called as “days from lockdown order” [16].

Statistical Analysis

Linear regression was used to evaluate the association between the independent and the dependent variables. The independent variables were tested for normality. Analysis was carried out in two stages.

In the first stage, each independent factor (demographic or risk factor) was examined for correlation with the outcome variable using Spearman’s rho or Pearson’s method, depending on whether the data were non-normal or normally distributed respectively (Table 1). The independent factors were also assessed for collinearity with each other. If the coefficient of correlation was +0.5 or greater, one or more of the collinear factors were excluded from the analysis in stage two.

In the second stage, all the factors that were significantly associated (level of significance at <0.05) with each of the outcome variables and had a coefficient of correlation at least±0.1 or above were included in multiple linear regression models using stepwise method (Table 2). Models with best R² value from the multiple regression were chosen for the outcome variable for interpretation. Effect size was reported as standardized coefficients beta with a 95% confidence interval for comparison and a p-value.

Ethics Statement

This study was exempt from institutional review board approval due to no individual patient information used. All data were obtained from publicly available sources.

RESULTS

A total of 569 counties met the inclusion criteria from a list of 3142 counties assessed, representing all 50 states. The socio-demographic, geographic, health and other outcome characteristics of the population of the counties are illustrated in Table 1, as well as the outcome variable. The coefficient of correlation between each factor with the outcome is shown in the table along with the information about the direction and significance of the correlation. The highest coefficient of correlation was observed between the days from the lockdown order and the DCR per 100 000 (-0.73) (Figure 1). The population density per square mile had the next best coefficient for confirmed cases and deaths (0.34) followed by percentage of uninsured (-0.21 and -0.26). All the variables showed non-normal distribution except smoking. Collinearity ≥+0.5 was found to be present between population and population density; female sex with African American race; Hispanic race with housing problems; education below high school with poverty; poor health with poverty, smoking, diabetes mellitus, food insecurity and uninsured; smoking with diabetes, obesity, food insecurity, and poverty; food insecurity with poverty, smoking, uninsured and poor health.

For DCR, the independent variables considered for the second stage were population density, unemployment, female, African American race, Hispanic, uninsured and days from lockdown order. The best fit model obtained by stepwise regression in Table 2 shows that only three factors were significantly associated with DCR after adjusting for the rest: days from lockdown order, uninsured, and unemployment. The R² value of 0.60 shows a good fitted model. For the state mandated lockdown order, placing of the order a day earlier would have decreased the DCR by 0.04 cases. For each 1% increase in uninsured population, the DCR was predicted to decrease by 0.03 cases, and for each 1% increase in unemployment, 0.1 cases.

DISCUSSION

In this study, we demonstrated that the timing of state mandated lockdown orders had a significant association with the spread of COVID-19 after adjusting for multiple socio-demographic, geographic and health-related factors. A proclamation of this order in the counties that implemented it even a day earlier would have reduced the DCR by 0.04.

Daily case incidence, reproduction number (R₀), and the susceptible infectious model (SI Model) are different methods used to scale the spread of the infection [17-19]. In this study, we utilized the rate at which the infection reaches from 0 to 50/100 000, which is a simple way to measure the speed of spread of the virus assuming that initially everyone in the community was susceptible to the virus.

Health disparities between the counties in the United States has been in existence for long and have been previously well studied [20,21]. Socioeconomic factors such as sex, race, age, income, and insurance have been shown to be associated with the incidence and spread of COVID-19 [22]. Counties with low population in the United States may not have the adequate health infrastructure to manage complicated COVID-19 patients, leading to movement of these patients to adjoining counties with better facilities. Patients from smaller counties may be referred to larger health centers in adjacent counties with larger population causing sampling bias in the analysis. Furthermore, individuals with appropriate health insurance coverage may seek out referral centers in adjacent counties with better health facilities. We included counties with a population of >100 000 population and a minimum case rate of 50/100 000 in order to reduce these biases.

In this study, we considered several covariates that may act as confounders and affect the relationship between timing of the lockdown and DCR. Higher population density has been shown to result in faster spread of the virus as well as higher mortality [23,24]. Higher rates of unemployment and poverty has also been shown to be associated with increased infection rates as they lead to increased eviction rate causing overcrowding doubling up, transiency, healthcare access limitations, and a reduced ability to comply with strategies to prevent infection (e.g., social distancing, self-quarantine, and hygiene practices) [25]. Another study done in the United States showed that counties with higher total population, more diverse demographics, higher education, and income level are at a higher risk of COVID-19 infection [26]. When considering the demographics, Hispanic and African-American have a higher severity of infection, hospitalization, and mortality [27,28]. Because these races have a more severe disease, they may also have proportionally higher DCR. Similarly, males have a higher rate of affliction and severity of COVID-19 infection [29,30]. Smoking, obesity, and diabetes mellitus are health-related risk factors that have been shown to have adverse effect on infection and mortality related to COVID-19 [31,32]. Data shows that about 33% of the counties in the United States are more susceptible to COVID-19 not only due to older and health-compromised individuals, but also due to poor access to care such as low number of physicians [33]. In this study, the ratio of the number of people to one physician and percentage of Medicare enrollees who received the seasonal influenza vaccine has been used as surrogates for healthcare access which may influence the diagnosis and spread of the virus in the community. The rate of social membership associations per 10 000 people was considered as a covariate as it may serve as a surrogate for social gatherings in the county which may lead to accelerated spread of the virus. Residential segregation, lack of insurance, and housing problems have been shown to be related to a higher infection rate and poor outcome related to COVID-19 infections at the county level [34]. In this study, the rate of uninsured and unemployment had a significant association with the DCR after adjusting for many other variables.

A number of covariates were considered, after adjusting for which days from lockdown order (coefficient of correlation -0.73, standardized beta of -0.70) was found to be significantly and inversely associated with the DCR. The differential effects of the timing of intervention in terms of social distancing, to prevent the spread of the virus, has already been established by Pei et al. [35]. The higher number of uninsured patients had a significant negative correlation with DCR, which can be explained by the differential accessibility of uninsured population to testing. Uninsured people are more likely to experience poverty and have transportation issues, therefore they may not be able to reach the few and far COVID-19 testing sites [36].

At the level of the community and the country, studies have shown that lockdowns are effective in reducing transmission, hospitalization, and death [37-39], while others have not found significant benefits preventing the spread of infection from very restrictive lockdowns [40]. This study has found a significant negative association (decreasing trend) of earlier stay-athome or lockdown with the DCR which is a surrogate for the spread of the virus in the community at the county level.

The major limitation of the study is that cases of COVID-19 in the counties cannot be directly linked to the subjects with the demographic, socioeconomic or geographic factor. This study only compares the population characteristics of the counties with the outcomes within the same and other counties. In addition, we considered about 20% of the most populated counties of the United States, which may lead to a bias resulting from the factors associated with less populated counties. Different rates of testing and reporting data in different counties have also been not accounted for in this study. Population movement due to referrals from satellite hospitals to large health centers may lead to statistical bias. Furthermore, this study did not consider other variables in the population such as compliance with quarantine rules, hand washing practices, and masking rates, which are other confounders.

Notes

CONFLICT OF INTEREST

The authors have no conflicts of interest associated with the material presented in this paper.

FUNDING

None.

ACKNOWLEDGEMENTS

Notes

AUTHOR CONTRIBUTIONS

Conceptualization: MMT, AD. Data curation: MMT. Formal analysis: AD. Funding acquisition: None. Methodology: MMT, AD. Project administration: MMT, AD. Visualization: MMT, AD. Writing – original draft: MMT, AD. Writing – review & editing: MMT, AD.

Figure. 1.

The correlation between the timing of the start of lockdown order and the daily case rate (DCR) per 100 000 people.

REFERENCES

• 1. Wang C, Horby PW, Hayden FG, Gao GF. A novel coronavirus outbreak of global health concern. Lancet 2020;395(10223):470-473ArticlePubMedPMC
• 2. Holshue ML, DeBolt C, Lindquist S, Lofy KH, Wiesman J, Bruce H, et al. First case of 2019 novel coronavirus in the United States. N Engl J Med 2020;382(10):929-936ArticlePubMedPMC
• 3. The New York Times. Coronavirus map: tracking the global outbreak 2020 [cited 2020 Jun 25]. Available from: https://www.nytimes.com/interactive/2020/world/coronavirus-maps.html?action=click&pgtype=Article&state=default&module=styln-coronavirus&variant=show&region=TOP_BANNER&context=storyline_menu
• 4. O’Kane C. What does a “stay-at-home” order really mean? CBS News; 2020 Apr 3 [cited 2020 Jun 27]. Available from: https://www.cbsnews.com/news/what-does-stay-at-home-order-mean-coronavirus/
• 5. Penman-Aguilar A, Talih M, Huang D, Moonesinghe R, Bouye K, Beckles G. Measurement of health disparities, health inequities, and social determinants of health to support the advancement of health equity. J Public Health Manag Pract 2016;22(Suppl 1):S33-S42ArticlePubMedPMC
• 6. Lowcock EC, Rosella LC, Foisy J, McGeer A, Crowcroft N. The social determinants of health and pandemic H1N1 2009 influenza severity. Am J Public Health 2012;102(8):e51-e58ArticlePubMedPMC
• 7. Karmakar M, Lantz PM, Tipirneni R. Association of social and demographic factors with COVID-19 incidence and death rates in the US. JAMA Netw Open 2021;4(1):e2036462ArticlePubMedPMC
• 8. James MK, Kishore M, Lee SW. Demographic and socioeconomic characteristics of COVID-19 patients treated in the emergency department of a New York City hospital. J Community Health 2020;1-8ArticlePubMed
• 9. Buja A, Paganini M, Cocchio S, Scioni M, Rebba V, Baldo V. Demographic and socio-economic factors, and healthcare resource indicators associated with the rapid spread of COVID-19 in Northern Italy: an ecological study. PLoS One 2020;15(12):e0244535ArticlePubMedPMC
• 10. Sannigrahi S, Pilla F, Basu B, Basu AS, Molter A. Examining the association between socio-demographic composition and COVID-19 fatalities in the European region using spatial regression approach. Sustain Cities Soc 2020;62: 102418ArticlePubMedPMC
• 11. United States Census Bureau. County population totals. 2010-2019 [cited 2020 Jun 26]. Available from: https://www.census.gov/data/tables/time-series/demo/popest/2010s-counties-total.html#par_textimage_242301767
• 12. USA Facts. Data. 2020 [cited 2020 Jun 26]. Available from: https://usafacts.org/data/
• 13. IndexMundi. United States. 2019 [cited 2020 Jun 26]. Available from: https://www.indexmundi.com/united_states/#Introduction
• 14. U.S. Department of Agriculture. Download data. 2019 [cited 2020 Jun 26]. Available from: https://www.ers.usda.gov/data-products/county-level-data-sets/download-data
• 15. University of Wisconsin Population Health Institute. Explore health rankings. 2020 [cited 2020 Jun 26]. Available from: https://www.countyhealthrankings.org/explore-health-rankings
• 16. Institute for Health Metrics and Evaluation. COVID-19 resources. 2020 [cited 2020 Jun 24]. Available from: http://www.healthdata.org/covid
• 17. Shim E, Tariq A, Choi W, Lee Y, Chowell G. Transmission potential and severity of COVID-19 in South Korea. Int J Infect Dis 2020;93: 339-344ArticlePubMedPMC
• 18. Collins J, Abdelal N. Spread of disease. 2018 [cited 2020 Jun 24]. Available from: https://calculate.org.au/wp-content/uploads/sites/15/2018/10/spread-of-disease.pdf
• 19. Steinmeyer SH, Wilke CO, Pepin KM. Methods of modelling viral disease dynamics across the within- and between-host scales: the impact of virus dose on host population immunity. Philos Trans R Soc Lond B Biol Sci 2010;365(1548):1931-1941ArticlePubMedPMC
• 20. Weinstein JN, Geller A, Negussie Y, Baciu A; National Academies of Sciences, Engineering, and Medicine; Committee on Community-Based Solutions to Promote Health Equity in the United States. Communities in action: pathways to health equity. Washington, D.C: National Academies Press; 2017. p. 57-97
• 21. Murray CJ, Kulkarni SC, Michaud C, Tomijima N, Bulzacchelli MT, Iandiorio TJ, et al. Eight Americas: investigating mortality disparities across races, counties, and race-counties in the United States. PLoS Med 2006;3(9):e260ArticlePubMedPMC
• 22. Baum CF, Henry M. Socioeconomic factors influencing the spatial spread of COVID-19 in the United States. SSRN 2020. doi: http://dx.doi.org/10.2139/ssrn.3614877Article
• 23. Rocklöv J, Sjödin H. High population densities catalyse the spread of COVID-19. J Travel Med 2020;27(3):taaa038ArticlePubMed
• 24. Ilardi A, Chieffi S, Iavarone A, Ilardi CR. SARS-CoV-2 in Italy: population density correlates with morbidity and mortality. Jpn J Infect Dis 2021;74(1):61-64ArticlePubMed
• 25. Benfer EA, Vlahov D, Long MY, Walker-Wells E, Pottenger JL Jr, Gonsalves G, et al. Eviction, health inequity, and the spread of COVID-19: housing policy as a primary pandemic mitigation strategy. J Urban Health 2021;98(1):1-12ArticlePubMedPMC
• 26. Abedi V, Olulana O, Avula V, Chaudhary D, Khan A, Shahjouei S, et al. Racial, economic, and health inequality and COVID-19 infection in the United States. J Racial Ethn Health Disparities 2021;8(3):732-742ArticlePubMed
• 27. El Chaar M, King K, Galvez Lima A. Are black and Hispanic persons disproportionately affected by COVID-19 because of higher obesity rates? Surg Obes Relat Dis 2020;16(8):1096-1099ArticlePubMedPMC
• 28. Selden TM, Berdahl TA. COVID-19 and racial/ethnic disparities in health risk, employment, and household composition. Health Aff (Millwood) 2020;39(9):1624-1632ArticlePubMed
• 29. Haitao T, Vermunt JV, Abeykoon J, Ghamrawi R, Gunaratne M, Jayachandran M, et al. COVID-19 and sex differences: mechanisms and biomarkers. Mayo Clin Proc 2020;95(10):2189-2203PubMed
• 30. Cai H. Sex difference and smoking predisposition in patients with COVID-19. Lancet Respir Med 2020;8(4):e20ArticlePubMedPMC
• 31. Hu L, Chen S, Fu Y, Gao Z, Long H, Ren HW, et al. Risk factors associated with clinical outcomes in 323 coronavirus disease 2019 (COVID-19) hospitalized patients in Wuhan, China. Clin Infect Dis 2020;71(16):2089-2098ArticlePubMed
• 32. Best JH, Mohan SV, Kong AM, Patel K, Pagel JM, Ivanov B, et al. Baseline demographics and clinical characteristics among 3471 US patients hospitalized with COVID-19 and pulmonary involvement: a retrospective study. Adv Ther 2020;37(12):4981-4995ArticlePubMedPMC
• 33. Peters DJ. Community susceptibility and resiliency to COVID-19 across the rural-urban continuum in the United States. J Rural Health 2020;36(3):446-456ArticlePubMedPMC
• 34. McLaughlin JM, Khan F, Pugh S, Angulo FJ, Schmidt HJ, Isturiz RE, et al. County-level predictors of COVID-19 cases and deaths in the United States: what happened, and where do we go from here? Clin Infect Dis 2020;ciaa1729
• 35. Pei S, Kandula S, Shaman J. Differential effects of intervention timing on COVID-19 spread in the United States. Sci Adv 2020;6(49):eabd6370ArticlePubMedPMC
• 36. Kamimura A, Panahi S, Ahmmad Z, Pye M, Ashby J. Transportation and other nonfinancial barriers among uninsured primary care patients. Health Serv Res Manag Epidemiol; 2018. doi: https://doi.org/10.1177/2333392817749681Article
• 37. Ghosal S, Bhattacharyya R, Majumder M. Impact of complete lockdown on total infection and death rates: a hierarchical cluster analysis. Diabetes Metab Syndr 2020;14(4):707-711ArticlePubMedPMC
• 38. Ji T, Chen HL, Xu J, Wu LN, Li JJ, Chen K, et al. Lockdown contained the spread of 2019 novel coronavirus disease in Huangshi City, China: early epidemiological findings. Clin Infect Dis 2020;71(6):1454-1460ArticlePubMed
• 39. Borjas GJ. Business closures, stay-at-home restrictions, and COVID-19 testing outcomes in New York City. Prev Chronic Dis 2020;17: E109ArticlePubMedPMC
• 40. Bendavid E, Oh C, Bhattacharya J, Ioannidis JP. Assessing mandatory stay-at-home and business closure effects on the spread of COVID-19. Eur J Clin Invest 2021;51(4):e13484ArticlePubMedPMC

Figure & Data

References

Citations

Citations to this article as recorded by  

• The Economic Power of Capital and the Spread of COVID-19 in the United States: An Empirical Study
  Junfu Zhao
  Review of Radical Political Economics.2025; 57(1): 30.     CrossRef
• Determining the best mathematical model for implementation of non-pharmaceutical interventions
  Gabriel McCarthy, Hana M. Dobrovolny
  Mathematical Biosciences and Engineering.2025; 22(3): 700.     CrossRef
• Predisposing, Enabling, and Reinforcing Factors of COVID-19 Prevention Behavior in Indonesia: A Mixed-methods Study
  Putri Winda Lestari, Lina Agestika, Gusti Kumala Dewi
  Journal of Preventive Medicine and Public Health.2023; 56(1): 21.     CrossRef
• Assessing the Impacts of COVID-19 and Social Isolation on Mental Health in the United States of America
  Alexander Fulk, Raul Saenz-Escarcega, Hiroko Kobayashi, Innocent Maposa, Folashade Agusto
  COVID.2023; 3(6): 807.     CrossRef
• Understanding mental health trends during COVID-19 pandemic in the United States using network analysis
  Hiroko Kobayashi, Raul Saenz-Escarcega, Alexander Fulk, Folashade B. Agusto, Karina Cardoso Meira
  PLOS ONE.2023; 18(6): e0286857.     CrossRef
• Conspiratorial Ideation Is Associated with Lower Perceptions of Policy Effectiveness: Views from Local Governments during the COVID-19 Pandemic
  Adam Mayer, Stacia Ryder
  Socius: Sociological Research for a Dynamic World.2023;[Epub]     CrossRef
• Effectiveness of social distancing measures and lockdowns for reducing transmission of COVID-19 in non-healthcare, community-based settings
  Caitriona Murphy, Wey Wen Lim, Cathal Mills, Jessica Y. Wong, Dongxuan Chen, Yanmy Xie, Mingwei Li, Susan Gould, Hualei Xin, Justin K. Cheung, Samir Bhatt, Benjamin J. Cowling, Christl A. Donnelly
  Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.2023;[Epub]     CrossRef
• Effects of Closures and Openings on Public Health in the Time of COVID-19: A Cross-Country and Temporal Trend Analysis
  Long Chu, R. Quentin Grafton, Tom Kompas, Mary-Louise McLaws
  Sage Open.2023;[Epub]     CrossRef
• Association mining based approach to analyze COVID-19 response and case growth in the United States
  Satya Katragadda, Raju Gottumukkala, Ravi Teja Bhupatiraju, Azmyin Md. Kamal, Vijay Raghavan, Henry Chu, Ramesh Kolluru, Ziad Ashkar
  Scientific Reports.2021;[Epub]     CrossRef