모바일 메뉴
Search
Search

KJO Korean Journal of Orthodontics

Open Access

pISSN 2234-7518
eISSN 2005-372X

퀵메뉴 버튼

Article

home All Articles View
Split Viewer

Original Article

Korean J Orthod 2023; 53(5): 328-335   https://doi.org/10.4041/kjod22.101

First Published Date September 25, 2023, Publication Date September 25, 2023

Copyright © The Korean Association of Orthodontists.

Does the quality of orthodontic studies influence their Altmetric Attention Score?

Thamer Alsaifa , Nikolaos Pandisb, Martyn T. Cobournea,c, Jadbinder Seehraa,c

aDepartment of Orthodontics, Faculty of Dentistry, Oral and Craniofacial Sciences, King’s College London, London, UK
bDepartment of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Bern, Switzerland
cCentre for Craniofacial Development & Regeneration, Faculty of Dentistry, Oral and Craniofacial Sciences, King’s College London, London, UK

Correspondence to:Jadbinder Seehra.
Senior Specialist Clinical Teacher, Centre for Craniofacial Development & Regeneration, Faculty of Dentistry, Oral and Craniofacial Sciences, King’s College London, Floor 25, Guy’s Hospital, Guy’s and St Thomas NHS Foundation Trust, London SE1 9RT, UK.
Tel +44-2071885665 e-mail jadbinderpal.seehra@kcl.ac.uk

How to cite this article: Alsaif T, Pandis N, Cobourne MT, Seehra J. Does the quality of orthodontic studies influence their Altmetric Attention Score? Korean J Orthod 2023;53(5):328-335. https://doi.org/10.4041/kjod22.101

Received: April 20, 2022; Revised: July 13, 2023; Accepted: August 25, 2023

Abstract

Objective: The aim of this study was to determine whether an association between study quality, other study characteristics, and Altmetric Attention Scores (AASs) existed in orthodontic studies. Methods: The Scopus database was searched to identify orthodontic studies published between January 1, 2017, and December 31, 2019. Articles that satisfied the eligibility criteria were included in this study. Study characteristics, including study quality were extracted and entered into a pre-pilot data collection sheet. Descriptive statistics were calculated. On an exploratory basis, random forest and gradient boosting machine learning algorithms were used to examine the influence of article characteristics on AAS. Results: In total, 586 studies with an AAS were analyzed. Overall, the mean AAS of the samples was 5. Twitter was the most popular social media platform for publicizing studies, accounting for 53.7%. In terms of study quality, only 19.1% of the studies were rated as having a high level of quality, with 41.8% of the studies deemed moderate quality. The type of social media platform, number of citations, impact factor, and study type were among the most influential characteristics of AAS in both models. In contrast, study quality was one of the least influential characteristics on the AAS. Conclusions: Social media platforms contributed the most to the AAS for orthodontic studies, whereas study quality had little impact on the AAS.

Keywords: Altmetric Attention Score, Social media

INTRODUCTION

The impact factor is a citation metric that has been used to measure the impact of scientific journals in a specific field.1 Additionally, the contributions and performance of researchers, universities, departments, and research groups are evaluated using citation counts, which has also become a key factor analysed before awarding research grants.2 Despite the widespread use of citation metrics, such approach is flawed due to factors such as the lack of consideration of the reasons for the citation of an article, manipulation of citation patterns, and underestimation of the impact of recently published articles because of the delay between the publication of the study and its first citation and indexing in a citation database.2-4

Social media platforms have revolutionized the dissemination of studies’ findings, allowing researchers to explore new approaches to publication accessibility for stakeholders and policymakers.2,4,5 Altmetric was introduced in 2010 and has been employed to measure the attention or social impact that publications attract on the social web.6 Altmetric Attention Score (AAS) (Altmetric LLP, London, UK) have been proposed to overcome the shortcomings of traditional journal citation metrics by facilitating publication updating in real time and providing a broader portrait of the influence of an article.7,8 The AAS is a weighted count of all the sources of online attention that a research article or researcher may receive. The sources that contribute to the AAS are public policy documents, blogs, mainstream media, citations, online reference managers, research highlights, post-publication peer-review platforms, social media, Wikipedia, Open Syllabus Project, patents, multimedia, and other online platforms (Altmetric LLP). Their increasing relevance is highlighted by several publishers of academic journals, including Elsevier, Oxford University Press, Springer, and Wiley, having recognized their use as a measure of impact.8 Furthermore, Altmetrics are being used by research funding bodies as an alternative to traditional citation metrics to assess research use.9 Articles with a higher AAS seem to correlate with a higher social impact.10,11 Social media platforms such as Twitter have been utilized to disseminate the findings of orthodontic-related studies.12,13 However, such platforms appear to be under-utilized, and improvements in accessibility for both scholars and non-scholars are required.14 Factors related to social media platforms like journal accounts5 and posts15 can influence whether an article will be shared on social media.

Despite its advantages, the AAS weighting surprisingly does not consider the quality of each study. The practice of evidence-based dentistry is advocated and strengthened by the use of high-quality studies to inform healthcare decisions.16 Study quality involves the assessment of the risk of bias (RoB), with various tools available for the evaluation of different study types. Both citation rates and journal impact factor have been associated with AASs.17 However, citation counts do not necessarily correlate with study quality.18 Therefore, the aim of this study was to determine whether an association between orthodontic study characteristics and their AAS exists; the null hypothesis was that study quality had no influence on the AAS.

MATERIALS AND METHODS

Eligibility criteria

Orthodontic studies with an AAS published between January 1, 2017, and December 31, 2019, were identified. Studies published in non-English languages, before January 2017 or after December 2019, as well as those unrelated to orthodontics were excluded.

Search and selection of studies

A search of the Scopus database (www.scopus.com) using the term “orthodontics” was undertaken by 1 author (TA) on July 9, 2020. The following search filters were applied: publication date (January 1, 2017, to December 31, 2019), language (English only), and journal title. The database search was performed on two occasions, four weeks apart, to evaluate changes in both the number of article citations and AAS; if a change was identified, the average score was recorded. All titles and abstracts were screened by the same author (TA) on both occasions.

Data extraction

All study characteristics were extracted by a single author (TA) and entered into a pre-piloted Microsoft Excel® (Microsoft, Redmond, WA, USA) data collection sheet. A second author (JS) cross-checked all variables from the extracted data independently to ensure the consistency of the data collected. The following characteristics were extracted from each study: year of publication, country of correspondence of the author (categorized as Europe, America, Asia, and rest of the world), journal title (categorized as American Journal of Orthodontics and Dentofacial Orthopaedics [AJODO], Australian Orthodontic Journal [AOJ], Journal of Orthodontics [JO], European Journal of Orthodontics [EJO], Journal of Clinical orthodontics [JCO], Journal of Orofacial Orthopaedics [JOO], Angle Orthodontist [AO], Orthodontics and Craniofacial Research [OCR], and other orthodontic journals), number of authors, study classification (appliances, diagnostic studies, materials, devices for patient use, software, pharmaceutical and others —including scanner, radiographic equipment, dental unit, curing light, laser system, and whitening systems), number of citations (reported citation counts obtained from [www.scopus.com]), AAS (Bookmarklet tool [www.almetrics.com]), social media platform that the article was shared on (categorized as Facebook, Twitter, Mendeley, Facebook and Twitter, or Multiple platforms [www.altmetric.com]), and impact factor (as reported by the Scimago Journal Rank). The quality of each article was assessed using a previously validated and predetermined set of criteria and graded accordingly (Table 1).19

Table 1 . Predetermined criteria used to assess study quality

GradeCriteria
A (high value of evidence)All criteria should be met:

Randomized clinical study or a prospective study with a well-defined control group

Defined diagnosis and endpoints

Diagnostic reliability tests and reproducibility tests described

Blinded outcome assessment

B (moderate value of evidence)All criteria should be met:

Cohort study or retrospective case series with defined control or reference group

Defined diagnosis and endpoints

Diagnostic reliability tests and reproducibility tests described

C (low value of evidence)One or more of the conditions below:

Large attrition

Unclear diagnosis and endpoints

Poorly defined patient material

Adapted from the article of Bondemark et al. (Angle Orthod 2007;77:181-91).19



Statistical analysis

Descriptive statistics were calculated for all study characteristics. To identify the influence of article predicator characteristics on the AAS, two machine learning algorithms were implemented: the random forest (randomForest package) and the gradient boosting machine approach (gbm package) in the R statistical software. The random forest20 model employs bootstrapping to create multiple copies of the original training dataset, fits a separate decision tree to each copy, and then combines all results to create a single predictive model. Gradient boosting21 operates similarly, however, trees are grown sequentially, and each tree is grown using information from previously grown trees. In all algorithms, Poisson regression was selected and was in agreement with the count nature of the dependent variable. All analyses were performed using Stata 16.1 (Stata Corp., College Station, TX, USA) and R Software version 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Criteria total of 586 eligible studies were identified (Figure 1). A 100% agreement was achieved between the two authors (TA and JS) for all the collected study characteristics. Within this sample, most studies with an AAS were published in 2018 (34.3%), with corresponding authors primarily based in Europe (41.5%). Studies classified as “others” (scanner, dental unit, radiographic equipment, curing light, whitening system, and laser system) most commonly possessed an AAS (58.0%). Twitter was the most popular social media platform to publicize the studies (53.7%). The most common study type was cross-sectional (23.2%). In terms of study quality, only 19.1% of the studies were rated as having a high level of quality, with 41.8% deemed as moderate quality. The mean number of authors was 4.5, and the mean journal impact factor and number of citations were 0.83 and 4.7, respectively (Table 2).

Figure 1. Flow diagram for the identification and selection of articles with an AAS.
AAS, Altmetric Attention Score.

Table 2 . Characteristics of articles with Altmetric Attention Scores

Article characteristicsn = 586
Year of publication
2017212 (36.2)
2018201 (34.3)
2019173 (29.5)
Journal title
AJODO55 (9.5)
JO17 (2.9)
EJO32 (5.5)
JOO2 (0.3)
AO20 (3.4)
OCR16 (2.7)
Other443 (75.7)
Continent of corresponding author
Europe243 (41.5)
Americas189 (32.3)
Asia and rest of the world154 (26.2)
Study classification
Erratum1 (0.2)
Appliances95 (16.2)
Diagnostic studies45 (7.7)
Materials74 (12.6)
Device for patient use6 (1.0)
Software15 (2.6)
Pharmaceutical10 (1.7)
Other*340 (58.0)
Study type
Erratum1 (0.2)
Systematic review49 (8.4)
Systematic review with meta-analysis41 (7.0)
Randomized clinical trial26 (4.4)
Case-control20 (3.4)
Cohort80 (13.7)
Cross-sectional study136 (23.2)
Case series8 (1.4)
Case report44 (7.5)
Opinion (editorials/letters/notes)59 (10.1)
Narrative review74 (12.6)
In-vitro41 (7.0)
Qualitative7 (1.1)
Study quality
High112 (19.1)
Moderate245 (41.8)
Low229 (39.1)
Type of social media platform
Not shared40 (6.8)
Twitter315 (53.7)
Facebook and Twitter97 (16.6)
Multiple134 (22.9)
AAS
Mean5
Median (IQR)1 (3)
Impact factor
Mean0.83
Median (IQR)0.78 (0.65)
Number of citations
Mean4.7
Median (IQR)2 (5)
Number of authors
Mean4.5
Median (IQR)4 (3)

Values are presented as number (%).

AJODO, American Journal of Orthodontics and Dentofacial Orthopaedics; JO, Journal of Orthodontics; EJO, European Journal of Orthodontics; JOO, Journal of Orofacial Orthopaedics; AO, Angle Orthodontist; OCR, Orthodontics and Craniofacial Research; AAS, Altmetric Attention Score; IQR, interquartile range.

*Scanner, dental unit, radiographic equipment, curing light, whitening system, and laser system.



Overall, the mean AAS of the samples was 5 and the average AAS per study characteristics is presented in Table 3. A higher mean AAS was evident for the following characteristics: articles published in 2017; articles published in Orthodontics and Craniofacial Research; articles with authors based in the Americas; studies classified as pharmaceutical; cohort-type studies; and studies rated as high quality.

Table 3 . The Altimetric Attention Score per study characteristics (n = 586)

Study characteristicsMeanMedianIQR
Year of publication
20176.413
20184.213
20194.312
Journal title
AJODO3.113
JO2.514
EJO4.54.57
JOO446
AO3.724
OCR5.814
Other5.412
Continent of corresponding author
Europe5.214
Americas5.712
Asia and rest of the world3.912
Study classification
Erratum110
Appliances5.114
Diagnostic studies4.314
Materials3.411
Device for patient use3.525
Software2.611
Pharmaceutical7.224
Other*5.513
Study type
Erratum110
Systematic review7.726
Systematic review with meta-analysis6.157
Randomized clinical trial3.214
Case-control3.324
Cohort9.613
Cross-sectional study3.512
Case series110
Case report2.411
Opinion (editorials/letters/notes)3.712
Narrative review4.41.53
In-vitro5.511
Qualitative3.634
Study quality
High6.127
Moderate5.512
Low412
Type of social media platform
Not shared110
Twitter1.610
Facebook and Twitter3.422
Multiple1588

AJODO, American Journal of Orthodontics and Dentofacial Orthopaedics; JO, Journal of Orthodontics; EJO, European Journal of Orthodontics; JOO, Journal of Orofacial Orthopaedics; AO, Angle Orthodontist; OCR, Orthodontics and Craniofacial Research; IQR, interquartile range.

*Scanner, dental unit, radiographic equipment, curing light, whitening system, and laser system.



To fit the machine learning algorithms, two outlier observations (namely, 396 and 140) were deleted because the results were unreliable. The randomForest approach ranked ten predictors associated with the AAS, and the variance explained was 42.5%, suggesting that other important unmeasured Altmetric parameters exist (Figure 2). Table 4 shows the ranking (1, highest influence; 10, lowest influence) of the predictors based on the two approaches implemented. The type of social media platform, number of citations, impact factor, and study type were among the most influential characteristics of AAS in both models. In contrast, study quality was one of the least influential characteristics of the AAS.

Figure 2. Percentage decrease in accuracy by elimination of the ten predictors, one at a time, using random forest (n = 584).

Table 4 . Ranking of the predictors based on the two machine learning algorithms (randomForest and gradient boosting machine), in which 1 represents the highest and 10 the lowest influence

PredictorRandom forestGBM (relative influence)
Social media platform11 (42.5)
Study type24 (6.3)
Impact factor33 (18.8)
Number of authors45 (5.6)
Year of publication56 (1.4)
Number of citations62 (23.3)
Study quality710 (0.02)
Journal88 (0.78)
Study classification97 (0.89)
Continent corresponding
author
109 (0.30)

GBM, gradient boosting machine.


DISCUSSION

AAS measures the amount of attention a study receives and is calculated using an automated algorithm that examines different sources in real time. Each source is weighted based on its relative impact. For example, if a study was mentioned in the news, this would be weighted more than a mention on Twitter or online reference managers. Based on this, the findings of the current study show that the greatest influence on AAS in orthodontic studies is the amount of attention received from social media platforms.

When comparing social media platforms, Twitter was the most commonly used social media platform in this study cohort (Table 1), which is similar to the findings of a previous report.5 Tweets are regarded as an effective method for sharing dental literature.22 However, Mendeley has also been reported as a popular social media platform to disseminate information to the community, and online attention to studies in terms of article access and citation counts has been correlated with downloads on Mendeley.13 The increased influence of social media platforms could also be a manifestation of publishers of orthodontic journals endorsing AAS as a measure of impact.8 Additionally, journals that possess social media accounts tend to have significant online attention compared to those without social media presence.23,24

The online attention of articles can include the number of times an article is accessed or downloaded, uploaded, discussed, bookmarked, cited, and recommended.25 In the current study, the number of citations and journal impact factors also strongly influenced the AAS for orthodontic studies; such finding is supported by the literature. In a systematic review of the associations between journal and article variables and AAS, both citation counts and journal impact factors were commonly associated with AAS.17 The accessibility and number of times an article was downloaded were beyond the scope of this study. However, free-access journals that facilitate access and downloading of articles tend to have higher online attention compared to subscription-based journals.23,26

Traditional metrics, such as the impact factor of a journal, do not necessarily correlate with the publication of high-level evidence within that journal.27 Within the current sample, approximately 19% of the studies were classified as having a high level of quality. Interestingly, the quality of the study was one of the characteristics that influenced AAS the least. This appears to conflict with the findings of a previous study, in which a high AAS of randomized clinical trials published in the field of total joint arthroplasty correlated with a high methodological quality.18 However, such study included a small sample of forty-two trials published in a single year, and potential biases during the assessment of methodological quality were not accounted for; notably, despite their reported findings, the authors concluded that clinicians should still critically evaluate studies before altering their clinical practice.18 AAS associated with orthodontic studies apparently cannot be considered a proxy for study quality.

In the current study, 586 studies which had an AAS were identified. As a study of this nature had not been previously conducted, ours represents a large sample size to explore the relationship between study quality and AAS. However, as solely a single database was searched (i.e., Scopus), language restrictions applied, and the broad search term “orthodontics” was used, we may have underestimated the true number of orthodontic studies possessing an AAS and, hence, the generalizability of the results may be limited. An assessment of study quality involves an assessment of the RoB in primary studies. We used a tool employed in a previously published systematic review. As described by the authors, this tool assesses both external and internal validity, the quality of the study methodology, and the statistical analysis performed.19 The study quality domains assessed in this tool are described in Table 1. The selected studies were classified into three broad categories. We acknowledge that to gauge a more detailed assessment of study quality, the use of specific RoB tools could be considered. Therefore, the results of this study should be interpreted with caution. Individual assessment of the 586 included studies using specific RoB tools was beyond the scope of this study, yet could be considered in a future study. Furthermore, the number of potential articles could have been increased by screening individual journal websites or by conducting a search via Medline via PubMed. The aim of the current study was not to precisely estimate the effect and precision of each predictor on ASS, but rather to provide initial insights into the AAS determinants, which can also be considered to answer other relevant questions. Articles were selected, and data extraction was primarily performed by a single author. However, to reduce possible biases, all articles included in the final analysis were independently cross-checked by a second author with complete agreement to ensure consistency.

CONCLUSIONS

In this exploratory cross-sectional study, social media platforms had the greatest influence on AASs in orthodontic studies. Among the study characteristics, the study quality had little impact on the AAS of orthodontic studies. Therefore, clinicians should critically evaluate the findings of these studies before implementing them in clinical practice.

FUNDING

None to declare.

AUTHOR CONTRIBUTIONS

Conceptualization: JS, NP. Data curation: TA, JS. Formal analysis: JS, NP. Investigation: TA, JS, NP. Methodology: JS, NP. Supervision: JS, NP, MTC. Writing–original draft: JS, TA NP, MTC. Writing–review & editing: JS, TA NP, MTC.

CONFLICTS OF INTEREST

No potential conflict of interest relevant to this article was reported.

References

  1. Sud P, Thelwall M. Evaluating altmetrics. Scientometrics 2014;98:1131-43. https://doi.org/10.1007/s11192-013-1117-2
    CrossRef
  2. Huang W, Wang P, Wu Q. A correlation comparison between Altmetric Attention Scores and citations for six PLOS journals. PLoS One 2018;13:e0194962. https://doi.org/10.1371/journal.pone.0194962
    Pubmed KoreaMed CrossRef
  3. Merton RK. The Matthew effect in science. The reward and communication systems of science are considered. Science 1968;159:56-63. https://doi.org/10.1126/science.159.3810.56
    Pubmed CrossRef
  4. Garcovich D, Adobes Martin M. Measuring the social impact of research in Paediatric Dentistry: an altmetric study. Int J Paediatr Dent 2020;30:66-74. https://doi.org/10.1111/ipd.12575
    Pubmed CrossRef
  5. Asyyed Z, McGuire C, Samargandi O, Al-Youha S, Williams JG. The use of twitter by plastic surgery journals. Plast Reconstr Surg 2019;143:1092e-8e. https://doi.org/10.1097/PRS.0000000000005535
    Pubmed CrossRef
  6. Brigham TJ. An introduction to altmetrics. Med Ref Serv Q 2014;33:438-47. https://doi.org/10.1080/02763869.2014.957093
    Pubmed CrossRef
  7. Elmore SA. The Altmetric Attention Score: what does it mean and why should I care?. Toxicol Pathol 2018;46:252-5. https://doi.org/10.1177/0192623318758294
    Pubmed KoreaMed CrossRef
  8. Livas C, Delli K. Looking beyond traditional metrics in orthodontics: an altmetric study on the most discussed articles on the web. Eur J Orthod 2018;40:193-9. https://doi.org/10.1093/ejo/cjx050
    Pubmed CrossRef
  9. Knight SR. Social media and online attention as an early measure of the impact of research in solid organ transplantation. Transplantation 2014;98:490-6. https://doi.org/10.1097/TP.0000000000000307
    Pubmed CrossRef
  10. Melero R. Altmetrics - a complement to conventional metrics. Biochem Med (Zagreb) 2015;25:152-60. https://doi.org/10.11613/BM.2015.016
    Pubmed KoreaMed CrossRef
  11. Champieux R. PlumX. J Med Libr Assoc 2015;103:63-4. https://doi.org/10.3163/1536-5050.103.1.019
    KoreaMed CrossRef
  12. Hughes H, Hughes A, Murphy C. The use of twitter by the trauma and orthopaedic surgery journals: twitter activity, impact factor, and alternative metrics. Cureus 2017;9:e1931. https://doi.org/10.7759/cureus.1931. Erratum in: Cureus 2018;10:c13.
    Pubmed KoreaMed CrossRef
  13. Amath A, Ambacher K, Leddy JJ, Wood TJ, Ramnanan CJ. Comparing alternative and traditional dissemination metrics in medical education. Med Educ 2017;51:935-41. https://doi.org/10.1111/medu.13359
    Pubmed CrossRef
  14. Delli K, Livas C, Spijkervet FKL, Vissink A. Measuring the social impact of dental research: an insight into the most influential articles on the Web. Oral Dis 2017;23:1155-61. https://doi.org/10.1111/odi.12714
    Pubmed CrossRef
  15. Zhang D, Earp BE. Correlation between social media posts and academic citations of orthopaedic research. J Am Acad Orthop Surg Glob Res Rev 2020;4:e20.00151. https://doi.org/10.5435/JAAOSGlobal-D-20-00151
    Pubmed KoreaMed CrossRef
  16. Madhavji A, Araujo EA, Kim KB, Buschang PH. Attitudes, awareness, and barriers toward evidence-based practice in orthodontics. Am J Orthod Dentofacial Orthop 2011;140:309-16.e2. https://doi.org/10.1016/j.ajodo.2010.05.023
    Pubmed CrossRef
  17. Araujo AC, Vanin AA, Nascimento DP, Gonzalez GZ, Costa LOP. What are the variables associated with Altmetric scores?. Syst Rev 2021;10:193. https://doi.org/10.1186/s13643-021-01735-0
    Pubmed KoreaMed CrossRef
  18. Kunze KN, Richardson M, Bernstein DN, Premkumar A, Piuzzi NS, McLawhorn AS. Altmetrics Attention Scores for randomized controlled trials in total joint arthroplasty are reflective of high scientific quality: an altmetrics-based methodological quality and bias analysis. J Am Acad Orthop Surg Glob Res Rev 2020;4:e20.00187. https://doi.org/10.5435/JAAOSGlobal-D-20-00187
    Pubmed KoreaMed CrossRef
  19. Bondemark L, Holm AK, Hansen K, Axelsson S, Mohlin B, Brattstrom V, et al. Long-term stability of orthodontic treatment and patient satisfaction. A systematic review. Angle Orthod 2007;77:181-91. https://doi.org/10.2319/011006-16R.1
    Pubmed CrossRef
  20. Breiman L. Random forests. Mach Learn 2001;45:5-32. https://doi.org/10.1023/A:1010933404324
    CrossRef
  21. Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Statist 2001;29:1189-232. https://doi.org/10.1214/aos/1013203451
    CrossRef
  22. Kolahi J, Khazaei S, Iranmanesh P, Soltani P. Analysis of highly tweeted dental journals and articles: a science mapping approach. Br Dent J 2019;226:673-8. https://doi.org/10.1038/s41415-019-0212-z
    Pubmed CrossRef
  23. Hassona Y, Qutachi T, Dardas L, Alrashdan MS, Sawair F. The online attention to oral cancer research: an altmetric analysis. Oral Dis 2019;25:1502-10. https://doi.org/10.1111/odi.13111
    Pubmed CrossRef
  24. Thoma B, Murray H, Huang SYM, Milne WK, Martin LJ, Bond CM, et al. The impact of social media promotion with infographics and podcasts on research dissemination and readership. CJEM 2018;20:300-6. https://doi.org/10.1017/cem.2017.394
    Pubmed CrossRef
  25. Garcovich D, Zhou Wu A, Sanchez Sucar AM, Adobes Martin M. The online attention to orthodontic research: an altmetric analysis of the orthodontic journals indexed in the journal citation reports from 2014 to 2018. Prog Orthod 2020;21:31. https://doi.org/10.1186/s40510-020-00332-6
    Pubmed KoreaMed CrossRef
  26. Konkiel SR. What can altmetrics tell us about interest in dental clinical trials?. Dent Hypotheses 2017;8:31-3. https://doi.org/10.4103/denthyp.denthyp_12_17
    CrossRef
  27. Rodrigues MA, Tedesco ACB, Nahas FX, Ferreira LM. Journal impact factor versus the evidence level of articles published in plastic surgery journals. Plast Reconstr Surg 2014;133:1502-7. https://doi.org/10.1097/PRS.0000000000000214
    Pubmed CrossRef

Article

Original Article

Korean J Orthod 2023; 53(5): 328-335   https://doi.org/10.4041/kjod22.101

First Published Date September 25, 2023, Publication Date September 25, 2023

Copyright © The Korean Association of Orthodontists.

Does the quality of orthodontic studies influence their Altmetric Attention Score?

Thamer Alsaifa , Nikolaos Pandisb, Martyn T. Cobournea,c, Jadbinder Seehraa,c

aDepartment of Orthodontics, Faculty of Dentistry, Oral and Craniofacial Sciences, King’s College London, London, UK
bDepartment of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Bern, Switzerland
cCentre for Craniofacial Development & Regeneration, Faculty of Dentistry, Oral and Craniofacial Sciences, King’s College London, London, UK

Correspondence to:Jadbinder Seehra.
Senior Specialist Clinical Teacher, Centre for Craniofacial Development & Regeneration, Faculty of Dentistry, Oral and Craniofacial Sciences, King’s College London, Floor 25, Guy’s Hospital, Guy’s and St Thomas NHS Foundation Trust, London SE1 9RT, UK.
Tel +44-2071885665 e-mail jadbinderpal.seehra@kcl.ac.uk

How to cite this article: Alsaif T, Pandis N, Cobourne MT, Seehra J. Does the quality of orthodontic studies influence their Altmetric Attention Score? Korean J Orthod 2023;53(5):328-335. https://doi.org/10.4041/kjod22.101

Received: April 20, 2022; Revised: July 13, 2023; Accepted: August 25, 2023

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Objective: The aim of this study was to determine whether an association between study quality, other study characteristics, and Altmetric Attention Scores (AASs) existed in orthodontic studies. Methods: The Scopus database was searched to identify orthodontic studies published between January 1, 2017, and December 31, 2019. Articles that satisfied the eligibility criteria were included in this study. Study characteristics, including study quality were extracted and entered into a pre-pilot data collection sheet. Descriptive statistics were calculated. On an exploratory basis, random forest and gradient boosting machine learning algorithms were used to examine the influence of article characteristics on AAS. Results: In total, 586 studies with an AAS were analyzed. Overall, the mean AAS of the samples was 5. Twitter was the most popular social media platform for publicizing studies, accounting for 53.7%. In terms of study quality, only 19.1% of the studies were rated as having a high level of quality, with 41.8% of the studies deemed moderate quality. The type of social media platform, number of citations, impact factor, and study type were among the most influential characteristics of AAS in both models. In contrast, study quality was one of the least influential characteristics on the AAS. Conclusions: Social media platforms contributed the most to the AAS for orthodontic studies, whereas study quality had little impact on the AAS.

Keywords: Altmetric Attention Score, Social media

INTRODUCTION

The impact factor is a citation metric that has been used to measure the impact of scientific journals in a specific field.1 Additionally, the contributions and performance of researchers, universities, departments, and research groups are evaluated using citation counts, which has also become a key factor analysed before awarding research grants.2 Despite the widespread use of citation metrics, such approach is flawed due to factors such as the lack of consideration of the reasons for the citation of an article, manipulation of citation patterns, and underestimation of the impact of recently published articles because of the delay between the publication of the study and its first citation and indexing in a citation database.2-4

Social media platforms have revolutionized the dissemination of studies’ findings, allowing researchers to explore new approaches to publication accessibility for stakeholders and policymakers.2,4,5 Altmetric was introduced in 2010 and has been employed to measure the attention or social impact that publications attract on the social web.6 Altmetric Attention Score (AAS) (Altmetric LLP, London, UK) have been proposed to overcome the shortcomings of traditional journal citation metrics by facilitating publication updating in real time and providing a broader portrait of the influence of an article.7,8 The AAS is a weighted count of all the sources of online attention that a research article or researcher may receive. The sources that contribute to the AAS are public policy documents, blogs, mainstream media, citations, online reference managers, research highlights, post-publication peer-review platforms, social media, Wikipedia, Open Syllabus Project, patents, multimedia, and other online platforms (Altmetric LLP). Their increasing relevance is highlighted by several publishers of academic journals, including Elsevier, Oxford University Press, Springer, and Wiley, having recognized their use as a measure of impact.8 Furthermore, Altmetrics are being used by research funding bodies as an alternative to traditional citation metrics to assess research use.9 Articles with a higher AAS seem to correlate with a higher social impact.10,11 Social media platforms such as Twitter have been utilized to disseminate the findings of orthodontic-related studies.12,13 However, such platforms appear to be under-utilized, and improvements in accessibility for both scholars and non-scholars are required.14 Factors related to social media platforms like journal accounts5 and posts15 can influence whether an article will be shared on social media.

Despite its advantages, the AAS weighting surprisingly does not consider the quality of each study. The practice of evidence-based dentistry is advocated and strengthened by the use of high-quality studies to inform healthcare decisions.16 Study quality involves the assessment of the risk of bias (RoB), with various tools available for the evaluation of different study types. Both citation rates and journal impact factor have been associated with AASs.17 However, citation counts do not necessarily correlate with study quality.18 Therefore, the aim of this study was to determine whether an association between orthodontic study characteristics and their AAS exists; the null hypothesis was that study quality had no influence on the AAS.

MATERIALS AND METHODS

Eligibility criteria

Orthodontic studies with an AAS published between January 1, 2017, and December 31, 2019, were identified. Studies published in non-English languages, before January 2017 or after December 2019, as well as those unrelated to orthodontics were excluded.

Search and selection of studies

A search of the Scopus database (www.scopus.com) using the term “orthodontics” was undertaken by 1 author (TA) on July 9, 2020. The following search filters were applied: publication date (January 1, 2017, to December 31, 2019), language (English only), and journal title. The database search was performed on two occasions, four weeks apart, to evaluate changes in both the number of article citations and AAS; if a change was identified, the average score was recorded. All titles and abstracts were screened by the same author (TA) on both occasions.

Data extraction

All study characteristics were extracted by a single author (TA) and entered into a pre-piloted Microsoft Excel® (Microsoft, Redmond, WA, USA) data collection sheet. A second author (JS) cross-checked all variables from the extracted data independently to ensure the consistency of the data collected. The following characteristics were extracted from each study: year of publication, country of correspondence of the author (categorized as Europe, America, Asia, and rest of the world), journal title (categorized as American Journal of Orthodontics and Dentofacial Orthopaedics [AJODO], Australian Orthodontic Journal [AOJ], Journal of Orthodontics [JO], European Journal of Orthodontics [EJO], Journal of Clinical orthodontics [JCO], Journal of Orofacial Orthopaedics [JOO], Angle Orthodontist [AO], Orthodontics and Craniofacial Research [OCR], and other orthodontic journals), number of authors, study classification (appliances, diagnostic studies, materials, devices for patient use, software, pharmaceutical and others —including scanner, radiographic equipment, dental unit, curing light, laser system, and whitening systems), number of citations (reported citation counts obtained from [www.scopus.com]), AAS (Bookmarklet tool [www.almetrics.com]), social media platform that the article was shared on (categorized as Facebook, Twitter, Mendeley, Facebook and Twitter, or Multiple platforms [www.altmetric.com]), and impact factor (as reported by the Scimago Journal Rank). The quality of each article was assessed using a previously validated and predetermined set of criteria and graded accordingly (Table 1).19

Table 1 . Predetermined criteria used to assess study quality.

GradeCriteria
A (high value of evidence)All criteria should be met:

Randomized clinical study or a prospective study with a well-defined control group.

Defined diagnosis and endpoints.

Diagnostic reliability tests and reproducibility tests described.

Blinded outcome assessment.

B (moderate value of evidence)All criteria should be met:

Cohort study or retrospective case series with defined control or reference group.

Defined diagnosis and endpoints.

Diagnostic reliability tests and reproducibility tests described.

C (low value of evidence)One or more of the conditions below:

Large attrition.

Unclear diagnosis and endpoints.

Poorly defined patient material.

Adapted from the article of Bondemark et al. (Angle Orthod 2007;77:181-91).19.



Statistical analysis

Descriptive statistics were calculated for all study characteristics. To identify the influence of article predicator characteristics on the AAS, two machine learning algorithms were implemented: the random forest (randomForest package) and the gradient boosting machine approach (gbm package) in the R statistical software. The random forest20 model employs bootstrapping to create multiple copies of the original training dataset, fits a separate decision tree to each copy, and then combines all results to create a single predictive model. Gradient boosting21 operates similarly, however, trees are grown sequentially, and each tree is grown using information from previously grown trees. In all algorithms, Poisson regression was selected and was in agreement with the count nature of the dependent variable. All analyses were performed using Stata 16.1 (Stata Corp., College Station, TX, USA) and R Software version 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Criteria total of 586 eligible studies were identified (Figure 1). A 100% agreement was achieved between the two authors (TA and JS) for all the collected study characteristics. Within this sample, most studies with an AAS were published in 2018 (34.3%), with corresponding authors primarily based in Europe (41.5%). Studies classified as “others” (scanner, dental unit, radiographic equipment, curing light, whitening system, and laser system) most commonly possessed an AAS (58.0%). Twitter was the most popular social media platform to publicize the studies (53.7%). The most common study type was cross-sectional (23.2%). In terms of study quality, only 19.1% of the studies were rated as having a high level of quality, with 41.8% deemed as moderate quality. The mean number of authors was 4.5, and the mean journal impact factor and number of citations were 0.83 and 4.7, respectively (Table 2).

Figure 1. Flow diagram for the identification and selection of articles with an AAS.
AAS, Altmetric Attention Score.

Table 2 . Characteristics of articles with Altmetric Attention Scores.

Article characteristicsn = 586
Year of publication
2017212 (36.2)
2018201 (34.3)
2019173 (29.5)
Journal title
AJODO55 (9.5)
JO17 (2.9)
EJO32 (5.5)
JOO2 (0.3)
AO20 (3.4)
OCR16 (2.7)
Other443 (75.7)
Continent of corresponding author
Europe243 (41.5)
Americas189 (32.3)
Asia and rest of the world154 (26.2)
Study classification
Erratum1 (0.2)
Appliances95 (16.2)
Diagnostic studies45 (7.7)
Materials74 (12.6)
Device for patient use6 (1.0)
Software15 (2.6)
Pharmaceutical10 (1.7)
Other*340 (58.0)
Study type
Erratum1 (0.2)
Systematic review49 (8.4)
Systematic review with meta-analysis41 (7.0)
Randomized clinical trial26 (4.4)
Case-control20 (3.4)
Cohort80 (13.7)
Cross-sectional study136 (23.2)
Case series8 (1.4)
Case report44 (7.5)
Opinion (editorials/letters/notes)59 (10.1)
Narrative review74 (12.6)
In-vitro41 (7.0)
Qualitative7 (1.1)
Study quality
High112 (19.1)
Moderate245 (41.8)
Low229 (39.1)
Type of social media platform
Not shared40 (6.8)
Twitter315 (53.7)
Facebook and Twitter97 (16.6)
Multiple134 (22.9)
AAS
Mean5
Median (IQR)1 (3)
Impact factor
Mean0.83
Median (IQR)0.78 (0.65)
Number of citations
Mean4.7
Median (IQR)2 (5)
Number of authors
Mean4.5
Median (IQR)4 (3)

Values are presented as number (%)..

AJODO, American Journal of Orthodontics and Dentofacial Orthopaedics; JO, Journal of Orthodontics; EJO, European Journal of Orthodontics; JOO, Journal of Orofacial Orthopaedics; AO, Angle Orthodontist; OCR, Orthodontics and Craniofacial Research; AAS, Altmetric Attention Score; IQR, interquartile range..

*Scanner, dental unit, radiographic equipment, curing light, whitening system, and laser system..



Overall, the mean AAS of the samples was 5 and the average AAS per study characteristics is presented in Table 3. A higher mean AAS was evident for the following characteristics: articles published in 2017; articles published in Orthodontics and Craniofacial Research; articles with authors based in the Americas; studies classified as pharmaceutical; cohort-type studies; and studies rated as high quality.

Table 3 . The Altimetric Attention Score per study characteristics (n = 586).

Study characteristicsMeanMedianIQR
Year of publication
20176.413
20184.213
20194.312
Journal title
AJODO3.113
JO2.514
EJO4.54.57
JOO446
AO3.724
OCR5.814
Other5.412
Continent of corresponding author
Europe5.214
Americas5.712
Asia and rest of the world3.912
Study classification
Erratum110
Appliances5.114
Diagnostic studies4.314
Materials3.411
Device for patient use3.525
Software2.611
Pharmaceutical7.224
Other*5.513
Study type
Erratum110
Systematic review7.726
Systematic review with meta-analysis6.157
Randomized clinical trial3.214
Case-control3.324
Cohort9.613
Cross-sectional study3.512
Case series110
Case report2.411
Opinion (editorials/letters/notes)3.712
Narrative review4.41.53
In-vitro5.511
Qualitative3.634
Study quality
High6.127
Moderate5.512
Low412
Type of social media platform
Not shared110
Twitter1.610
Facebook and Twitter3.422
Multiple1588

AJODO, American Journal of Orthodontics and Dentofacial Orthopaedics; JO, Journal of Orthodontics; EJO, European Journal of Orthodontics; JOO, Journal of Orofacial Orthopaedics; AO, Angle Orthodontist; OCR, Orthodontics and Craniofacial Research; IQR, interquartile range..

*Scanner, dental unit, radiographic equipment, curing light, whitening system, and laser system..



To fit the machine learning algorithms, two outlier observations (namely, 396 and 140) were deleted because the results were unreliable. The randomForest approach ranked ten predictors associated with the AAS, and the variance explained was 42.5%, suggesting that other important unmeasured Altmetric parameters exist (Figure 2). Table 4 shows the ranking (1, highest influence; 10, lowest influence) of the predictors based on the two approaches implemented. The type of social media platform, number of citations, impact factor, and study type were among the most influential characteristics of AAS in both models. In contrast, study quality was one of the least influential characteristics of the AAS.

Figure 2. Percentage decrease in accuracy by elimination of the ten predictors, one at a time, using random forest (n = 584).

Table 4 . Ranking of the predictors based on the two machine learning algorithms (randomForest and gradient boosting machine), in which 1 represents the highest and 10 the lowest influence.

PredictorRandom forestGBM (relative influence)
Social media platform11 (42.5)
Study type24 (6.3)
Impact factor33 (18.8)
Number of authors45 (5.6)
Year of publication56 (1.4)
Number of citations62 (23.3)
Study quality710 (0.02)
Journal88 (0.78)
Study classification97 (0.89)
Continent corresponding
author
109 (0.30)

GBM, gradient boosting machine..


DISCUSSION

AAS measures the amount of attention a study receives and is calculated using an automated algorithm that examines different sources in real time. Each source is weighted based on its relative impact. For example, if a study was mentioned in the news, this would be weighted more than a mention on Twitter or online reference managers. Based on this, the findings of the current study show that the greatest influence on AAS in orthodontic studies is the amount of attention received from social media platforms.

When comparing social media platforms, Twitter was the most commonly used social media platform in this study cohort (Table 1), which is similar to the findings of a previous report.5 Tweets are regarded as an effective method for sharing dental literature.22 However, Mendeley has also been reported as a popular social media platform to disseminate information to the community, and online attention to studies in terms of article access and citation counts has been correlated with downloads on Mendeley.13 The increased influence of social media platforms could also be a manifestation of publishers of orthodontic journals endorsing AAS as a measure of impact.8 Additionally, journals that possess social media accounts tend to have significant online attention compared to those without social media presence.23,24

The online attention of articles can include the number of times an article is accessed or downloaded, uploaded, discussed, bookmarked, cited, and recommended.25 In the current study, the number of citations and journal impact factors also strongly influenced the AAS for orthodontic studies; such finding is supported by the literature. In a systematic review of the associations between journal and article variables and AAS, both citation counts and journal impact factors were commonly associated with AAS.17 The accessibility and number of times an article was downloaded were beyond the scope of this study. However, free-access journals that facilitate access and downloading of articles tend to have higher online attention compared to subscription-based journals.23,26

Traditional metrics, such as the impact factor of a journal, do not necessarily correlate with the publication of high-level evidence within that journal.27 Within the current sample, approximately 19% of the studies were classified as having a high level of quality. Interestingly, the quality of the study was one of the characteristics that influenced AAS the least. This appears to conflict with the findings of a previous study, in which a high AAS of randomized clinical trials published in the field of total joint arthroplasty correlated with a high methodological quality.18 However, such study included a small sample of forty-two trials published in a single year, and potential biases during the assessment of methodological quality were not accounted for; notably, despite their reported findings, the authors concluded that clinicians should still critically evaluate studies before altering their clinical practice.18 AAS associated with orthodontic studies apparently cannot be considered a proxy for study quality.

In the current study, 586 studies which had an AAS were identified. As a study of this nature had not been previously conducted, ours represents a large sample size to explore the relationship between study quality and AAS. However, as solely a single database was searched (i.e., Scopus), language restrictions applied, and the broad search term “orthodontics” was used, we may have underestimated the true number of orthodontic studies possessing an AAS and, hence, the generalizability of the results may be limited. An assessment of study quality involves an assessment of the RoB in primary studies. We used a tool employed in a previously published systematic review. As described by the authors, this tool assesses both external and internal validity, the quality of the study methodology, and the statistical analysis performed.19 The study quality domains assessed in this tool are described in Table 1. The selected studies were classified into three broad categories. We acknowledge that to gauge a more detailed assessment of study quality, the use of specific RoB tools could be considered. Therefore, the results of this study should be interpreted with caution. Individual assessment of the 586 included studies using specific RoB tools was beyond the scope of this study, yet could be considered in a future study. Furthermore, the number of potential articles could have been increased by screening individual journal websites or by conducting a search via Medline via PubMed. The aim of the current study was not to precisely estimate the effect and precision of each predictor on ASS, but rather to provide initial insights into the AAS determinants, which can also be considered to answer other relevant questions. Articles were selected, and data extraction was primarily performed by a single author. However, to reduce possible biases, all articles included in the final analysis were independently cross-checked by a second author with complete agreement to ensure consistency.

CONCLUSIONS

In this exploratory cross-sectional study, social media platforms had the greatest influence on AASs in orthodontic studies. Among the study characteristics, the study quality had little impact on the AAS of orthodontic studies. Therefore, clinicians should critically evaluate the findings of these studies before implementing them in clinical practice.

FUNDING

None to declare.

AUTHOR CONTRIBUTIONS

Conceptualization: JS, NP. Data curation: TA, JS. Formal analysis: JS, NP. Investigation: TA, JS, NP. Methodology: JS, NP. Supervision: JS, NP, MTC. Writing–original draft: JS, TA NP, MTC. Writing–review & editing: JS, TA NP, MTC.

CONFLICTS OF INTEREST

No potential conflict of interest relevant to this article was reported.

Fig 1.

Figure 1.Flow diagram for the identification and selection of articles with an AAS.
AAS, Altmetric Attention Score.
Korean Journal of Orthodontics 2023; 53: 328-335https://doi.org/10.4041/kjod22.101

Fig 2.

Figure 2.Percentage decrease in accuracy by elimination of the ten predictors, one at a time, using random forest (n = 584).
Korean Journal of Orthodontics 2023; 53: 328-335https://doi.org/10.4041/kjod22.101

Table 1 . Predetermined criteria used to assess study quality.

GradeCriteria
A (high value of evidence)All criteria should be met:

Randomized clinical study or a prospective study with a well-defined control group.

Defined diagnosis and endpoints.

Diagnostic reliability tests and reproducibility tests described.

Blinded outcome assessment.

B (moderate value of evidence)All criteria should be met:

Cohort study or retrospective case series with defined control or reference group.

Defined diagnosis and endpoints.

Diagnostic reliability tests and reproducibility tests described.

C (low value of evidence)One or more of the conditions below:

Large attrition.

Unclear diagnosis and endpoints.

Poorly defined patient material.

Adapted from the article of Bondemark et al. (Angle Orthod 2007;77:181-91).19.


Table 2 . Characteristics of articles with Altmetric Attention Scores.

Article characteristicsn = 586
Year of publication
2017212 (36.2)
2018201 (34.3)
2019173 (29.5)
Journal title
AJODO55 (9.5)
JO17 (2.9)
EJO32 (5.5)
JOO2 (0.3)
AO20 (3.4)
OCR16 (2.7)
Other443 (75.7)
Continent of corresponding author
Europe243 (41.5)
Americas189 (32.3)
Asia and rest of the world154 (26.2)
Study classification
Erratum1 (0.2)
Appliances95 (16.2)
Diagnostic studies45 (7.7)
Materials74 (12.6)
Device for patient use6 (1.0)
Software15 (2.6)
Pharmaceutical10 (1.7)
Other*340 (58.0)
Study type
Erratum1 (0.2)
Systematic review49 (8.4)
Systematic review with meta-analysis41 (7.0)
Randomized clinical trial26 (4.4)
Case-control20 (3.4)
Cohort80 (13.7)
Cross-sectional study136 (23.2)
Case series8 (1.4)
Case report44 (7.5)
Opinion (editorials/letters/notes)59 (10.1)
Narrative review74 (12.6)
In-vitro41 (7.0)
Qualitative7 (1.1)
Study quality
High112 (19.1)
Moderate245 (41.8)
Low229 (39.1)
Type of social media platform
Not shared40 (6.8)
Twitter315 (53.7)
Facebook and Twitter97 (16.6)
Multiple134 (22.9)
AAS
Mean5
Median (IQR)1 (3)
Impact factor
Mean0.83
Median (IQR)0.78 (0.65)
Number of citations
Mean4.7
Median (IQR)2 (5)
Number of authors
Mean4.5
Median (IQR)4 (3)

Values are presented as number (%)..

AJODO, American Journal of Orthodontics and Dentofacial Orthopaedics; JO, Journal of Orthodontics; EJO, European Journal of Orthodontics; JOO, Journal of Orofacial Orthopaedics; AO, Angle Orthodontist; OCR, Orthodontics and Craniofacial Research; AAS, Altmetric Attention Score; IQR, interquartile range..

*Scanner, dental unit, radiographic equipment, curing light, whitening system, and laser system..


Table 3 . The Altimetric Attention Score per study characteristics (n = 586).

Study characteristicsMeanMedianIQR
Year of publication
20176.413
20184.213
20194.312
Journal title
AJODO3.113
JO2.514
EJO4.54.57
JOO446
AO3.724
OCR5.814
Other5.412
Continent of corresponding author
Europe5.214
Americas5.712
Asia and rest of the world3.912
Study classification
Erratum110
Appliances5.114
Diagnostic studies4.314
Materials3.411
Device for patient use3.525
Software2.611
Pharmaceutical7.224
Other*5.513
Study type
Erratum110
Systematic review7.726
Systematic review with meta-analysis6.157
Randomized clinical trial3.214
Case-control3.324
Cohort9.613
Cross-sectional study3.512
Case series110
Case report2.411
Opinion (editorials/letters/notes)3.712
Narrative review4.41.53
In-vitro5.511
Qualitative3.634
Study quality
High6.127
Moderate5.512
Low412
Type of social media platform
Not shared110
Twitter1.610
Facebook and Twitter3.422
Multiple1588

AJODO, American Journal of Orthodontics and Dentofacial Orthopaedics; JO, Journal of Orthodontics; EJO, European Journal of Orthodontics; JOO, Journal of Orofacial Orthopaedics; AO, Angle Orthodontist; OCR, Orthodontics and Craniofacial Research; IQR, interquartile range..

*Scanner, dental unit, radiographic equipment, curing light, whitening system, and laser system..


Table 4 . Ranking of the predictors based on the two machine learning algorithms (randomForest and gradient boosting machine), in which 1 represents the highest and 10 the lowest influence.

PredictorRandom forestGBM (relative influence)
Social media platform11 (42.5)
Study type24 (6.3)
Impact factor33 (18.8)
Number of authors45 (5.6)
Year of publication56 (1.4)
Number of citations62 (23.3)
Study quality710 (0.02)
Journal88 (0.78)
Study classification97 (0.89)
Continent corresponding
author
109 (0.30)

GBM, gradient boosting machine..


References

  1. Sud P, Thelwall M. Evaluating altmetrics. Scientometrics 2014;98:1131-43. https://doi.org/10.1007/s11192-013-1117-2
    CrossRef
  2. Huang W, Wang P, Wu Q. A correlation comparison between Altmetric Attention Scores and citations for six PLOS journals. PLoS One 2018;13:e0194962. https://doi.org/10.1371/journal.pone.0194962
    Pubmed KoreaMed CrossRef
  3. Merton RK. The Matthew effect in science. The reward and communication systems of science are considered. Science 1968;159:56-63. https://doi.org/10.1126/science.159.3810.56
    Pubmed CrossRef
  4. Garcovich D, Adobes Martin M. Measuring the social impact of research in Paediatric Dentistry: an altmetric study. Int J Paediatr Dent 2020;30:66-74. https://doi.org/10.1111/ipd.12575
    Pubmed CrossRef
  5. Asyyed Z, McGuire C, Samargandi O, Al-Youha S, Williams JG. The use of twitter by plastic surgery journals. Plast Reconstr Surg 2019;143:1092e-8e. https://doi.org/10.1097/PRS.0000000000005535
    Pubmed CrossRef
  6. Brigham TJ. An introduction to altmetrics. Med Ref Serv Q 2014;33:438-47. https://doi.org/10.1080/02763869.2014.957093
    Pubmed CrossRef
  7. Elmore SA. The Altmetric Attention Score: what does it mean and why should I care?. Toxicol Pathol 2018;46:252-5. https://doi.org/10.1177/0192623318758294
    Pubmed KoreaMed CrossRef
  8. Livas C, Delli K. Looking beyond traditional metrics in orthodontics: an altmetric study on the most discussed articles on the web. Eur J Orthod 2018;40:193-9. https://doi.org/10.1093/ejo/cjx050
    Pubmed CrossRef
  9. Knight SR. Social media and online attention as an early measure of the impact of research in solid organ transplantation. Transplantation 2014;98:490-6. https://doi.org/10.1097/TP.0000000000000307
    Pubmed CrossRef
  10. Melero R. Altmetrics - a complement to conventional metrics. Biochem Med (Zagreb) 2015;25:152-60. https://doi.org/10.11613/BM.2015.016
    Pubmed KoreaMed CrossRef
  11. Champieux R. PlumX. J Med Libr Assoc 2015;103:63-4. https://doi.org/10.3163/1536-5050.103.1.019
    KoreaMed CrossRef
  12. Hughes H, Hughes A, Murphy C. The use of twitter by the trauma and orthopaedic surgery journals: twitter activity, impact factor, and alternative metrics. Cureus 2017;9:e1931. https://doi.org/10.7759/cureus.1931. Erratum in: Cureus 2018;10:c13.
    Pubmed KoreaMed CrossRef
  13. Amath A, Ambacher K, Leddy JJ, Wood TJ, Ramnanan CJ. Comparing alternative and traditional dissemination metrics in medical education. Med Educ 2017;51:935-41. https://doi.org/10.1111/medu.13359
    Pubmed CrossRef
  14. Delli K, Livas C, Spijkervet FKL, Vissink A. Measuring the social impact of dental research: an insight into the most influential articles on the Web. Oral Dis 2017;23:1155-61. https://doi.org/10.1111/odi.12714
    Pubmed CrossRef
  15. Zhang D, Earp BE. Correlation between social media posts and academic citations of orthopaedic research. J Am Acad Orthop Surg Glob Res Rev 2020;4:e20.00151. https://doi.org/10.5435/JAAOSGlobal-D-20-00151
    Pubmed KoreaMed CrossRef
  16. Madhavji A, Araujo EA, Kim KB, Buschang PH. Attitudes, awareness, and barriers toward evidence-based practice in orthodontics. Am J Orthod Dentofacial Orthop 2011;140:309-16.e2. https://doi.org/10.1016/j.ajodo.2010.05.023
    Pubmed CrossRef
  17. Araujo AC, Vanin AA, Nascimento DP, Gonzalez GZ, Costa LOP. What are the variables associated with Altmetric scores?. Syst Rev 2021;10:193. https://doi.org/10.1186/s13643-021-01735-0
    Pubmed KoreaMed CrossRef
  18. Kunze KN, Richardson M, Bernstein DN, Premkumar A, Piuzzi NS, McLawhorn AS. Altmetrics Attention Scores for randomized controlled trials in total joint arthroplasty are reflective of high scientific quality: an altmetrics-based methodological quality and bias analysis. J Am Acad Orthop Surg Glob Res Rev 2020;4:e20.00187. https://doi.org/10.5435/JAAOSGlobal-D-20-00187
    Pubmed KoreaMed CrossRef
  19. Bondemark L, Holm AK, Hansen K, Axelsson S, Mohlin B, Brattstrom V, et al. Long-term stability of orthodontic treatment and patient satisfaction. A systematic review. Angle Orthod 2007;77:181-91. https://doi.org/10.2319/011006-16R.1
    Pubmed CrossRef
  20. Breiman L. Random forests. Mach Learn 2001;45:5-32. https://doi.org/10.1023/A:1010933404324
    CrossRef
  21. Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Statist 2001;29:1189-232. https://doi.org/10.1214/aos/1013203451
    CrossRef
  22. Kolahi J, Khazaei S, Iranmanesh P, Soltani P. Analysis of highly tweeted dental journals and articles: a science mapping approach. Br Dent J 2019;226:673-8. https://doi.org/10.1038/s41415-019-0212-z
    Pubmed CrossRef
  23. Hassona Y, Qutachi T, Dardas L, Alrashdan MS, Sawair F. The online attention to oral cancer research: an altmetric analysis. Oral Dis 2019;25:1502-10. https://doi.org/10.1111/odi.13111
    Pubmed CrossRef
  24. Thoma B, Murray H, Huang SYM, Milne WK, Martin LJ, Bond CM, et al. The impact of social media promotion with infographics and podcasts on research dissemination and readership. CJEM 2018;20:300-6. https://doi.org/10.1017/cem.2017.394
    Pubmed CrossRef
  25. Garcovich D, Zhou Wu A, Sanchez Sucar AM, Adobes Martin M. The online attention to orthodontic research: an altmetric analysis of the orthodontic journals indexed in the journal citation reports from 2014 to 2018. Prog Orthod 2020;21:31. https://doi.org/10.1186/s40510-020-00332-6
    Pubmed KoreaMed CrossRef
  26. Konkiel SR. What can altmetrics tell us about interest in dental clinical trials?. Dent Hypotheses 2017;8:31-3. https://doi.org/10.4103/denthyp.denthyp_12_17
    CrossRef
  27. Rodrigues MA, Tedesco ACB, Nahas FX, Ferreira LM. Journal impact factor versus the evidence level of articles published in plastic surgery journals. Plast Reconstr Surg 2014;133:1502-7. https://doi.org/10.1097/PRS.0000000000000214
    Pubmed CrossRef