CiteScore-Based Quartiles for Scientometric Analysis

: This study intends to review and discuss some topics around CiteScore (here abbreviated as CS) which is a new citation impact indicator of Scopus. Qualitative analysis of journals based on scientometric indicators has been usual and is a good way of rapid evaluation of them. Here, we use them to reach a conclusion. We believe that reliability of CiteScore-based quartile is more and this index is preferred because in comparison to IF-based quartile, there are some reasons of preference of CS. In addition, CS can provide a more natural perception on citations compared to SJR-based quartile computation. The result of this study can be considered as a proof that both types of SJR- and CiteScore-based quartiles should not be assumed as baseline of quality evaluations at the same time.


Introduction
CiteScore is a new scientometric indicator (citation impact metric) introduced by Scopus, a product of Elsevier, in 2016 (Zijlstra and McCullough, 2016a) for tracking journals performance in terms of citation analysis. Before introducing CiteScore (CS), Scopus has had several science quality assessment metrics like Source Normalized Impact per Paper (SNIP) and Scimago Journal Rank (SJR) indicators. However, these indicators could not be strongly pervasive as rivals of Impact Factor (IF) by Journal Citation Report (JCR) related to Web of Science (WoS). Maybe the reason is to use much different definitions in all the prior Scopus indicators compared to the IF, which cannot illustrate a natural perception or normal concept of yearly citation per document to a researcher who is not expert in scientometrics. Fortunately, CS is able to give us a strong perception on the concept of citation per document because its definition is relatively near to IF, but with some further advantages. As Fig. 1, we see a transparent definition of IF, whereas CS details are observable in Fig. 2. These figures show that there are detailed differences in definitions of CS and IF. Now, we aim to have a brief review on advantages of CS in comparison to IF. IF has two forms of 2-year and 5-year. The main form is, of course, based on 2-year evaluation frame. On the other hand, CS is calculated for a term of 3-year. It seems that 2-year period is a little short (particularly for fields that are slow in availing newer scientific findings, e.g., many areas of engineering studies) and 5year is relatively long. However, 3-year term is much better and can provide sufficient time for most of research disciplines in order to count an effective number of citations, thus a 3-year value would be more accurate, adequate and fair.
The second issue is the broader and more comprehensive indexing, abstracting and citation tracking coverage of Scopus compared to WoS. WoS generally has less coverage on peer-reviewed scientific materials as abstracted (while it plays role of an indexing directory), and even all of its abstracted journal articles are not used for citation analysis, for example, specialized articles which are exclusively indexed and abstracted by Zoological Records database (it is not a citation database; more up-to-dated details of WoS should be reviewed at Clarivate Analytics website). Therefore, this comprehensiveness of Scopus can give us a more realistic citation analysis and consequently, Scopus citation impact indicators would be currently more reliable. A third issue can be discussed around new service of Scopus entitled "CiteScore tracker" as the monthly version of CS with interesting abilities for time series prediction problems in scientometrics. Scopus is updating CS monthly and reports CS tracker whereas there is no similar facility in WoS towards IF this time. And as a final remark, Scopus considers some offsets of IF in calculating CS, it is towards a fairness and makes CS more accurate. In fact, denominator of IF is restricted to just citable documents but citations to a journal may be related to all published items including its citable/non-citable documents. Because CS does not consider difference between citable and non-citable documents, this indicator has no offset (or bias dedicated to a specified journal/publisher). Besides, a research (Eigenfactor, 2019) shows that the IF definition provides an additional bias for some types of scientific journals and also some specific fields whether in terms of having offset or short time frame (for example, Nature group's journals which are published in the field of medicine are benefiting from this bias to reach higher IFs). According to these facts and an intellectual reasoning, we can prefer to use CS. Another issue may also be discussed is this disappointing fact that despite Scopus, WoS does not compute IF and then its Q index for many of WoS-indexed journals, out of WoS core collection or even for Emerging Sources Citation Index (ESCI) database from the core. This can be an important reason for more reliability of Scopus compared to WoS.
As the paper organization, in the second part, our methodology is presented and in the last part, discussions and conclusions will be given.

Methodology
Nowadays, qualitative analysis of journals based on scientometric indicators has attracted more attention. WoS has provided a classification based on statistical quartile concept using computations of median on journals with IF descending order, and for each subject category. Scopus has similarly classified all of the covered journals for assigning a quartile rank (Q index) through ScimagoJR database on which all journals are classified into four classes Q1, Q2, Q3 and Q4 (Q1 consists of top journals of the subject category) based on values of their SJR indicator.
Also, other periodical/serial publications including conference proceedings and book series may be considered for Q index, but some details exist which are addressed to (Zijlstra and McCullough, 2016a;Zijlstra and McCullough, 2016b;Scopus, 2016;Borrego, 2018) on which all types of serial titles (journals, magazines, book series, and conference proceedings having ISSN) can be allocated to a CSbased quartile, but continuously published conference proceedings are not currently evaluated for SJRbased quartile computations (even though SJR is computed for them). Also, scientific and trade journals along with book series can be classified by SJR-based quartiles. Using the qualitative ranking of Q index has two benefits; at first, perfect perception and interpretation of quantitative indicators (e.g., SJR) is difficult, therefore, the preference is to convert quantitative information (IF, SJR and CS) to qualitative results like all types of quartiles (IF-based quartiles in WoS, and SJR-based/CS-based quartiles in Scopus) in order to compare journals with each other, for example, "is a journal with SJR=0.45 high-quality?" surely response to this question is not easy. The second benefit is to attend to effect of different scientific areas, for example, an engineering journal with IF=3 may be Q1, but a medical journal with IF=4 may be Q2; these observations are due to various subject categories in Q index classification (amongst quantitative indicators, SNIP is also trying to create such benefit, but when we are hearing SNIP value of a journal, we do not interpret it like IF or CS towards the concept of citation per document). In general, all types of Q are inherently normalized per subject category. In addition to quartile-based classification, there are other kinds of qualitative rankings, Scopus provides percentile-based classification for all indexed journals based on CS values. This classification can give us the second benefit, but might not be well-interpreted for the first one. This is because percentiles are not as sound as quartiles on the viewpoint of qualitative inference. For instance, suppose there are 120 journals in a subject category which should be compared with each other. A journal is directly ranked as 106 th from 120 (descending order; lower direct ranks have higher values of numerical indicators and indicate better quality). Its percentile would therefore be 11 th .
It is explicit that descriptions of 106/120 (fully quantitative view) and Q4 (qualitative view) are more descriptive than the 11 th percentile. Therefore, we believe that quartiles are better than percentiles for quick description of journals' quality. Scopus has determined a formula for computing CS-based quartile (Zijlstra and McCullough, 2016a;Zijlstra and McCullough, 2016b;Scopus, 2016;Waltman, 2016), but its results are currently not given directly by its main database. If we wish to compute CS-based quartile for each journal in order to have all advantages of CS in comparison to other numerical indicators and also, all the benefits of quartile classification (especially compared to SJR-based quartile because we know both are using Scopus scientific coverage and are similar in this view), a formula can be derived as follows. The formula is another description of a formula based on statistical computations provided by Scopus and makes a connection among direct rank, percentile rank and quartile rank with applying ceiling function. Assume that a journal based on CS information is ranked m th among all n journals existing in that category (normally n>>1). Thus, CS-based percentile and quartile are specified as below. 0 th percentile class (or rank) includes the lowest quality journals, with a CS less than the first statistical percentile value. On the other hand, 99 th percentile and Q1 classes include the best journals (definitely accuracy of percentile is higher). Scopus already provides the fraction of m/n and the percentiles directly based on CS. In ScimagoJR (a part of Scopus), in addition to m/n, quartiles are provided directly based on SJR.
For n<100, computation of percentile is not suitable and can be without importance, so in these cases, just a quartile computing is sufficient. However, the equation is able to compute P, and based on this computed P, we can calculate Q. Normally, while having n<100, some high-rank classes of P like 99 th , 98 th , etc., include no journal. Also, in general form for any n, it is possible that the worst direct rank of journals in a category becomes n-1, n-2, n-3, etc. (less than n), this happens because some journals have a similar value of CS or SJR, and therefore, they achieve the same direct rank; in these cases, some lower percentile ranks (0 th , 1 st , etc.) may not contain any journal. As a final point, we should say that this definition may be a little different from the Scopus definition for percentiles because of there is no unique definitions for it in statistics, however our definition is simply computed. Just in less number of journals in a specific subject category (much less than 100), we might accidentally observe a difference of 1 or 2 for percentile value between our definition and Scopus's formal definition on which in rare cases it may affect Q ranking, however in practice number of journals are not often less than 100, thus we can ignore this difference approximately.

Analysis and Conclusions
As a consequence, we believe that reliability of CS-based quartile is more and this index preferred (read a similar work about journal reliability in (Khosravi, 2018)), because: -In comparison to IF-based quartile, we discussed some reasons of preference of CS compared to IF. -In comparison to SJR-based quartile, CS can provide a more natural perception on citations. Moreover, investigation in case studies shows that there are differences in results of CS-based and SJR-based quartiles for many journals. In Table 1, details of CS and SJR are observed for Library Hi Tech News (LHTN) journal. The table shows that in this journal for a specific year and a selected subject category, quartiles extracted based on SJR and CS may be different from each other, although both are communicating citation statistics of Scopus. In addition to the difference in supporting conference proceedings, some of other concurrent resources of SJR have not been covered by CS (see denominators in direct CS and SJR ranks). This can be considered as a proof that both types of quartiles should not be assumed as baseline of quality evaluations at the same time, it is better to select only one of them.