A. Procedural Issues
1. The Executive Committee of the IREG Observatory on Academic Ranking and Excellence, at its meeting in Brussels on 4 April 2014, decided to grant the Centre for Higher Education (Centrum für Hochschulentwicklung; CHE) the right to use the “IREG Approved” label in relation to the CHE University Ranking for the period ending on 31 December 2017.
The IREG label is used [in printed and Internet presentations] strictly in connection with the ranking that has been evaluated in the course of the audit according to the procedures for IREG Ranking Audit which are defined in the IREG Ranking Audit Manual.
2. The Executive Committee of the IREG Observatory has studied the report prepared by the Audit Team which evaluated the CHE University Ranking as well as the response of the CHE. The Audit Team, set up by the Executive Committee of IREG Observatory, was composed of the following three independent experts who served in their individual capacity:
- Attila Pausits, Dr., Head of Department of Continuing Education and Educational Management, Danube University Krems, Austria (Chair);
- Bohdan Macukow, Professor, Head of the Department of Computer Science and Numerical Methods at the Faculty of Mathematics and Information Science, Warsaw University of Technology, Warsaw, Poland
- Dimitrios Noukakis, Dr., Office of International Affairs and Accreditation Ecole Polythechnique Federale de Lausanne, Switzerland
3. The Audit Team reviewed the Self-Report on the basis of twenty criteria developed in reference to the Berlin Principles on Ranking of Higher Education Institutions. The criteria, with assigned numerical scores and weights, cover the following aspects of ranking:
- Criteria 1-3: pertain to purpose, target groups, and basic approach as well as to institutional diversity, linguistic, cultural, economic and historical context;
- Criteria 4-10: pertain to the methodology and the importance of ensuring that rankings choose indicators according to their relevance and validity, measure outcomes and are transparent;
- Criteria 11-14: pertain to the publication and presentation of ranking results;
- Criteria 15-17: relate to the transparency and responsiveness; and
- Criteria 18-20: relate to quality assurance of the ranking.
4. The Centre for Higher Education (CHE) is an independent, private not-for-profit organization which was founded in 1994. The CHE defines itself as a think tank for higher education which is – based on international comparisons - engaged in developing models for the modernization of both, higher education systems and institutions. Its philosophy is based on the assumption that German university departments/faculties are not of the same quality but differ in profile and performance. Its first publication of its annual ranking appeared in 1998 covering two fields. To-day, the CHE ranking covers over 300 universities in 43 fields. Between 2004 and 2008, a process of internationalization took place with the inclusion of universities from Austria, Switzerland and the Netherlands.
B. Substantive Findings
5. The CHE applies a unique methodology; its ranking is a field-based, multi-dimensional tool using a grouping approach (top group, intermediate group, bottom group) which does not calculate a league table. It is important to note that no weights are attached to any of the indicators. Neither absolute scores are given to them nor an aggregation to an overall score is applied. In the view of the Audit Team this approach avoids false impressions of exactness and of differences between universities as given by league tables. Since 2001, the data are available online which underlines the main purpose of the CHE Ranking, namely to offer prospective students detailed comparative information leaving to them the decision which indicators they select and wish to rank.
6. The Audit Team observed that the set of indicators chosen for the different fields and the fact that those indicators are neither weighted nor aggregated match well with the claimed purpose of the CHE ranking. In connection with the process of internationalization since 2004 the Team drew the attention to the question whether different national higher education systems might imply differences in the definition and use of indicators.
7. The Audit Team stated that the methodological approach being developed in 1998 remained fairly consistent over the years. The methodology of the ranking which is explained and available online is well described and established. However, the relation of the number of input versus output indicators requires further theoretical reflection.
8. The publication and presentation strategy of results is highly appreciated by the Audit Team. The CHE Ranking is published annually (since 2005 in cooperation with the weekly newspaper DIE ZEIT which is recognized as the leading German newspaper in the fields of education, science and culture). All in all, the CHE developed a threefold publication strategy. Besides the presentation of selected results about the updated fields in DIE ZEIT a second publication is the annual study guide which contains primarily information for prospective students. Finally, the online version of the CHE Ranking should be mentioned again which contains the complete data set and results; it is free of charge and available in both German and English.
9. The Audit Team welcomed that the CHE Ranking is fully transparent with regard to the applied methodology and the information collected. Also, the option to correct data in the online version throughout the year as well as the advanced information of all participating universities and departments/faculties receiving a detailed analysis of their data are mechanisms which assure good working contacts with them. The Audit Team suggested that a formal procedure for the correction of data should be published.
10. The Audit Team fully shared the view that field-specific Advisory Boards plays an important role in its two meetings per year. The Team suggested that the composition of the board, which changes depending upon the fields being analysed/updated in the respective years, should also include representatives from abroad thus taking into account the process of internationalization which took place since 2004. The CHE already started to implement this proposal: in addition to the field-specific boards, the CHE set up a general advisory body in 2013 which also included a renowned member from abroad.
11. In its conclusions, the Audit Team underlined that the CHE Ranking has established itself as a credible and transparent multi-dimensional ranking that seems to be widely respected in Germany. The Team also appreciated the constructive dialogue and initiatives which CHE undertook in response to methodological and political concerns.