A. Procedural Issues
1. In the period between March 2012 and May 2013, the Audit Team, set up by the Executive Committee of IREG Observatory on Academic Ranking and Excellence, conducted an audit of the Perspektywy University Ranking (Poland) produced by the Perspektywy Education Foundation.
The Perspektywy University Ranking has been evaluated following the procedures and rules adopted by the IREG Observatory and prescribed in the IREG Audit Ranking Manual (www.ireg-observatory.org) . The Audit Team has been composed of the following three independent experts who served in their individual capacity:
- Jamil Salmi, Global Tertiary Education Expert [former World Bank Tertiary Education Coordinator], Bogota, Colombia (Chair);
- Carla van Cauwenberghe, Coordinating Inspector Higher Education, Dutch Inspectorate of Education, The Netherlands;
- Patricia Georgieva, Managing Director, Qualifications and Quality Support Centre, [former Secretary General in the National Evaluation and Accreditation Agency (Higher Education)], Bulgaria.
2. The Audit Team reviewed the Self-Report on the basis of twenty criteria formulated in IREG Ranking Audit Manual which had been developed in reference to the Berlin Principles on Ranking of Higher Education Institutions. The criteria, with assigned numerical scores and weights, cover the following aspects of ranking:
- Criteria 1-3: pertain to purpose, target groups, and basic approach as well as to institutional diversity, linguistic, cultural, economic and historical context;
- Criteria 4-10: pertain to the methodology and the importance of ensuring that rankings choose indicators according to their relevance and validity, measure outcomes and are transparent;
- Criteria 11-14: pertain to the publication and presentation of ranking results;
- Criteria 15-17: relate to the transparency and responsiveness; and
- Criteria 18-20: relate to quality assurance of the ranking.
3. The Perspektywy University Ranking has been published annually since the year 2000 in order to provide information on how the Polish higher education institutions are assessed and perceived by various stakeholders. The rankings results and applied methodology are published at the same time by the Perspektywy magazine, the daily newspaper Rzeczpospolita, and online on the dedicated websites: www.ranking.perspektywy.pl (in Polish) and www.ranking.perspektywy.org (in English).
B. Substantive Findings
4. The Perspektywy University Ranking combines indicators based on hard data with indicators based on survey data for such criteria as academic reputation and employers’ perceptions. The number of indicators has grown from 15 in 2000 to 33 today. They are allocated as follows: academic prestige (4 indicators), research measured by academic potential (5) and academic effectiveness (8), teaching and learning (6), innovation (3) and internationalization. The methodology of the Perspektywy ranking has changed significantly over the years. The rankers have done a reasonably good job in signaling the changes from one year to the other and in providing explanations about the rationale behind the changes, except for the change in the weight of the teaching and learning indicators last year. The Audit Team is satisfied that the general approach has remained fairly consistent over the years. The Perspektywy University Ranking has established itself as a reliable source of information and assessment of performance of higher education institutions in Poland.
5. The Audit Team reviewed the Self-Report on the basis of the twenty criteria formulated in IREG Ranking Audit Manual and made the following observations:
- the ranking favors research results therefore it is desirable to put more emphasis on the assessment of the quality of teaching and learning;
- in order to measure the quality of education, four of the five indicators chosen were input measures (staff/student ratio, e-holdings, printed library holdings, and support of student scientific interests), and the fifth one (sports achievement) should be seen as less relevant. Therefore, more emphasis should be given to output and outcome measures;
- the recently introduced third mission of higher education performance (innovation)represented a rather small number of indicators as compared to research;
- concerning the surveys, information about the response rates and the structure of the questionnaires was only partially provided. It should be mentioned, however, that the Perspektywy Educational Foundation organizes seminars every year shortly after the publication of the rankings where they enter into a dialogue about the results with concerned institutions. The ranking is also regularly discussed during meetings of the Rectors’ Conference.
- while recognizing an overall coordinating and supervising role of the body called Kapitula, an independent quality assurance mechanism should be introduced in order to enhance the credibility of the ranking.
6. The Perspektywy Education Foundation accepted most of the proposals put forward by the Audit Team, in particular:
- giving up the indicator “sports achievement” with the 2014 edition of ranking;
- moving the results of external quality assurance activities to the “teaching and learning” group;
- increasing the information in English on its website (www.ranking.perspectywy.org) in order to offer detailed information about the academic and the employer survey to supplement the information currently is available on the website in Polish;
- accepting the suggestion of publishing the formal procedure for correcting errors after the rankings are published;
- introducing an independent quality assurance mechanism through invited experts.
7. The Foundation has also proposed that in order to avoid eventual linguistic misinterpretation which might occur during the audit process to include into the Audit Team an expert knowing the national language of the ranking concerned.
8. In addition, the Audit Team suggested that it would be useful, as part of the audit process, to have the chance of interviewing some of the users of the ranking in order to have an external view of the ranking.
9. The general assessment of the Audit Team is that the Perspektywy University Ranking has successfully established itself as a credible and transparent multi-dimensional ranking that seems to be widely respected in Poland. The Perspektywy Education Foundation has made consistent efforts over the years to improve the ranking methodology and relevance without substantially modifying its logic. The Audit Team recognizes the pioneering contribution of the Perspektywy University Ranking in the European higher education landscape.
10. The Audit Team concludes that the Perspektywy University Ranking (Poland) meets the criteria set in the IREG Ranking Audit Manual and recommends the Executive Committee to grant Perspektywy Education Foundation the rights to use the “IREG Approved” label in relation to the Perspektywy University Ranking (Poland) for the period ending on December 31, 2016. The Executive Committee of IREG Observatory extended this period to 31 December 2017.