Insights Xchange: Conversations Shaping Academic Research

Phil Baty on the Times Higher Education World University Rankings

• ScienceTalks • Season 2 • Episode 6

Share episode

In today's episode, Nikesh Gosalia speaks to Times Higher Education (THE) Chief Knowledge Officer Phil Baty about university rankings. Phil talks about THE’s origins and his personal history within the company and delves into the THE World University Rankings and shares insider info on the inner workings of the rankings — he covers everything from performance indicators to the incorporation of the UN's Sustainable Development Goals. Phil addresses some controversies about ranking tools and reiterates the importance of transparency in creating beneficial ranking systems. Next, Phil discusses the key trends in university rankings, including the shift in power from the West to East Asia. He shares his tips for universities to improve their rankings and touches on the consultancy services offered by THE to help universities improve their reputation, including THE’s newly acquired business ‘The Knowledge Partnership’.

Phil Baty is an established expert in university performance and strategy. With 25 years of experience in global higher education, including a decade as editor of the Times Higher Education World University Rankings, Phil is an award-winning journalist and highly sought-after speaker. He is also the creator of the THE’s World Academic Summit. Phil can be reached on Twitter and LinkedIn. 

Insights Xchange is a fortnightly podcast brought to you by Cactus Communications (CACTUS). Follow us:

Hi everyone. Welcome to All Things SciComm! Today’s guest is an international authority on university performance and strategy. He has 25 years of experience in global higher education, including a decade as editor of the Times Higher Education World University Rankings. He is an award-winning journalist and highly sought-after speaker. He is the creator of the THE World Academic Summit and oversees the team that organizes the event, which engages tens of thousands of senior university leaders, business executives, and policymakers annually. Everyone, please welcome Phil Baty. Thank you very much for having me, it's a pleasure. Can you tell those who don't know, what is Times Higher Education? If you can talk a little bit about your history within the company, what kind of roles have you played, and what were you responsible for? Thank you very much. Well, that's a very big question. We describe Times Higher Education actually internally as a 51-year-old startup. We have a heritage that goes back to 1971, so we just recently celebrated our 50th birthday. But really, our heritage goes back to being a weekly newspaper for primarily the UK higher education sector. We were a spinoff of the Times of London. We were originally called the Times Higher Education Supplement. We were a supplement of The Daily Times, focused on higher education. It was all about content and traditional media and some job advertising primarily in the UK. But we really dramatically changed our outlook, changed, our vision, and today we really see ourselves as a global data and insights business. I think it all changed in around about 2004 when we first published the World University Rankings. That became an extraordinary global phenomenon. There was a huge gap and a real appetite for people to understand comparative data to compare universities cross-borders in what was rapidly emerging as a global higher education sector, a global community with increasing global mobility, increasing global research collaboration. Everything changed really from 2004. We moved dramatically into being global. We moved dramatically into being data-driven in our approach. That's the origins of THE. Today, we are the largest platform for university professionals globally. We recently acquired Inside Higher Ed in America. So, between ourselves and Inside Higher Ed we have around about 50 million people come to our websites every year. They are all professionals, academics in higher education, and some students who look at rankings as well. We run international events. We have a consultancy business. Our core mission of course around data and analytics and insights is still very much a part of who we are and the driving force of who we are. My personal journey started in 1996 when I joined Times Higher Education as a very junior news reporter. I had actually had an even longer association with Times Higher Education, I was a freelance writer as a student in the mid-90s. But I joined in ’96 as a junior reporter. I just fell in love with the higher education sector. Writing news about the sector was inspiring. It was varied. Universities are the greatest force for good in the world, as far as I am concerned, so it was a pleasure and honor to report on higher education and to understand the higher education sector. I've just stayed at Times Higher Education for pretty much my whole career, more than 25 years now, in varying different roles. Now, of course, I am the Chief Knowledge Officer. My responsibilities include a lot of our public engagement. Recently, a lot of our work are with governments, we do a lot of consultancies with governments. As you mentioned very kindly in your introduction, I created the World Academic Summit and the World Summit Series, but I also look after our international events business. And that is a wonderful global community of academic leaders administration and academic leaders who connect across the world for various summits and forums to share good practice to meet network and really to drive forward the success of universities globally. It’s a huge honor to have the role I've got and a huge privilege to have been working in the sector for such a long time. I often pinch myself that I have such a wonderful, privileged position in higher ed, and I am incredibly lucky and incredibly grateful to have this opportunity. I guess everybody within the industry and perhaps even outside, so aspiring students, research institutes, universities, governments really look forward to the THE University Rankings. It is highly prestigious. Could you talk a little bit about how do the university rankings work, and what are some of the critical datapoints that you are tracking for these universities? It’s very important that rankings are transparent. We must understand the data that goes in them, and that they are not really very valuable unless we're able to be be very, very transparent about the methodology. So, of course, happy to explain. World University Rankings, the flagship Times Higher Education ranking is based on 13 separate performance indicators, which we think cover the whole of the core missions of a research-intensive global university. We have five metrics looking at what we call the teaching environment, that's looking at teaching factors, factors that will influence teaching such as faculty-student ratios, resources, reputation. We look at research in a very in-depth way. We look at research reputation. We surveyed 30,000 scholars around the world this year for our reputation surveys. But also, when we are evaluating research, we evaluated 15 million research outputs and more than 100 million citations to those outputs to understand the influence, the reach, and the quality of each university's research. It’s a huge data undertaking. But as well as teaching and research, we also look at international outlook, so a university’s connections with the international community through research collaboration. We look at the ability of universities to draw in international talent, both among faculty and staff as well as students. And then we look at industry links as well. We look at a university's connections to the sort of so-called outside world of business and industry, their ability to attract funding to R&D projects from the private sector, from industry. We think it's the most comprehensive, the most balanced evaluation system. The Times Higher Education World University Rankings give you a very holistic, big overview of a university. A lot of rankings can be much more narrow, much more focused on specific areas. We give the holistic view, which is actually fantastic for the opportunity to evaluate and monitor performance at the institutional level but also of course at the national level. We also have other rankings. We believe there is no single correct version of excellence. There's no right or wrong answer when it comes to excellence. Different metrics measure different things. We also look at a university’s social and economic impact. We have what we call the impact rankings. They use the sustainable development goals as the framework for evaluating university's impact. We look at that in four aspects. We look at a university's research, their ability to bring new ideas, new thinking, new knowledge in driving forward the Sustainable Development Goals. We look at their teaching again, provision of skilled graduates and their activities in the fields of each of the SDGs. We look at their stewardship, their management of their own resources and assets, their people, their campuses, make sure they are doing that in a sustainable way. Also, we look at their outreach to the wider community, to governments, to businesses and to charities. We have a huge array of metrics for a very, very wide array of institutional activity. We have many, many millions of datapoints. And most recent World Rankings, we had more than 2500 submissions. We should be ranking about 1800 universities in the next ranking in October 2022. We are sitting on this wonderful data resource, allowing us to see universities in the world. The amount of work that goes behind getting these rankings in place, I mean that's just phenomenal. Since you mentioned SDGs have grown in prominence over the last couple of years like never before. But fair credit to THE, I think you all already perhaps anticipated it is really, really going to be extremely important for everybody in this world to look at the SDGs. Can you talk a little bit about how did the process work, how did you all go about anticipating and coming up with those rankings? How do these Sustainable Development Goals play a role in the rankings? Yeah, of course. Obviously, we've been producing the world rankings since 2004. They had become a gold standard as a way of understanding excellence in a rather traditional way, excellence in terms of world-class research, world class reputation for academic excellence. It tended to be a ranking that privileged those very wealthy, elite North American and Western European schools. It was Harvard, and Stanford, and MIT and Oxford, and Cambridge, making the top echelons. It is quite an Anglo-Saxon model of excellence that university perfected in this post-war era in the United States. The world rankings do sort of measure a particular type of institution, but we understand at Times Higher Education, because we're global, because we work within the sector deeply. We understand that excellence can come in many forms, many shapes and sizes, and inequalities of wealth, brain drain, shouldn't be a way of detracting from the great work that universities can do in more challenging conditions with less resource. Of course, we also found that governments around the world wanted to understand what they are getting in return for their investment in universities. How much of it is esoteric research? That’s great for scholarship and learning, but governments, taxpayers want to know what impact are our universities making? Are they changing the world, are they driving the economy? For several years, we've been under pressure to say, how do we measure impact, how do we measure the impact universities make to society? In many ways, that's often looked at as economic impact, do they have spinoff companies? Do they create intellectual property that gets converted into a product that makes money? Do they have licenses and IP? But we also wanted to understand impact in a much more profound way. Universities change society. They fuel democracies. They create engaged global citizens. They fuel the economy through skilled workers, but also, they addressed some of the world's biggest challenges. Climate change cannot be solved without universities inventing things, developing new technologies, pushing us into alternative energy sources, constantly innovating. The pandemic showed us as well that universities will be the key to getting humanity out of these tremendously challenging awful situations through rapid development of vaccines. We draw upon the great global body of knowledge that was already there in terms of understanding pandemic and the Coronavirus variant that hit the world. We've been seeing this pressure to understand impact for many years and we finally cracked it actually after many years of scratching our heads, when Duncan Ross, our Chief Data Officer realized that the Sustainable Development Goals provided almost a perfect framework to understand impact. They are all agreed by the global community. They are all understood to be incredibly pressing. They are all interrelated. If you address quality education, you can then bring sustained citizens and address issues of inequality through education. If you address gender, you can bring greater talent and make sure we are fully utilizing the talent we have in society. Through universities we can drive economic growth and innovation. Through universities we can tackle climate change. It was when we aligned on the SDGs as the framework, which was probably in sort of 2017-2018, we realized that that was the way of framing impact in the most broad, multidisciplinary way we can. And then it was a question of almost working with the UN goals in terms of what the indicators are, how we would measure that. We had a long process of consultative conversations about how to achieve that and we were very, and we were very, very ahead of the game in the sense that we published this ranking in 2019. It's exploded in engagement. The numbers of participants are exploding every year. I think it really just hits the right moment that the world was looking to universities to demonstrate impact. The universities were looking at ways of proving their impact to their taxpaying public, to the governments and policymakers. We also wanted to sort of help universities celebrate the impact they make. The thing that’s so powerful about the impact rankings I think is the diversity of excellence. The world rankings have just the US and UK in the top 10. The impact rankings have many more countries. They have countries in the Middle East, countries in East Asia, as well as Western European, North American countries, so the impact rankings show a much greater diversity of excellence. And universities are embracing them to say, we want to demonstrate we care about sustainable living. We want to show the world we have great impact, even if we don't always have vast wealth and resource. I think what's interesting as well, students now are saying we want to test and challenge the universities to make sure they are committed to this agenda. We want to challenge them, make sure they are willing to be accountable for their contributions to delivering sustainable development. It's been an incredibly exciting and humbling process. But now we do have this unique dataset, an utterly unique way of understanding a university's impact on the society across all of the 17 individual goals. It's an incredible resource. We want to try and turn that resource into action, supporting collaboration, supporting investment, helping universities find one another, helping funders of the SDG agenda to find pockets of excellence in higher education, helping individual scholars of course to promote themselves and give themselves visibility for contributing to this agenda. It's been an incredibly exciting process and it was a long time in development. But we are delighted now. We seem to have really hit one of the most important issues facing universities today which is proving impact and demonstrating their sort of social commitment to making the world a better place. Absolutely Phil. I fully acknowledge that. I remember talking to one of my colleagues who in his previous life I think worked as an Impact Manager for one of the universities. And the first year when the SDG Impact Rankings were out, I think we were discussing that we'll monitor how this goes, we'll see the number of participants, and yeah, I think this sounds interesting. But the very next year, like you correctly mentioned, the number of participants and the whole framework and the rankings just exploded. Like you said, it's not easy work, there was a lot of years of work. But just so perfectly timed and such a useful set of framework that's out there now. Just going back on the whole issue around rankings, Phil, Phil, I am sure you've kind of heard both sides to the whole ranking and the mechanism and the framework. In your opinion,why do rankings matter and how should we look at them? Yeah, sure, I mean, obviously, they can be very controversial. They do provoke heated conversations. If we're really honest, rankings do reduce universities and all their wonderful complexity to simple numbers. They aggregate scores, can often hide nuance, hide pockets of excellence. The available data, the available metrics can often miss really amazing things universities do. Of course, rankings can be inherently simplistic, inherently crude, so they are by no means infallible. I think what's exciting about ranking is their potential to genuinely provide information that doesn't exist, and insights. It's actually very interesting that the origins of the Times Higher Education ranking actually came not from the demand for consumer information for students, they actually came from the government. There was a government report by the former Head of the Confederation of British Industry who wrote a report for the UK Treasury back in 2003 called the Lambert Review. The Lambert Review said, we need to understand our investment in research in British universities. How does it play out globally? British universities compare themselves domestically. They have no mechanism for understanding themselves against the rest of the world. They shouldn't be competing domestically, we should be competing globally. We wanted a mechanism for evaluating the strength of our universities, the competitiveness of our universities on a global stage and really our position in the global knowledge economy. The original driver came as a tool for governments to provide policy insight, policy development opportunities. That's how we originally built the Times Higher Rankings. We actually were serving first the needs of the university sector wanting information about their strategy, information about their strengths and weaknesses, information about their performance relative to the world on a world stage. It was only really after that, that the students embraced this as a fantastic tool to help them inform decisions. We had an explosion of international student ability over recent decades. Students are increasingly looking to the world for their education, investing really serious money in the most important decision of their lives, who to trust with their higher education, which university to put your time and engagement in. Then they became a massive consumer tool after that. We get tens of millions of students coming to our site to look at the rankings to help them make decisions. The point is, rankings provide transparency. They provide genuine data and information and insights that are otherwise unavailable. They inform students and their families around making sensible choices. They democratize the university decision-making process for students and families. But more importantly, I think for Times Higher Education in terms of who we engage with, they provide genuine insights to the people who lead and run universities about how their universities are performing. They provide genuine insights increasingly to governments. We are finding governments around the world who want to work with us to understand and help them to develop and improve their university system performance, and to look at the size and shape of university systems, and to support the development of higher education for the benefit of society. I feel very, very passionate that rankings play a really, really important and really, really positive role in the world. But of course we have to make sure that we look at rankings in a critical way. Some rankings are much more superficial than others. Some rankings privilege certain factors that might have perverse incentives, how exclusive a university is, how high the barriers to entry are, which might encourage universities to close doors rather than open doors. We have to be very, very mindful of perverse incentives, very, very mindful of rankings being too narrow or too superficial. But as long as we are clear and transparent on methodology, and we can break the data down, and share that data in the right format, I think the rankings provide the framework of an incredibly important way of understanding the global knowledge economy and helping people make informed decisions. That's the sort of driving force. I accept and welcome constructive criticism. I understand some ranking organizations are more exploitative, more superficial, create rankings more as a marketing tool, but we feel our rankings are based on incredibly rigorous research and data and provide a really powerful tool to the global higher education community to make better decisions for the good of the sector and the good of society. I agree with you. Considering the fact that there are just so many data points that go into the ranking, just so much of useful information, and we clearly need those rankings. Staying on with talking a little bit about the datapoints and the fact that you explained Times Higher Education goes into so much of detail to put together the rankings. Are there some common trends that you've noticed through this data that might be worth sharing with the listeners? I think in terms of trends, there's definitely a sense that’s clear in the rankings over time that the world is changing. I think we have seen what should be seen as a very welcome shift in the balance of power, if that's the right way of phrasing it. Traditionally, in the world rankings the West has dominated, as I mentioned earlier, North America in particular. Western Europe have dominated the rankings. They tend to have had the benefits of brain drain from other parts of the world. The US in particular has obviously led the global higher education community as a magnet for talent as leading institutions with very, very high levels of funding through controversial issues like high tuition fees, etcetera. But I think we're seeing a shift in the balance of power. We have absolutely seen over time a very clear rise of East Asia in particular. China has been leading that. It's interesting that nations have been able to make significant improvements to their university systems, and they've seen them really powering up the rankings. China, I think had two universities in the top 200 back in 2016. They've now got 10. And if you look at the trend lines in terms of scores, China and America are starting to get much, much closer. The US can't rely on its dominance forever. It is being challenged by other nations. That's a really major theme. And it's not just China. You've got Hong Kong, South Korea, Singapore, also doing incredibly well. So that East Asia rising, the balance between East and West changing is a major trend. It's about nations investing in knowledge capacity. China wanted a shift towards – partly China wanted visible dominance and national pride that comes into play, top ranked universities, but actually it's about driving the knowledge economy. They were shifting their economy very much from manufacturing to knowledge and skills. And same in South Korea, dramatic changes to the shape of economy driven by outstanding universities and great research and great innovation. So, that's a major, major trend. We are also now seeing I think that coming through in the Middle East. In Saudi Arabia, United Arab Emirates, they are really building research capacity and getting very, very ready for the post-oil economy, digital transformation, skills innovation, hugely important to those nations’ future success. Saudi Arabia and UAE are doing very well in rankings, rising up quite rapidly. Actually, some of the North African nations, Egypt I think is doing very well as well, having some great success. Pakistan's engagement is improving. And of course India, I think some way behind China in terms of reform, but the educational reform is coming through for India, that the focus on building capacity in India is also very, very exciting. The most exciting thing for me in rankings trends is the reduction of any real dominance of any one particular nation or one particular culture for a much more equal and much more diverse global knowledge economy, which I think is good news for the whole world and good news for those Western nations as well, because the sources of knowledge diversify. It's the metaphor that, a rising tide lifts all boats. It doesn't have to be – rankings are a zero-sum game in the way they are formulated, but actually strengthening in other areas doesn't necessarily mean other areas of weakening, it just means that, as I say, a rising tide lifts all boats. So, that's an exciting, big development I've seen in recent years. Just to understand that better, I am sure there are a lot of universities in different countries, different geographies, who are aspiring to improve their rankings. Typically, what kind of advice would you give to these universities on ways to improve? Yeah, I mean, it very depends on individual missions of universities. But fundamentally, I think to rise up the Times Higher Education World Rankings your strongest area will be to address research. So, building research capacity, strengthening your research base. So, a lot of that might be really bringing on more and more Ph.D.’s, collaborating more internationally with other research partners, just strengthening in research excellence. I always think actually some of the lower weighted areas in rankings are actually some of the most powerful and driving improvements in performance and they are the international metrics. Actually, we have a metric that rewards collaboration across borders and research and we find that internationally collaborative research is more highly cited, it is more widely disseminated, so that strengthens the visibility of research, strengthens the citation. But also inherently improves the research base by sharing and collaborating with other scholars in other parts of the world, bringing diverse ideas to the fore, capitalizing on strengths outside the country. So, that's really powerful. Attracting international talent again brings in additional voices and talent so the community rises up in terms of the specific metric and rankings but also improves those connections and that visibility. I do think there's a really important role for communications though. A lot of work is just around how do you better communicate your research excellence. How do you disseminate the research you are doing in the most effective way? A lot of academics can be skeptical about marketing, and PR, and comms. But actually, it's a highly competitive, busy world out there. So, if you've got scholars doing great work, you do need to find ways of communicating that. How do you disseminate that most effectively? How do you get that research noticed? It’s not just all about PR, it's about genuinely sharing your discoveries. How do you expose your discoveries to the widest audience? How do you make sure your incremental additions to knowledge are seen by other scholars and built upon by other scholars for the good of the research base and the good of society? I do think universities could look at how they use social media, how they engage with the mainstream media, how do they find ways of communicating and disseminating in the right journals and often outside and beyond journals. It's an important part of the process. Because obviously, reputation again is also a big chunk of the ranking. We survey 30,000 scholars for the World University Rankings each year, and we ask them who they believe are doing great things. That's based on their subject knowledge. But it's often based on papers they've read, conferences they have attended, social media accounts they follow, how they absorb information about who is doing great work in their community. Reputation really matters. Dissemination, communication, marketing, public relations, I think does really matter. But fundamentally, you won't rise in the rankings unless you are really producing great research and high-quality research that gets disseminated in the right way. I think mission specific, one of the things Times Higher Education does is we have the ranking itself and the five pillars that go into the ranking in the public domain, but we can break down the data to the 13 metrics and work with universities. So okay, which of these areas do you have the greatest weaknesses compared to peers? You can select peers, select countries, benchmark yourself across all the 13 different indicators to find out where you have particular strengths or weaknesses, and you can address the weaknesses. One of the things I love most about the data and analytics we do for universities is we often do a simple SWOT charter: strengths, weaknesses, opportunities, and threats, where we map the hard metrics of research excellence, the citation analysis, against the soft metrics of research excellence, the reputation. And a lot of universities are doing outstanding work in particular fields of research that just isn't widely understood. You have a very strong bibliometric citation performance for a particular field but a very low reputation. We can actually work with universities to address that and say, well,the data shows you are doing outstanding work in this field, but you haven't communicated that to the rest of the global community, or particular regions, or particular parts of the world don't know that. How about you look at a communication strategy? How about you look at a collaboration strategy to make sure people do understand the strengths that you do have? It very much varies by institution and their own particular missions and breaking down the scores. But there's a couple of areas where you can make some significant improvements. International collaboration, international talent, and communication and dissemination, for me is crucial. Absolutely, thank you so much Phil, and I fully agree with you. I think in our experience we are seeing similar trends working with academic publishers, scholarly societies, research institutes. I think there is an increasing need to repurpose content, the traditional content, just easy to understand formats, make it more discoverable. Like you rightly mentioned Phil, I think there's just so much good stuff that is available out there. But yeah, it's not really going to work out if it's going to be not discoverable. And research institutes, researchers themselves and the other stakeholders, if they don't talk about it, if they don't promote it, a lot of the benefits could really not be highlighted. Just staying with that topic, I know that THE offers consultancy services for universities, and because of such in-depth understanding of how they work just numerous data points, understanding what the gaps are, what kind of strategies does THE develop for universities? Is the whole research, promotion, reputation building, also part of some of the offerings? Yeah, absolutely. As a traditional media outlet, we do a lot of work with universities to help them build reputation and brands. A lot of universities come to us just with very traditional advertising. We have this huge platform, as I mentioned, 50 million people on our platforms across Times Higher Education and Inside Higher Ed. So, universities just wanting to communicate in the most simple way to the global academic community will come and run advertising with us or build content on our THE connect pages and use the platform to push out their reputation. As well as giving them some of the solutions around reputation through the traditional media platforms, we also are able to do a deep dive into their reputation themselves. We have, of course, the hundreds of thousands of data points from our reputation survey. What do people think of you? How are you perceived? Which areas are you known for in terms of your research excellence? By field, which areas are you better recognized in terms of geography around the world? We can do a lot of diagnosis around reputation, strengths and weaknesses. We also have a student panel, 150,000 international students we can use as a research panel to ask some questions about reputation and how students perceive you. We can do a lot of reputation work. Actually just recently we acquired a business called‘The knowledge Partnership’ which runs something really exciting for me called the ‘World 100 Reputation Network.’ The World 100 bit is slightly misleading because it's a network of university senior communication managers who literally join a membership community where they share good practice, they share insights and intelligence about what the best mechanisms for communication are in terms of dissemination of research and in terms of building your brand and reputation

with different stakeholders:

governments and students and the wider community. That's a really exciting peer-to-peer group of people, who just come together and share good practice. One of our offers is around reputation management, reputation building, and then some solutions. We also do straightforward performance analysis, what are your strengths and weaknesses against all your competitors or your chosen benchmark institutions. We do social impact work, so how do we make sure that you are maximizing your impact on society using the SDG database. We have a very, very holistic range of solutions. We increasingly support universities on their international strategies and are looking at their international collaboration networks, looking at any gaps or weaknesses in the geography or subject areas in terms of their collaborations. Because I think we have such a huge amount of data and also a great, talented, international consultancy team, we can increasingly take a very, very holistic approach to what is the university problem that wants solving, and we can build a bespoke solution. But I would say, it really frames around performance analysis, strategic support around networks, collaborations, international strategy, strategic reputation and brand building, social impact. And more and more actually we are doing work with governments around the size and shape of national education systems and national evaluation systems, national rationalization programs, mergers, etcetera, etcetera. It’s almost anything you want, to be honest. We got this wonderful sort of single source of intelligence and information which is proving very, very popular.