This is the response by the Media, Communication and Cultural Studies Association (MeCCSA) to the Independent Review of the Teaching Excellence and Student Outcomes Framework (TEF) launched on 18 January 2019 (Review of TEF: terms of reference). The response was prepared by the MeCCSA ADSC sub-committee and agreed by the executive, and should be read in the context of the consultation briefing document.
5 Are you responding on behalf of an organisation (eg. higher education provider, student union or representative group)?
Yes
a. If yes, what is the name of your organisation?:
Media, Communications and Cultural Studies Association
6 Have you been involved preparing for or writing a TEF or subject TEF submission?
No
7 Have you been involved as a TEF assessor or panel member (for provider TEF or in the subject pilots)?
No
8 Would you like us to keep your responses confidential?
No
If yes, what is the reason for confidentiality?:
9 Please indicate which UK country/other country you are responding from.
Please indicate which UK country/other country you are responding for. :
UK-wide
If you are responding from a country outside of the UK, please write this in below.:
Why have TEF?
10 Do you support the aim of assessing the quality of teaching excellence and student outcomes across providers of higher education?
Yes
Please explain why:
We are certain that providing prospective and existing students with valid information is essential, and that this information should be available to a wider public, including the academic community and prospective employers. For these reasons it is vital that the information is comprehensive and reliable.
Why have TEF?
11 These purposes fall into two main areas: providing information, and enhancing the provision of higher education.
Both are equally important
b. Please outline below the reasons for your answers :
Both are necessary. We believe it is essential that both aims are met by any assessment of teaching as to prioritise either would weaken both.
12 Should there be any other purposes for TEF?
Should there be any other purposes for TEF? :
We do not believe TEF as currently constituted meets its current stated purposes therefore could not suggest further purposes.
What is TEF?
How does TEF work?
13 Are the criteria used in TEF (see Figure 1 for a list of the criteria) appropriate?
No
If not, what criteria would be more appropriate? :
We do not think judging the quality of teaching provision by the median incomes of graduating students is appropriate. This seems to be a category mistake. The test of a good education is surely what students know and understand, and this does not translate into income necessarily. In general the TEF process, while properly referring to the value of intellectual rigour and committed teaching, gives undue attention to employment rates and earnings as relevant outcomes of high quality teaching. We are also very unconvinced about ‘rigour and stretch’. Is this one thing anyway and what on earth is ‘stretch’?
14 There is no direct measurement of teaching quality currently available. As a result, the TEF uses existing data as indirect measures of teaching quality. These measures are known as “proxies”.
No
b. If you answered no, what metrics would be more suitable proxies? :
Our main objection, whether here or elsewhere, is to the use of ‘proxies’. The use of the NSS is especially flawed. NSS scores are not designed to provide, and methodologically are incapable of allowing comparison across institutions. There is no way of knowing how students use criteria concerning level of assessment and feedback in the NSS. In principle we see no proper proxy that gives a true measure of teaching quality.
15 The TEF metrics are benchmarked to account for factors such as the subject of study, prior attainment, ethnicity and educational disadvantage of the provider’s student intake (see that ‘What is TEF?’ section for detail).
No
b. Does TEF benchmark for the right factors?:
On the question of benchmarking, there would seem on the surface to be a measure of proper consideration of differing student intakes and demographics, but the exercise could unfairly and too readily become something of a catch-weight contest which could be used to patronisingly create a sort of league table.
16 The TEF process uses both quantitative evidence (for example, the core metrics) and qualitative evidence (for example, the written submission).
a. What are your views about the balance of quantitative and qualitative evidence considered in arriving at ratings?:
We see no balance that could give a valid and credible measure of teaching quality using this broad approach.
b. Are there any other aspects of the process that you wish to comment on?:
We are especially concerned that students are able to understand the range of provision available across the sector in our field, from courses mainly based in humanities to those more rooted in the social sciences, and from those with a strong vocational or ‘practice’ orientation to those with little or none. Programme labels are often of limited help in a diverse field such as ours, and it is unlikely that TEF helps with this important problem.
Are the ratings right?
17 Are the purpose(s) of TEF met by:
No
Please explain your answer:
Too simplistic and crude, and vulnerable to both misinterpretation and misuse. Not Answered
Please explain your answer :
No
Please explain your answer :
We find the rating system and especially its ‘quasi-Olympic’ labelling objectionable and crude. It both encourages unhelpful and damaging comparisons and competition, and misinforms.
18 If you answered no, what alternatives you would suggest.
a. For provider-level TEF?:
b. For subject-level TEF?:
Please see our comments earlier on the important need for students to be aware of the range and variation in programme content across a diverse field such as ours. At subject level students already make use of DLHE and Unistats data.
c. If your previous response(s) reflected on the impact of the TEF on the international reputation of institutions and/or the UK as a whole, we would welcome any evidence or information you can provide that might support your view or help inform the independent review.:
Has TEF changed anything?
19 Has the introduction of TEF positively changed the educational experience of students (e.g. teaching and learning)?
Not Answered
If yes, how?:
20 Has the introduction of TEF negatively changed the educational experience of students (e.g. teaching and learning)?
Not Answered
If yes, how?:
21 Has the introduction of TEF impacted positively on research and/or knowledge transfer?
Not Answered
If yes, how?:
22 Has the introduction of TEF impacted negatively on research and/or knowledge transfer?
Yes
If yes, how? :
Our general impression from members is that the administration of TEF related work is onerous and time-consuming, and detracts from the time needed both for research and for directly teaching related tasks.
Is TEF worth it?
23 Does TEF help you as a student/provider/employer?
No
Please explain the reasons for your answer.:
We find little evidence that TEF is either designed for use by academics, nor that its introduction has improved the standing or quality of teaching. This is despite the fact that communications and media studies, in its various formulations, is popular with students and has an excellent record for the employability of its graduates.
24 Explaining your reasoning, what are the most significant costs of:
a. Provider-level TEF?:
b. Subject-level TEF?:
25 Explaining your reasoning, what are the most significant benefits of: a. Provider-level TEF?:
b. Subject-level TEF?:
Is TEF fair?
26 Are there particular types of students, provision or providers that are disadvantaged by the current design of TEF, in a disproportionate way?
No
If so, what changes could be made to address this?:
There continues to be both a worry and perception that TEF is designed , deliberately or inadvertently, to privilege STEM subjects and larger, wealthier providers, while offering little directly to those (academic staff) who deliver teaching.
27 Are there particular types of students, provision or providers that are advantaged by the current design of TEF, in a disproportionate way?
Not Answered
If so, what changes could be made to address this?: