So let's get started with our first plenary session.
Okay. It's an honor for me to introduce
Agnieszka Szarkowska, Did I pronounce it well?
Right, yes. Okay.
She's a university professor in
the Institute of Applied Linguistics at the University of Warsaw,
head of the research group Audio Visual Translation Lab (AVT Lab)
and honorary Research Associate at University College London.
I said I don't like reading, but bios I always read them.
Agnieszka is a researcher,
academic teacher, ex translator and translator trainer.
Her research projects include eye-tracking studies and subtitling,
audio description, multilingualism
in subtitling for the deaf and hard of hearing,
and respeaking and drawing on her passion for teaching,
she has co-founded AVT master class, an online platform
for professional Audiovisual Translation Education.
Agnieszka is a member of the European Association
for Studies in Screen Translation (ESIST)
and European Society for Translation Studies (EST)
and an honorary member of the Polish Audiovisual Translators Association.
And she is one of the big names
of media accessibility in Europe and abroad.
And for me, I met her... I don't know if I met her in a conference before,
but I really remember meeting her for Clipfair
in in our trip to Galway in Ireland.
And since then, I have followed her both, professionally
she's a great scholar and I can't wait to listen to what
she has to say about experimental research in Didactic AVT today.
So thank you really, for everything and for being here, of course.
Thank you Noa. I just put my time around, and I just want to say I'm very honored
to be here and I'm very humbled to actually be among the first class
of specialists in Didactic Audiovisual Translation,
considering the fact that I haven't worked in the field for a decade
since Clipfair, I feel like an imposter here.
Okay, so sorry if I say something that's not exactly right.
So, I'm coming here from a little bit of an external perspective,
but I just want to say I've got this great appreciation
for all the work that Noa, Jennifer and all the colleagues -can't enumerate
so many of you here in this room-.
You've created a field that is growing and this conference
I'm sure, is the first of many to come.
And the idea of Didactic Audiovisual Translation I think is just so clever,
we need to share it more with the world so that people realize
that it's great and that's what I'm trying to do here.
Okay, let's see.
So, experimental research in Didactic Audiovisual Translation.
I haven't worked in this field for a decade, so perhaps I'll ask
chatGPT to help me, you know.
What is Didactic Audiovisual Translation?
And, what scholars are its most prominent representatives?
And, what are the main findings of the field?
And have a look, why is this not working?
Well, it's a video. Well, it's not.
Again.
Yeah, okay. Well, it was just me. Thanks.
Okay, so let's have a look.
What chatGPT thinks Didactic AVT is.
I think it got it wrong in the sense that it's not really..
It's about learning languages, right?
It's not about learning other things.
I mean, not only about this. Most prominent scholars,
we've got Jorge here and Frederic.
What does it say here?
I think what it got right
is the use of technology, right?
And the use of all the different modes of Audiovisual Translatio.
So, only partly right.
So, I thought perhaps this better people, this definition there...
So, what are we talking about here?
So Didactic AVT, these are all those tasks that are,
that have the learner in the center, right?
They're interactive and they can be carried out, both
individually and in groups, either independently,
are with the teacher and promote all sorts of learning activities. Right?
Or, even a simpler definition by Jennifer and Noa,
it's what? We're talking about actively engaging
learners through Audiovisual Translation.
And thanks to that they're supposed to learn foreign languages.
And as I said, I think it's a brilliant idea
and I hope it will be promoted and recognized.
And they said, okay, there has been some projects
and I tried to put them in my slides, but they're, I mean,
I started looking in that and there were so many that didn't fit the slides.
Yes, so I just I just worked in Clipflair
and there's been so much going on,
so congratulations on this.
You've been very successful
in getting funding as well, which is a great skill here.
So, well done in those times, as well.
And, what are the benefits of Didactic AVT, right?
Plenty, yeah. So we've seen, thanks to the research
that many of you sitting in this room have carried out and that
we can observe language gains.
So people, indeed, are learning languages through Didactic AVT.
Various skills are being learned, right?
Not only one skill, but all sorts of skills,
and integrated skills, as well.
And it's fun, it's engaging learners.
And I've already mentioned different types of Audiovisual Translation
so captioning, revoicing, but also harnessing
media accessibility for didactic and AVT
and fostering all sorts of learning skills.
And what I really love about the field
is that it has those practical implications, right?
And that you are talking to teachers,
we are teachers as well, right?
You've created so many resources
for teachers and not only do the didactic resources, like lesson plans
and assessment rubrics, etc., but also AVT tools.
Starting with with Clipfair and now with the with the new project.
To support teachers in their efforts,
so really impressed with all this.
I remember when I was studying and I attended
my didactic lessons at the university, translation was banned, right?
Because it was seen right, this coming from the grammar translation method
back in the old days.
So we were all trying to use
the communicative approach and translation
was this pariah, right?
And now where are we?
We've got mediation, we've got the European Union supporting this,
so we've come a long way.
And that is, I think, thanks to the work of the people in this room.
And I just want to stress,
it's not that we are working in our own bubble,
we are like this field that doesn't connect with any other fields,
I think it was already mentioned here in the opening panel
that there are other fields, neighboring fields that should be mentioned,
and I think we should perhaps make connections to them.
I'm listing just a few of them this morning.
So on the one hand, some foreign language learning,
teaching or education, whatever you want to call it, Second Language
Acquisition and Audiovisual translation, psycholinguistics,
translation process research, multimedia learning and more.
But where are we here?
What's the position of didactic AVT here?
Are we this Cinderella? Right?
We discussed with Alejandro are you centering
compared to other fields?
What's our position?
Are we strong or not as strong as we should be?
So I thought perhaps what can be done is that we can use experimental
research to help elevate the the position of the of the discipline.
And experimental research, I think, can be used
and it has lots of advantages.
So first of all, we can actually learn about learning,
learn about how people learn languages through Didactic AVT.
And thanks to that, we can get some empirical evidence
that we can use later to support the creation
of language curricula and then advanced discipline.
So far, I think in this field we have focused
a lot on measuring the product of learning, right?
So we've measured, we have some assessments done
post hoc, like post task, and perhaps it's a good time now
to start thinking about the process,
measuring the process, what is happening actually,
and when people are doing the tasks, how are they learning, actually?
So online, meaning, during the process here.
So the message that was frequently
used before, and they were still be used, they're good methods, right;
don't get me wrong.
These are surveys or we did some pre and post
post-test and then we compared how much people learned. Right.
What did they know before? What they know after that.
And there were also some interviews, right.
We were asking people what they thought and their responses
from what they've learned. They were overwhelmingly positive, actually.
and what I like to throw at you here, that perhaps we can
add some more things to the equation here.
So, a couple of online methods, perhaps
that could be also used, think-aloud protocols.
So, essentially they're asking people to verbalize what they're doing
while they're doing it. It doesn't come without problems as well.
So you're basically tracing how people are using the keyboard and the mouse
and what exactly they are doing while they're carrying out the tasks.
My favorite, of course, eye tracking.
I can see where people are looking and what the implications of this are.
Okay.
And that that comes, most often, with the screen recording.
So if you haven't got an eye tracker screen recording is quite nice, as well.
There are other tools like EEG or EDA,
which is electro dermal activity, something like trying to measure how
stressed people are like myself here and the heart rate and stuff.
I'm not sure to be honest. Point number five,
I'm not sure if that's very useful for didactic AVT, let's see.
So let's have a look at all of them, one by one now.
So, as I said, think aloud protocols, right?
So you're asking people to verbalize
what they're doing during the task, while they're doing the task.
And there are two types. Concurrent, which is the one
that is actually happening while they're doing this.
But when you think about that, how can you do a task while talking about the task?
So, it's sort of impossible. So some other people said, okay,
perhaps we should do retrospective. So, after the task.
And so this is a whole discussion of the usefulness of TAPs there.
We can combine TAPs with keylogging
and keylogging is particularly interesting,
I think, to those of you who are interested
in the process of writing, in learning how
people write, and you can trace different,
you know, actions that they conduct like pauses,
rewriting all the deletions and insertions, etc..
And to those of you who are interested,
there's been special software developed
to use keylogging, and it's out there and available.
I have an example here
of how keylogging could be combined with TAPs.
So in this table, on the right hand side, you've got
the output from a keylogger.
So, it's a task somebody was translating
from Dutch, I think, it is right.
And they were translated from English into Dutch
and they were translating and they stopped for 10 seconds
and we don't know why they stopped.
Okay, so the keylogger doesn't tell us.
But then when you look at on the left in the column,
they're actually verbalizing what they're doing.
So they're trying to say the translation that they're creating.
And we can see that
they have a problem with the verb, which they want to you to use
to consume, to use alcoholic beverages, okay?
So when you combine that, you are able to see what was going on.
Now, I don't know what kind of typist you are,
but essentially if you're interested in the
process of writing and the two types of people
and I want to go to the doctor, they always do this.
So that's called a hunt-and-peck typing
and there is also touch typing, right?
And we sort of put learners in one basket,
but people are different, right?
Different people create, start texting in different ways.
And so research has shown that actually
different types of typists work differently.
And again,
we have some evidence from combining keylogger and eye tracking.
So this picture here shows a person who's this hunt-and-peck typist
and they're typing 'Twinkle, twinkle, little star'
and you can see the little boxes, black boxes there.
So that's the moment when they hit the keyboard
and the diamonds there
it's that moment when they're looking at the screen.
So you can see the person is really typing, typing;
and the moment they made a typo, they only then look at the screen
and they continue typing without looking.
Whereas a person who's a touch typist or close to that,
you can see they're typing
and they also at the same time, looking at at the screen
And only when they hear 'Mary had a little lamb', I think
they only look back again when they made a typo.
So like different people that create texts in a different way
and I think combining eye tracking and keylogging could be a great way
of finding out how reading and writing
can come together, because here we are producing text;
but you're also reading at the same time.
So you can learn a lot about this.
Right, eye tracking.
So I think so far eye-tracking has been used mostly
to study the reception of audiovisual translation. Right?
Only a tiny fraction of research has focused
on the production translation process research.
And didactic AVT here, I think combines reception
and production in a very interesting way.
I have a fragment here of what we did.
This is a study combining keylogging and eye tracking, so you can see
a subtitler who is creating subtitles,
she's timing the text that she has and the blue
lines and dots are her gaze, so where exactly she's looking.
So you can she's looking at the text, but she spends a lot of time on the audio wave
She's looking to be precise with the shot changes
with how to time the subtitles.
So, the same way you can combine, eye tracking with keylogging
here and see how people are creating
the text that you've asked them to create
and where the issues are, what are the problems there, during the task.
And the eye tracking research has brought us lots of information.
We know that viewers read subtitles automatically,
it doesn't matter what language they're reading and what type
of subtitles we're dealing with, whether the audio is present or not.
And then different types of viewers also read subtitles differently.
And and I've got a slide, in a moment, on that.
And not all the words are read in the same way. Okay?
So if you're studying vocabulary,
that might be particularly interesting.
Let's have a look at the examples.
So, I have two pictures here, on the left hand side
you can see a scanpath, a visualization
of how a deaf person is reading subtitles.
So they start in the middle,
they continue reading the subtitle,
and then they finish and then go back to the center of the screen.
On the right hand side, we have a hard of hearing participant
who's comparing what they can hear with what they can see.
So, this scanpath is much different.
So, I wonder how many different types of learners and
you know, do that.
Here what you can see, this animal is called the dugong
and it lives in the sea and the visualization is essentially
showing that different types of words read in a different way.
So if you have a word like a dugong or herbivore, perhaps a word
that is a low frequency where it's not very common
so people would spend more time looking at this at this word
compared to easier words such as those like sea, etc..
And we've done a lot of research on the processing of different types
of subtitles in AVT, you can look at those titles
processing, reading, visual attention, etc.
But in AVT there hasn't been much work, actually,
on how that relates to learning.
So, we had this study, for instance, on different types of subtitles
interlingual English to Chinese, right?
And comparing English to English, bilingual subtitles,
Chinese and English and also the non subtitling condition.
And we just, in a naivety, we just focused on
people spend more time on this condition versus the other.
But how does that relate to learning?
And I found the answer actually in some SLA studies
that I'm listing here, some of them.
So they did a very similar study, comparing four conditions
and again, interlingual subtitles, English to Chinese;
Chinese to Chinese, and then bilingual subtitles, etc.
And they found that when people spend more time
on certain words, that means learning gains. Okay?
So they learn these words
better and they found learning in all conditions.
So in all types of the subtitles actually.
I think a good idea is to combine offline and online measures.
Right? So you can combine eye tracking,
Eye tracking by itself won't tell you much, okay?
So it's good to combine it with, I don't know, with an interview
or do a focus group.
So it's good to talk to people and let me look at the equipment.
So I've done all the reading
what's been going on in Didactic AVT?
So I had a look at Jennifer's book, thank you for this.
So she listed conveniently all the experimental studies
that were there with standard interlingual subtitles
reverse subtitles, intralingual subtitles, all sorts of things.
And there's more to it, right?
I also had to look at some SLA studies.
They were interested in different types of
language combinations, interlingual subtitles, keyword captions.
Right? Some bilingual subtitles as well.
And they haven't found any detrimental effect of captions
and only facilitating effects.
And I said, okay, maybe I missed something. So...
I went to ChatGPT again and I asked, perhaps could you please list
the experimental studies in Didactic Audiovisual Translation?
And it's thinking, usually it just gives you answers already.
And then here's the list with it.
And I was like, Oh my God, I don't know this.
I don't know these scholars.
Anybody knows them?
Do they look Iranian to me?
Huh? These guys look Spanish, judging by the names.
Are you familiar with this work?
I was like, Oh my God, I have to read so much.
There's been so much going on, right?
This one Chinese.
Well, and they all look
very authentic, right?
Yeah, and, you know? Focusing on the impact of subtitling on vocabulary...
So this is key for us.
This is what we want to know.
And I was like, okay, has any of these studies
used eye tracking or keylogging perhaps?
What experimental methods were used in these studies?
I was interested to see.
Again, thinking.
Suspense.
Oh, they vary. Yes.
The impact of different translation strategies on language learning.
I don't think that's what we've been doing. Pre and post tests.
Yes, that's true. We've measured that
to measure language acquisition. Questionnaires.
Very good. Quantitative analysis. Yes.
And then it goes on and tells me that the first study that it listed
used, let's see...
pre and post test etc. Right.
So, perhaps I'll stop here and move on.
So I went and look at these studies.
And well, I didn't find them, actually.
So I wrote to ChatGPT 'So, I looked them up and they don't seem to exist.
So did you invent it? If not, please, provide a source.'
And it goes 'Oops, apologize for the error.
It appears it really doesn't exist.'
You know, but if you're interested in research on incidental
vocabulary, learning and AVT, I can recommend some other studies.
Jorge, your study 'Subtitling and dual coding theory'
Ah, you know. Okay.
So, I thought that's quite funny and perhaps I shouldn't
rely too much on this for the moment.
Yes, so, moving on.
From what I know,
I think what's been happening
is like, a lot of SLA research, right?
And FLE focus on reception
and didactic AVT is doing both.
And the result that we got
in didactic AVT research, very positive, very very good.
We've got higher scores, users love it.
All the skills are enhanced.
So we get always the the growing, the learning gain curve.
And I was like, that's amazing.
Is this always true? Is it always working?
I thought maybe there are some things we can do better.
So I had a quick look, and here's my list
and we can perhaps continue the discussion.
I think perhaps what we are doing is we are citing our own bubble.
I mean, look at my papers. It was Jorge,
Jorge, Jorge. Right? What we can improve, as well
I think is design and data analysis, right?
Having larger samples, not only convenient samples,
and we sort of tend to focus on the micro-level
and maybe it's fun to look into the lower level details.
So what can we do? Perhaps venture out, talk to people from other disciplines?
We can also try to develop or, you know, look at the models
that are out there, some theories and perhaps criticize them.
Are they right or not? Conduct some experiments.
Of course, with online processing.
And one last thing I want to say is share the data.
Okay? So that we have this transparency
and moving beyond our bubble.
So I had a look at some of the presentations
and it was like Noa, Noa, okay...
Frede, yes.
Jorge, of course. We keep citing the same thing.
I'm guilty of this as well, right?
It's like, I'm there as well.
And I was thinking, are we publishing in the best places?
Perhaps we're just not talking to the right people from other disciplines.
They don't know about us. Okay, so what are our topic journals?
You don't have to answer now, but think about it.
Which journals do you follow?
And what are the best journals for us?
And I had a look at the Scopus
the Scopus journal ranking, the first twenty
or something, and I marked in the blue box
some of the journals that I think we could publish in
so Modern Language Journal, Language Learning, right?
The first two top journals, they seem very relevant
studies in Second Language Acquisition, Language Teaching,
classroom discourse, right? Journal of Second Language Writing.
These are all officially good journals, right?
And I had a look, did they have any research on subtitling?
And so... Language Learning, I put subtitling as a keyword.
There was a study by Martin Denham from 1992 and one more from last year.
Right? So not much. Captions, the same.
Single papers. Okay?
Why are we not talking to these people?
So I was like super excited when I was reading their paper
and from last year, about one third of press.
And I saw...
And I saw Rula, Patrick, Noa... some people there.
Yeah, they know about us, right?
They read us and then, I read on
they are saying it's like a study on other studies
and they're saying we're not going to discuss other uses
at a context in which L2 learners create videos.
They won't be included.
So that was the only, they cited this
to say we're not going to cite it, because why not?
Right. So that's a bit tricky. Yes.
And one thing I mentioned, perhaps we can focus a little bit
on the micro-level
and trying to study one thing at a time, like writing or speaking
or some aspect of a language
cognates, perhaps how people learn cognates versus non cognates.
Right.
And trying to control the variables that we've been using.
So instead of saying intralingual subtitles can improve
skills in online settings, which is perhaps true, but very general,
we could say interlingual subtitling helps intermediate learners
decode and segment speech, recognize word boundaries and map sounds onto words.
Speak more specific, okay?
That's what I mean by micro and macro level here.
Okay. Now more powerful stats.
I looked at the analysis.
Just look at my old papers as well.
I got the same stuff.
T-tests, ANOVA, correlation.
Perhaps there's been an, I think, excessive focus on descriptive statistics only.
And some people say, okay, these figures are just illustrative, right?
You know, we're not really doing any, any serious stats.
I think we should.
So, typically what happens like we have, as I said, pre-test
and do we measure the results with a post-test?
So it's so simple t-test there and then we get the results
and it's typically higher in the experimental group.
But, oops, sorry.
When you think about that language gains, the way people learn
are influenced by so many different things.
People are different, languages are different,
the levels are different.
So we've got those work related factors
and learner related factors, right?
Just to mention just a few in terms of words
that frequency of a word, how long it is,
the type of the word it, it's a noun or it's an adverb, etc.
How frequently does it occur in the material that you're testing?
Does it occur once or does it occur ten times? Does it matter?
Of course, if it's going to occur ten times, it's more likely
to be learned, right?
It's audiovisual translation, does this word have a correspondence
in the image? You've seen my dugong example.
Okay, if you just read 'Dugong' maybe it's not clear enough
you should see the dugong.
It's just so much better to learn, right?
So that's important. Cognates again, right?
Is this a word? like Google have a lot of
English and Spanish, English and whatever, right?
Is this word a cognate or not?
And isn't this important? Yes, it is. Right.
Lots of learning related factors, as well.
So how proficient the learners are and it's
as I've learned in very bad reviews that I got when I used self-reports
asking people what English level they had.
So we should use some validated tests for this, right?
So like, I don't know, like LexTALE,
Cambridge or Vocabulary Size Test, etc.
And apparently does this massive, massive effect
that learners with larger vocabulary sizes have higher vocabulary
gains rates on everybody's
working memory capacity, the aptitude that people have, right?
Learners differ in their aptitudes, in obviously, in their education and and age.
I'm not going to learn another language, my age now.
So obviously, you know, things are different.
And again, AVT and I think this is what we got
quite correctly, right? With lots of different language
combinations and different types of of AVT
are some types a bit better than others in learning some specific skills?
Probably, yes. And I wanted to advertise something.
So if you want to combine all this,
all the word related factors, learner related factors,
all sorts of factors, we need to move away,
I think, from t-tests and ANOVAS and use mixed models.
And, I don't know much about them, to be honest.
Right. I've got Valentina sitting here
She knows much, but she knows a lot more than I do.
So it's a learning curve for me.
What I know, I should be using them and I'm struggling to learn them.
And I know we can incorporate all sorts of things into the model.
So rather than do pre-test post-test, you can put everything there
and see what significant and what's important.
So, rather than having a simple data set,
here, I'm showing an example from my previous study, right?
Compared to speed, slow and fast. Right.
And that means fixation, duration.
So you have data from eye tracking data
from participants who watched lots of different subtitles.
And I just have one number for all the subtitles, all the words
but one participant.
Okay? One for the slow, one for the high speed.
And that's actually throwing away loads of data that can be useful.
So instead, I can have a larger and data set that
I can include every single subtitle as a data point.
Data point meaning one row here or I can actually go
even deeper and include one word as data point. Okay.
And it grows and grows and stuff then becomes more powerful.
Exciting.
Yes. For this, I think we need to have different members of the team. Right?
The mistake I made with my recent grant, I didn't get a statistician.
So we have to struggle ourselves.
So the next time you're applying for a grant, get a statistician. Okay?
So instead of having those sort of results in our papers, we should have
something that looks close to this.
I mean, to understand what that means, okay.
To be able to judge.
And so all of serious and very professional and stuff.
But I'll show you something in a minute.
And yes, I also mentioned some theoretical
models or some hypotheses or
that we can perhaps test. I wanted to mention three, briefly.
A cognitive load theory and
Redundancy Effect and Involvement Load Hypothesis. Okay?
I just found out about the last one quite recently myself.
So cognitive load, those of you who are interested in cognitive load,
there's this theory that says there's different types of load
and there's this load that's called intrinsic.
That's essentially about the difficulty, how difficult
the material that we are learning is.
Okay, so if it's rocket science or present simple right?
And there's extraneous cognitive load, which basically is
about the way we present a task; about the instructions,
are we making it easy for the learner to do the task or do we overcomplicate things?
And germane is just the remaining cognitive resources.
In the context of AVT
and some people said that captions
create extraneous load because viewers have to divide
their attention between the video and the captions.
Okay. And the same with the redundancy effect.
So there's been literature,
people are saying captions are difficult
because you're getting as if you have the same information
in the audio and in the captions.
And it's just, it creates cognitive, you know, extraneous cognitive load.
And it's just viewers are not coping.
That's like, seriously?
But still you've got this guru, Richard Mayer
from multimedia learning.
He did the study comparing
how people learn with subtitles, without the subtitles
and his findings are that adding captions
and created demands for extraneous processing,
and that led to lower scores.
That's something else that we can get,
because we always get higher scores in great activity.
And he says this is a serious challenge. Right?
And that may impact learners, cognitive capacity.
He didn't find any differences
actually, in the subtitles and non subtitles condition.
And he says, I think subtitles were
not an advantage, not a disadvantage
but we don't have a support
for the proposal that we can use subtitles here.
And he says, maybe we should use eye tracking.
Maybe people were not looking at the subtitles as like,
Hey, done in studies previously, right?
Look at professor d'Ydewalle, for instance,
who did a lot of eye tracking studies and showed
that people are actually looking at the subtitles.
Most of the time, regardless of what the type of subtitles are.
There was other study by our colleague Jan-Louis Krueger, right.
Who also tested how learners
watch videos with English subtitles
and with, it's a sort of South African language.
And again, he found that the split attention effect was not there.
People could cope.
But still, in the multimedia learning bubble, people published this.
This is from 2021.
Right. Are captions in video tutorials a bad idea? And they said yes.
Redundancy principle tells us it's a bad idea.
Yes, people can't cope.
They have lower scores like how come? Yeah.
And there are lots of studies like this. Right?
This one from last year, I think. Disadvantage of subtitled videos.
It's like how come?
And I think it comes from the biases that we have.
Right? When you look at Mayer and he says
the results of his experiment don't provide support.
Right.
So he's entrenched in his idea of redundancy and cognitive load.
And then Krueger coming from AVT says, well, there's no
evidence for the split effect. Okay.
They got the same results
that none of them actually got any significant insights.
And they got, they arrived at a very different results.
And the final thing, the involvement load, I saw this,
it's quite exciting and quite relevant.
I didn't know about it much until recently.
So essentially is this concept that
there's this task induced involvement
and it consists of three parts.
One relates to motivation.
So if you're a learner and
a teacher tells you to do something, your motivation is lower.
But if you yourself want to learn something, your motivation is much
higher, you're more strongly motivated.
And then a search.
So if you try to find the meaning of the word yourself
rather than be given the word, it's a much more likely
that you will learn the word and remember the words better.
And then if you're comparing
this word with some other words, for instance, for translation,
it's even better because the motivations are the evaluation,
your cognitive capacities are
working there and a lot.
Essentially the involvement load hypothesis tells us that the retention of unfamiliar
words depends on how involved the learners were in the process.
So, who has said the task, the teacher or yourself?
Have you searched the word?
Have you compared the word with others? Right.
So if so, the greater the involvement, the better
the retention, the better the people remember this.
And yes, and they had come up with this idea of involvement load index.
So whenever you're designing experiments,
you can have one task that is less involving
and another task that is more involving.
For instance, reading comprehension would be much less involving compared
to, I think and I didactic AVT task, which is much more involving.
And here. Yes.
So when you think about what we know about
vocabulary learning and the and the retention
and is that the words that are looked up in a dictionary,
you remember them better that you negotiate
in some communicative situation as you again remember them better.
And if you have productive tasks rather than receptive task,
you can remember these words better.
So that just tells us what kind of tasks we should
perhaps be creating.
And I was wondering if
anybody actually used the involvement load hypothesis in didactic AVT
and which tasks are more involving, you know, how can we make
these tasks more involving?
And I want to say a few words about sharing research data, quickly.
So whenever we do research it's, I think, good practice to actually
share it with the world for transparency and for scrutiny for other research.
That's what we've been doing recently.
So we share all sorts of things, right?
The Excel files, the subtitles, whatever you can, right?
Not everything you can due to copyrights, but what you can.
So I was very happy to see that this study that I mentioned before,
using the four conditions
with Chinese and bilingual subtitles, they shared the data.
So what I did, I looked at their subtitles.
Obviously. That's the study, right, that they showed before.
So I import that to excel and calculated
very quickly the reading speed of the subtitles, the use.
And as you can see in the right hand column, the subtitles,
these are bilingual subtitles, here,
they're all in red, meaning above 20 characters per second.
The second subtitle 50 characters per second. 39.
Right.
So, I mean, nobody can read this, right?
Yeah.
And all conditions, you know, I looked at all the conditions
and they were all really bad in terms of speed.
And then they tell you in the results
when people were reading bilingual subtitles,
the ones that you can see
in the picture here, they spend less time in the second line.
I wonder why. Right.
So, you know, and this was published in an excellent journal. Yeah?
So I can sort of interpret these results my own way
now, once I had a look at the and subtitles.
I just want to say so replication is really necessary.
We need to do more replication and you know, to be able to
be sure that we are getting the, the certain effects that are there.
And we're facing this publication crisis.
Some people are saying most research findings are false
now because the research that was done before was maybe
not the best than when they tried to replicate a hundred
studies in psychology
where 97% of them had significant results.
Only one third of these results were actually replicated.
So I wonder what's going to happen in AVT, what holds, what doesn't?
And you may be wondering.
Well, it's publication.
Sorry, is the replication sexy?
Can I publish this?
Val got a paper rejected when we applied for a conference
where a replication. Maybe weren't innovative enough.
I don't know. I'll tell you later when we get some.
We just submitted a paper.
We'll see if it's publishable.
So trying to finish off and give it time for questions.
I think experimental research can help us really answer
some of the research questions that we have, should have.
We still have lots of questions.
And hopefully elevate the status of this discipline.
And you can ask Val and myself and a lot of people involved, It's fun,
right? And just, yeah.
Whoops. Did you?
I did not.
Just still had one, disappeared.
My time is up obviously, but.
That's this.
One thing. Thank you.
You know.
The previous slide and I'm done.
Just wanted to finish off with ChatGPT.
So they're not. Oh, not doing anything.
It's always like that with AVT, right?
I mean, this is not a paper. It doesn't go wrong.
Okay. Yeah.
So anyway, so what I had in my slide, what I wanted
to finish off with is I could still use the slide.
Actually, I'm just trying to
do the time until you actually I didn't have 700 slides here.
Well, I had too many, but not 700.
Yes. So I think
we need to talk to people
from other disciplines and we need to cooperate with them
and also trying to embrace the new technologies in didactic AVT.
How are we going to, what are we going to do with ChatGPT
with automation, with automatic speech recognition,
machine translation, with all the CAT tools.
How are we going to use them?
And I wanted to finish off. Yes, with the...
Well, anyway, perhaps I can share it later, somehow. Right.
And well, this was actually the best of ChatGPT
and I wanted to end up with maybe next time
instead of me you're going to have an AI
speaker here because it have some good recommendations.
I wanted to thank my team as well that I'm working with.
Can't see the pictures of them, but I'm learning a lot through them.
So I just want to say
working in a team is great and I've got a great team here.
Thank you.
Well, maybe it is next
and you can show that in a second.
Thank you very much.
I was thinking, well, I was listening to you
that I need to watch the video later. Yeah.
I need to, you know,
capture everything and revisit all the interesting things you've been saying.
So thank you very much for the wonderful presentation.
We don't have much time for questions,
but maybe a couple of them.
Either there is anybody here or online.
I think Anka was controlling the online questions.
For a couple of questions
or because we need to move to the parallel sessions.
I know, but
nobody we have a mic here.
I think he's trying to fix it.
Yeah, I think I just want to say that, you know,
the field is great, is young and it's developing.
It has so much potential that we can still explore.
So I think that's my get home message.
Thank you.
Alejandro, yeah. You want the mic?
Mic, yeah.
Thank you.
Is it working?
Thanks, Agnieszka, for such a fantastic presentation.
You raised an important topic, which is statistics, experimental studies.
Many of us might find it a bit overwhelming
when we're approaching this kind of methods,
not least because we're not that experienced
in that kind of approaches or software.
So wondering you as an expert in this field, eye tracking, etc.,
what kind of suggestions would you make to research groups
like ourselves, which, you know, we might want to embrace
these kind of technologies, software methods, etc.?
Well, I think it's essential
I mean, Alejandro shouldn't overestimate me.
I know very little about statistics.
Yes. Don't get me wrong.
I'm learning myself, right?
When I graduated, I,
you know, hadn't had a single class in research methods.
So I think it's imperative
that we introduce research methods starting, I think, at the M.A.
level to make it compulsory for the four PhD levels
so that we actually know what we're talking about.
We just need to learn those things
There are books we need to study, there are online courses as well, right?
So if you want to learn, are there free programs.
Valentina is a big fan of our,
if you ever like
programing, it's great software, you got lots of support and
and there are courses as well, even at UCLA right? I remember
I learned a lot when I'm at UCL so there's no excuse.
We just need to learn this stuff.
And having said that, there's so much to learn,
it's probably impossible to learn everything.
So get a team, get people who know more than you.
That's what we've been doing
o recently with the work with SLA colleagues, people
who know much more about statistics and we can find parallels.
So we know about AVT, they know about Language Learning,
and we are starting to work together on cognates, for instance.
So I feel, so yeah,
I'd say we just need to learn.
Thank you. One more question?
Maybe? Comment? because I'm trying.
I hope this works and you can do your last slides.
So maybe a question in the meantime.
Well, I have a comment.
Just, it's interesting.
Very interesting what the review of journals you gave and everything.
And it's more than a comment.
It's a personal experience.
And my personal experience
with this big journals, language learning is desk rejections
because we are the Cinderella somehow.
Yeah, we are not in language learning, we are not audiovisual
translation, we are not in translation.
Where are we? Well, we are rejected in many ways
because they don't really know where to put us.
And we've tried, but we don't get the peer review.
We don't get to the peer review and it's tiring, really.
Yeah. And I think you're absolutely right.
But I think we've been there with
translation studies like my department is called Applied linguistics.
We all do translation, but it's in the system.
It's, it's applied linguistics, right?
It was the same with AVT within translation studies, I think.
Right? Used to be a Cinderella.
Now it's not the Cinderella, right.
So I think that there is hope.
We should keep trying, although of course it is pretty depressing
to be getting rejections. I know, yeah.
So yeah, I'm with you here.
And I think, one final thing, perhaps.
I know that people were not so successful.
Some people was getting grants because the funding bodies,
the committees and the funding bodies
in the funding bodies there were like made up of linguists and SLA
people rather than AVT people.
So that's there's a systemic problem there.
I think that we need to address.
Okay. Thank you.
Somehow, I'm sorry, we've missed your two last slides.
And I didn't break this. I'm sorry.
But you know, these things with computers, sometimes with technology.
So we have our parallel sessions.
Some here, some in sala de Grados and the other two rooms upstairs.
So please enjoy. And then let's meet at coffee at 12:00.
Okay.