Data-driven grammar teaching and adolescent EFL learners in Greece

Nikoletta Rapti
University of Nottingham

Abstract

Data-driven learning (DDL) is a ground-breaking approach and is attracting increasing interest in language teaching and learning, but there is a need for more empirical research with younger, lower level students and a wider focus on aspects of language learning before concordances find their way into EFL classrooms. This paper reports on a corpus-based grammar study which was conducted in Greece with a group of adolescent learners and investigates the impact of DDL on motivation and the learning of grammar. To this end, concordance-based tasks were designed in printout form for use with the experimental group, whereas a conventional grammar textbook was used with the control group. The empirical evidence drawn from the qualitative data and test performances underlines the significance of teacher mediation and suggests applying DDL in the classroom as a complement to conventional approaches initially, until learners become more comfortable with corpus exploration and the concordance format.

1. Introduction

Recent decades have been an important period for the development of corpus linguistics in language learning and teaching. Corpus research has highlighted the potential of corpora and data-driven learning (Johns & King 1991: iii) for language pedagogy. Emphasis is shifted from deductive to inductive learning, and ‘noticing’ (Schmidt 1990) of corpus data is promoted in the form of concordance citations as language input and self-discovery of lexicogrammatical patterns (see O’Keeffe et al. 2007, Mukherjee 2006, Römer 2006, Gabrielatos 2005, Braun 2005, Bernardini 2004, Hunston 2002). More and more corpus-based empirical studies have been carried out in EFL contexts and yet data-driven learning (DDL) has not been integrated into mainstream teaching practices. The absence of corpora from the EFL environment is attributed to a number of reasons such as lack of teacher training and limited number of corpus-based classroom materials. Furthermore, most studies to date targeted students at tertiary education (see Boulton 2008, Vannestål & Lindquist 2007, Koosha & Jafarpour 2006, Cheng et al. 2003, Passapong 2002, Todd 2001, Cobb 1997), whereas the empirical research carried out at lower levels investigating the role of DDL on aspects of language learning is limited (see Braun 2007, Sun and Wang 2003).

The present study involved a group of adolescent learners in Greece in a corpus-based approach. The focus was on grammatical aspects, as it was observed that learners often lacked motivation to study grammar because of great attention to detail and test items. This paper attempts to investigate the degree to which this young age group is motivated to study grammar when involved in DDL and whether DDL has a greater impact on the learning of grammar, as opposed to the prominent deductive and mostly passive nature of the students’ previous learning experience. It was also an opportunity to observe the response of this age group to the nature of corpus input, examine preferences and difficulties when students were exploring the innovative data and suggest ways to improve the materials and method.

2. Background to the study

The process of language learning is mainly exam-centred in the EFL environment in Greece and it is commonly believed that English can be taught and learnt more effectively at supplementary evening courses offered by private schools, which ensure high success rates, whereas the state school does not require learners to ‘reach near-native speaker proficiency’ (Sifakis 2009: 233) or take any official exams. Sifakis (2009: 233) gives an account of the educational and EFL situation in Greece and points out that ‘having a C2-level proficiency certificate is considered as essential for future employment as having basic computer skills. These certificates have lifelong validity and are considered by many as having as much weight as a university degree’. Prodromou (2007: 26, 33) also offers an insight into the system of preparation for official exams which often takes the form of ‘covert testing’, that is ‘the unconscious adoption of “testing” procedures in activities which would normally be classified as “teaching/learning” activities’; he underlines the importance of ‘minimising the negative washback’ immanent in the Greek language classroom and ‘maximising the positive washback of exams’ and their effectiveness in terms of learning and motivation.

The present study, although restricted by the examination framework, was an attempt to enhance the participants’ intrinsic motivation and to soften the heavy burden of the exams placed on them by introducing different materials and a new method, the promising and ground-breaking DDL approach. It was carried out in a private language school in a relatively small town in the northern part of Greece and it lasted five months, approximately two terms.

3. Method

The following sections describe the population and outline the materials design and procedures involved.

3.1 Participants

A total of 28 young learners formed two groups comprising a control and an experimental group. The former consisted of 6 male and 8 female students and the latter was composed of 7 male and 7 female students aged between 13 and 15. The participants were mostly Greek but there were also some students of mixed nationalities [1]; the first group consisted of twelve Greek students including one ‘Greek Pontian’ from the former Soviet Union (see Palaiologou 2007: 99), one Albanian, and one Armenian and the second one was comprised of one Georgian, two Albanian and eleven Greek students (see Table 1 and 2).

 

Sex [2]

Age

Nationality

S#1 [3]

F

14

Greek Pontian

S#2

M

14

Greek

S#3

F

14

Greek

S#4

F

14

Greek

S#5

M

14

Greek

S#6

F

14

Albanian

S#7

F

14

Greek

S#8

M

15

Armenian

S#9

F

14

Greek

S#10

F

14

Greek

S#11

M

14

Greek

S#12

M

14

Greek

S#13

M

14

Greek

S#14

F

14

Greek

Table 1. List of students of the control group

 

Sex

Age

Nationality

S#1

M

14

Greek

S#2

M

14

Greek

S#3

F

15

Greek

S#4

F

14

Greek

S#5

F

14

Georgian

S#6

F

14

Greek

S#7

M

14

Greek

S#8

F

13

Greek

S#9

M

14

Greek

S#10

F

14

Greek

S#11

M

14

Greek

S#12

M

14

Greek

S#13

F

14

Albanian

S#14

M

14

Albanian

Table 2. List of students of the experimental group

The division into these two groups was based on the order of enrolment, on a first come first served basis, and their preference for the schedule, either the early evening or the late evening course, that was more convenient to them. Both classes consisted of mixed ability students who attended mostly the second year of compulsory secondary school (8th grade), with the exception of one student who was in the first year (7th grade) and another two students who attended the first year of post-compulsory school (10th grade).

They were all in the sixth year of EFL learning, B1 level, and their main short-term learning objective was to obtain a B1 certificate in English, according to the Common European Framework, at the end of the academic year. To this end, they attended six 50-minute evening sessions at the private school three times per week including a grammar session of one academic hour for each group. Students of this level have already acquired many aspects of grammar during the previous years. This class, then, is the last year of revision of the main grammatical points, but introduction to a few more advanced grammatical features is also emphasised in the syllabus (see section 3.3).

3.2 Nature of the data

The control group received conventional instruction using a grammar textbook, while the experimental group navigated through concordance-based materials. Prepared printouts were thought to be more appropriate than hands-on concordances given the age of the participants, the fact that they were introduced to a new approach and new materials and the limited time frame of the study. Boulton (2010: 534) suggests ‘taking the computer out of the equation’ at the start before accessing corpus data directly, an idea which was initially introduced by Johns (1991: 31), the main inspiration of the DDL movement: ‘experience in using concordance data reactively has indicated that it could be used proactively also in a more traditional teacher-centred setting and has suggested also a range of concordance-based exercise types’.

The design of the concordance-based tasks was based on two online corpora, COBUILD and the BNC, but also on CANCODE. [4] The selection of these particular types of corpora was based on certain criteria. The first two were freely available online and therefore easily accessible, which offered any motivated learners the possibility to follow-up with their own searches. COBUILD contains both British and American English, written and transcribed speech, with output available in KWIC concordance format and the query syntax allows the user to specify word combinations, wildcards, part-of-speech tags, and so on. The BNC corpus is a collection of spoken and written samples of current British English and the query results appear on the screen in full concordance format, which is more manageable for beginners. CANCODE is an example of a specialised corpus as it includes a wide selection of authentic spoken discourse and conversational extracts of different spoken genres. It allows observation of query results in KWIC format and access to the full transcribed conversation of each concordance line on request, while it supplies information about the type and setting of the conversation under examination as well as the age and social status of the interlocutors. Therefore, the corpus data employed in the DDL tasks ranged from full concordances to KWIC concordance format and conversational extracts, and offered varied opportunities for grammar observation and for noticing of forms of spoken grammar (see Carter 2004: 30–34).

3.3 Materials design

The corpus-based grammar units designed for the study were determined by the syllabus and were twenty in number. The order of the grammatical items and patterns examined during the grammar sessions followed the sequence of the DDL units and therefore both groups explored the same grammatical feature every week. Since the conventional grammar books adopt a system of twelve ‘tenses’ in line with the ELT materials and textbooks employed in Greece (see Gabrielatos 2004), all tense-aspect combinations were consequently thought to be better identified with the familiar term ‘tenses’, including the tense-aspect combinations expressing futurity, for both groups during the study, for the sake of the students’ better understanding.

The main focus of the first part of the units, 1 to 12, was on tense and time aspects, which were examined during the first term. The contents of the second section, units 13 to 15, were processed in the second term and included conditionals, passive voice and reported speech, and units 16 to 20 introduced additional grammatical items and patterns: gerund and infinitive, would rather and had better, pseudo-passive with have, or causative form as is presented in the grammar books, and modal verbs. A breakdown of the contents is given in Table 3.

DDL UNITS

GRAMMATICAL FOCUS
INTRODUCTION DDL notions
UNIT 1 Splang you like bananas? Simple Present
UNIT 2 He is constantly talking on the phone! Present Progressive
UNIT 3 Do you think she is seeing another man? Stative Verbs
UNIT 4 Did you know he lost all his property? Simple Past
UNIT 5 Have you ever been to England? Present Perfect Simple
UNIT 6 I have been crying for help for over an hour but in vain! Present Perfect Progressive
READING AND WRITING Tense and Aspects
UNIT 7 I used to get up early in the morning! Simple Past - Used to
UNIT 8 What were you thinking of when you answered the door? Past Progressive
UNIT 9 This is the last warning! Stay here till she comes back. Adverbial Clauses of time
UNIT 10 I had had the money but… no luck! Past Perfect
UNIT 11 I had been trying to learn Thai for 6 years Past Perfect Progressive
UNIT 12 Will you have a look at this? Simple Future
REVISION Units 1–12
UNIT 13 If I won the Lottery… Conditional clauses
UNIT 14 Call the police! The Smiths’ house has been broken into! Passive Voice
UNIT 15 He offered to give me a lift but I should have walked instead! Reported Speech
READING AND WRITING Conditionals, Passive Voice, Reported Speech
UNIT 16 I miss having everything done for me! Gerund
UNIT 17 My job is to answer the telephone inquiries and your duty is to fill in the forms. Infinitive
UNIT 18 Ready…Set…Go! would rather-had better
UNIT 19 I’ll have my hair cut exactly like David Beckam’s pseudo-passive with have
UNIT 20 I should have known better than to invite her to my party. Modal Verbs
REVISION Units 13–20

Table 3. Summary of grammar focus of the DDL units

The nature of tasks promoted the inductive approach and the lead-in tasks invited students to observe a restricted amount of data and make inferences in terms of form and function (see Figure 1).

Figure 1. Extract from DDL unit 10 based on CANCODE © Cambridge University Press

A variety of tasks was also designed in an attempt to appeal to different ways of processing of information. However, learning styles and intelligences were not measured, since participating in an experiment was a new experience for the students; the new method and materials were enough of a challenge for this young age group, and consequently additional questionnaires at the beginning were thought to add undue pressure to them. It is believed, though, that every human does not possess necessarily only one learning style, although sometimes there is a marked tendency to use one rather than the other depending on the nature of the task, and ‘all students are quite capable of using both analytic and relational faculties’ (see Kinsella & Sherak 1998: 91). Similarly, Boulton (2009: 87) points out that learning styles are not static, but are ‘subject to change along with the various learning experiences’. In this vein, different corpus-based activities were designed for each unit such as problem-solving and game-like tasks, guessing mystery words or making one choice for multiple contexts, multiple matching, error correction, production of short stretches of language and activities involving translation, when mismatches between mother tongue and target language impeded, taking advantage of the well organised neurolinguistic system that already exists in L1. An example of a game-like gap-fill task on reporting verbs and speech acts is illustrated in Figure 2.

Figure 2. Extract from DDL unit 15 based on COBUILD

Many opportunities were also given for pair or group work and consequently different types of learners cooperated in order to perform a task. Figure 3 is an example of pair-work comparison activity of two structures of used to and contrastive analysis of the same grammar pattern in L1 and L2 raising language awareness.

Figure 3. Extract from DDL unit 7 based on CANCODE

Each unit did not exceed a ceiling of five tasks, no more than three pages, making the content easier to deal with and the tasks more attainable. Four revision sections (see Table 3; Reading and Writing, and Revision) were designed in order to help participants to consolidate and reconstruct the already processed information a few units after the learning episode, giving time to learners to take it in.

3.4 Pre-editing of data

All concordance lines and different forms of context employed in the activities were carefully selected to illustrate the intended grammar aspect as clearly as possible. Concordance lines with non-cohesive text or with very advanced vocabulary were excluded. In addition, some pre-editing was considered necessary, although it could be argued that editing of corpus data questions its authenticity. It is important, nonetheless, to take into consideration the needs of the target group in the process of materials design. In the case of adolescent learners minimal mediation (see Example 1 and 2) was advisable in order to ease their way in the observation of the innovative data and to make the tasks as minimally daunting as possible. Carter (1998: 52) suggests ‘text modification’ and ‘re-modelling’ of data as ‘a middle ground between authentic and concocted data’ in the preparation of discourse grammar materials. McCarthy (2004: 6) highlights the need for corpus informed materials looking like traditionally prepared materials and points out that even ‘textbook writers observe usage in corpora and adapt corpus texts and conversations so that they will not be intimidating or confusing for learners’. In this vein, words particularly from the beginning or/and the end of the concordance lines were in some cases omitted, if they did not contribute to meaning, in order to help students focus on the part of the concordance line, which illustrated the grammar item/pattern. More examples of remodelled data aimed at reducing the pressure on the learner’s task of interpreting corpus input, such as in the process of pre-editing conversational extracts taken from CANCODE © Cambridge University Press. Most transcription symbols were omitted, so that students were not distracted by them but could focus instead on the rest of the context, while highly complicated and anomalous forms of spoken discourse were removed in just a few cases for the sake of a better understanding of the articulated message (see Example 1 and 2).

Example 1. Original extract from CANCODE © Cambridge University Press
1. <$1> <$H> Early. </$H>
2. <$2> No. The only reason we have to get up early is for the dog.
3. <$1> Oh. <$E> laughs </$E>
4. <$2> Let him out. So. <$=> <$G2> though when you get when you're used to waking up
5. early all the week long you don't </$=> <$1> I wake up early. I always wake up at half
6.  past five regardless of+
7. <$2> Mm.
8. <$1> +what day it is. But <$=> I can </$=> I'm one of these people that can go back
9. to sleep and wake up at like eight o'clock.
10. <$2> Yeah. It'll be about eight o'clock when I get up.
Example 2. Remodelled extract for use in DDL Unit 7
1. <$1> Early.
2. <$2> No. The only reason we have to get up early is for the dog.
3. <$1> Oh. (laughs)
4. <$2> Let him out. So when you get when you're used to waking up early all the week
5. long you don’t
6. <$1> I wake up early. I always wake up at half past five regardless of+
7. <$2> Mm.
8. <$1> +what day it is. But I can I'm one of these people that can go back to sleep and
9. wake up at like eight o'clock.
10. <$2> Yeah. It'll be about eight o'clock when I get up.

It is not clear whether the word so is used as a discourse marker, given the capital letter and the fullstop, or as a conjunction (see Example 1 line 4). It seems that it is related to the following utterance, although two more conjunctions follow, though and when. It is also observed that the message is not completed, as the utterance is interrupted by the first speaker’s turn (see Example 1 lines 5 and 6 utterance displayed in blue). In the remodelled extract the conjunction though was omitted and the remaining conjunctions were joined together retaining the spoken mechanism of relexicalisation that follows (see Example 2 line 4), while the interruption was made more distinct by separating the speakers’ turns (see Example 2 line 6). Therefore, the remodelling of data did not disturb the characteristics of naturally-occurring discourse of the original extract (see Carter 1998: 47), such as relexicalisation, interruption, the contentless utterance Mm, which indicates that contact is being maintained (Example 2 line 7) and the continuation of the first speaker’s turn (see Example 2 lines 6 and 8). Thus, learners could observe and comment on these mechanisms commonly used in everyday conversations to communicate meaning, while they were involved in a grammar comparison activity already presented in Figure 3.

4. Triangulation

A combination of different methods of data collection was thought to give a more rounded picture of the participants’ attitudes and evaluation of materials, as well as of the impact of DDL on motivation, learning and performance. To this end, the triangulation of data through classroom observation, questionnaires, interviews and tests, was considered a good basis for generating insights. Transcripts of grammar sessions offered insights into the students’ interactional competence and response to the DDL tasks (see section 5). In addition, two questionnaires were distributed to the two groups at the end of the study, which can be divided into two broad sections. The first section, based on the Likert scale, elicited the learners’ attitudes towards studying grammar and their preferences for the inductive and/or the deductive approach. The participants were also invited to rate the impact of both modes of teaching on the learning process. The second section surveyed the students’ evaluations of the grammar units identifying problems, preferences and level of difficulty through closed and open-ended questions.

Towards the completion of the study, interviews were also conducted with all students of both the experimental and the control group. After the questionnaires were distributed and completed in class, each student was invited to a one-to-one discussion with the teacher. The interview questions followed the order of questions presented in the questionnaire aiming at eliciting more feedback from the participants. The integration of interviews was also considered necessary in order to clarify any possible inaccurate or incomplete responses due to lower proficiency interference, as the questionnaires were completed in the target language, or to elicit attitudes that might not be directly observable through classroom observation or questionnaires.

The qualitative research was also complemented with quantitative data drawn from tests, whichoffered an objective report of inter-group and intra-group performances. They were not matched tests in the experimental sense as the teaching context meant that they had to be designed based on the demands of the course, according to which each test examines the input processed during the academic term(s), and consequently each test represents a gradual increase of level of difficulty and a cumulative content. One test preceded the empirical study and two tests were taken after the completion of the first and second term respectively. The third test, in particular, required students to revise almost all the materials being processed throughout the grammar study. The tests were common to both groups so that the results were comparable and the types of activities were in line with the framework of the official examinations, such as gap-fill, key word transformation and error correction. The results give an indication of the degree to which the students were able to cope with and gain an understanding of the progressive input of grammatical items which their syllabus dictated, against the background of two different modes of teaching and two different sets of material, seen through the lens of their general scoring profiles on different occasions.

It must be noted that the questionnaires and tests administered during the study were not scientifically designed and they were not tested for statistical significance. Brief questionnaires were used instead in order to be more manageable for participants of this young age group and were designed to get students thinking and responding in relation to the new learning experience. The tests were graded, as regards the level of difficulty, in line with the regulations of the course and provided an underpinning to the direct reports of the students’ experiences and to the observations of events in the classroom during the teaching programme.

5. Corpus application and classroom interaction

There were many opportunities for participants to make inferences not only in terms of grammar but also in relation to context and meaning, such as during whole-class discussions based on corpus input of the lead-in tasks of each unit. The learners started to become more observant and ‘notice’ input, as in the following brief example of a student’s effort to construct her own context when observing concordances (see Transcribed Extract 1).

Transcribed Extract 1 [5]
1. T: b…by the time we left it had been there several years
2. What do you think they are talking about?
3. Ss: <Pause>
4. T: There is no right or wrong answer. It’s just for you to decide.
5. So what could it be?
6. S#13: A car
7. T: Right it could be an old wrecked car which was parked outside the building
8. S#13: Antique
9. T: Yes and we abandoned it and nobody cared about it. So it had been there
10. several years.

The teacher’s initial question (see line 2) is modified lexically after a short silent period followed by an additional comment emphasising the nature of the referential question and urging students to respond according to their intuition (see lines 4 and 5). The reply (see line 6) is verified by the teacher in the follow-up move, providing further imaginary description of the car and extending this way the student’s contribution (see line 7). The student, urged by the teacher’s reaction, adds more information in a successful turn-taking attempt (see line 8), which leads to another follow-up move expanding the initial answer (see lines 9 and 10).

The participants were also invited to work in pairs and perform completion tasks in different DDL units, such as the task on introductory verbs used in Indirect Speech (see Figure 2). After giving the pairs some time to examine the sets of concordances, the teacher invited them to provide their answers (see Transcribed Extract 2).

Transcribed Extract 2
1. T: Let’s read the first three lines
2. I would be greatly offended if a guest…to pay for anything.
3. Offended?
4. S#?: (translates)
5. T: Yes insulted.
6. I was so thrilled at myself I…to pay in advance. A huge meal
7. I think I've given my best and I…to take less money. Meanwhile
8. Which is the missing word here?
9. Ss: <Pause>
10. T: What part of speech do we need?
11. S#4: Verb
12. T: So which is the word that is common to all three lines?
13. Warned? Denied?
14. S#?: offered
15. T: Offered.  Write it down in all three gaps.  Let’s move on to the next set.
16. think it would happen.  Nadine…to share her prize with her
17. I think.  Before they went they … to come back soon. One could
18. party system.  The president … to hold a meeting on the issue 
19. What is the missing word?
20. S#8: Denied
21. T: Denied okay it could be used in the first line but in the second before they went they denied to
22. come back soon I think we need a positive meaning
23. S#6: Promised
24. T: Yes write it down. The next
25. S#?:                                    ⌊refused⌋
26. T: Exactly
27. He refused to stop. She refused to pay more for another instructor and he refused to believe that
28. probably something wrong happened to his father.
29. The next set now.
30. friends Burned Man and Oracle often…me to be honest with myself and
31. S#10: advised
32. T: advised yes let’s check the rest of the lines
33. The doctors advised me to weigh up the quality of life
34. he didn't think I needed surgery. He
35. S#5:   ⌊advised⌋
36. T: spelled with an s or a c?
37. S#5:   With an s
38. T: as it is a verb and not a noun. Let’s move on
39. The London Business School…that unemployment would rise to one
40. S#10: warned
41. T: Very good.
42. who was also a hypnotherapist but warned me that it would not be easy to
43. Number 6
44. my time's up. So when a friend…me to join her women's theatre-
45. What did that friend do to
46. S#11:                              ⌊ invite ⌋
47. T: Invit↑…
48. S#11: ed
49. T: Invited very good.

The students are slightly reluctant in the beginning, but from line 14 onwards they are more cooperative and provide all answers effortlessly. After making sure what part of speech is needed in each set of concordances with a reformulation of the initial question (see lines 10 to 13), the teacher elicits the answers for each set separately. Student 8 suggests an introductory verb that is not possible for all three lines (see line 20), so the teacher stresses the need for positive meaning (see lines 21 and 22), but the rest of the learners’ replies are correct and come out naturally. Lines 25, 35 and 46 present spontaneous answers, taking the turn successfully from the teacher, and are believed to be the outcome of the effective pair-work.

The final part of the lesson involves the interpretation of the verb patterns (see Transcribed Extract 3).

Transcribed Extract 3
1. T:  All these words we wrote replace which words we have learned so far?
2. S#?: told
3. T: Yes…said told asked very very good and what sort of words come
4.  after the ones we wrote down?  What is their structure?  Look at them
5.  vertically.  What word follows them?
6. S#?:  To
7. T: Right infinitive
8. S#?: Me
9. T: Pronoun and what else?
10. S#?: Him
11. T: Yes another object pronoun.  And even the word↑…
12. Ss: <Pause>
13. T: that.  Right. 

The learners may have not provided the grammar terms needed but they seem to have realised their function and the deviation from the common introductory verbs. Therefore, by the time they reached unit 15, they had developed the capacity to cope with incomplete concordances to a certain extent or at least to compromise with their complicated appearance. The above task promotes communication between learners and, although the concordances are not further sorted (they are only sorted first word to the right), they manage to observe the patterns and draw their first conclusions.

The students’ replies are brief in all transcribed extracts presented in this section, as they are guided by the teacher’s controlled questions, and the teacher accepts even one-word responses following the IRF model (Initiation-Response and Follow up; see Sinclair and Coulthard 1975), but there are also spontaneous utterances which are indicative of desirable friendly classroom conditions that put most students at ease. The teacher’s contributions contain interactional features, such as display questions, direct repair, extended teacher turn (see Walsh 2006) aiming at facilitating learners to produce the newly acquired grammar pattern. ‘Feeding in’ with essential comments and explanations is indicative of scaffolding, an important mechanism in providing learners with ‘cognitive support through dialogue as they engage in tasks that may lie outside their capabilities’ (Walsh 2006: 120). According to Vygotsky’s zone of proximal development (ZPD) paradigm, scaffolding has an important role when the input is higher than the students’ current stage of learning, which is the case with the corpus-based materials (Vygotsky 1978).

6. Materials evaluation

The feedback from a few of the 18 statements and questions displayed in the questionnaires are discussed in this section with a focus on the accounts and reports of the participants of the experimental group.

The first statement of both questionnaires asked students to rate their attitude towards grammar seeking to investigate the degree of motivation after the completion of the study (see Figure 4).

Figure 4. Attitudes towards grammar acquisition

The dispersion of ratings suggests that some participants found studying grammar enjoyable, while others were not fond of it. Closer observation of the choices of the experimental group shows that there was no complete rejection of grammar, while a few students totally agreed with the statement. The majority of the participants of the control group agreed to a certain extent, while approximately one third of the class disagreed either totally or to some extent.

The participants of the experimental group were invited to evaluate the effectiveness of the DDL units and the use of concordances on an agreement scale with regard to learning (see Figure 5).

Figure 5. Ratings on the contribution of concordances to learning

The ratings suggest that most students acknowledged either totally or to a certain extent the contribution of corpora to the learning process and yet some students totally disagreed. Similarly, half of the participants admitted that they managed to enhance their inductive skills (either totally agreed or to some extent) but a considerable number of students had a different opinion (see Figure 6).

Figure 6: Ratings on the inductive approach

The following interview extract presents a student’s interesting comment on a corpus-based activity, which invited the learners to observe a set of concordances taken from COBUILD and complete a table on the formation of Passive Voice (see Interview Extract 1). The student seems to have benefited from the inductive approach of the unit.

Interview Extract 1
T: Now th the exercises that I had given you like this exercise ehm…did they actually help you learn…like all these sheets that I have given you
MS14 EG [6; see section 7.1]: Yes they helped me a lot and especially passive voice
T: Passive voice.  How did they help you?
MS14 EG: Because it says about…tenses…it says simple present am is are plus PP (means Past Participle)
T: But you actually wrote the table.
MS14 EG: Yes that’s why I remember it.

From further ratings on similar statements, as well as from the interviews, it was observed that the majority of the students expressed their preference for the combination of the inductive and deductive approach. This brings forward the issue of age restriction on the one hand and their general educational experience on the other, according to which they expect teachers to teach them things rather than them learning independently. And yet, the students’ preference for a combination of the deductive and inductive approach can possibly be regarded as a step closer to the intended result, that is the inductive approach, given the prominent deductive nature of their previous learning experience.

However, fewer participants agreed that concordances motivated them to study grammar further, while rank 3, the neutral option, received the highest percentage revealing their indecision and uncertainty (see Figure 7).

Figure 7. Degree of motivation to study grammar

There were also opposing views with regard to the level of difficulty of the format and content of concordances (see Figure 8).

Figure 8. Ratings on the format and content of concordances

According to the classroom observation and the students’ reports, their main difficulty was the KWIC format of concordances, as is indicated in the following three representative extracts, which are taken from the students’ written responses to the open-ended question regarding the difficulties that arose when dealing with concordances (see Written Extracts 1, 2 and 3).

Written Extract 1
HS3: Sometimes I didn’t understand what the sentence meant.
Written Extract 2
HS5: They were difficult and hard to understand.
Written Extract 3
HS8: The problems were that some sentences are not complete and sometimes I can’t understand.

The interviews elicited more details about the problems they had to deal with when observing the new input and the outcome of the one-to-one discussions concerned not only the appearance of concordances but the complexity of vocabulary as well (see Interview Extracts 2, 3 and 4).

Interview Extract 2
T: Did you find the activities I gave you difficult?
HS3: Yes some of them.
T: What made them difficult?
HS3: Sometimes I don’t understand the sentences
T: Why?
HS3: Because…there were more…unusual words.
Interview Extract 3
HS10: If we don’t know if I don’t know one word I can ask you but if I don’t understand what it all means…
T: You mean the context?
HS10: Yes
T: <pause> although I may give you the meaning of the unknown words you still have difficulty
HS10: Yes sometimes
T: So do you prefer to have a grammar book?
HS10: Yes
T: Why is that?
HS10: Because…ehm…it…it shows us the rules…and sometimes it has got translation…so we can…
T: Understand it better
HS10: Understand it better and ehm…it is difficult+
T:                                                                            ⌊To find your way on your own⌋
HS10: +and it is difficult without… ehm…examples
Interview Extract 4
T: Did you find the exercises I gave you difficult?
HS13: ahm…some of them
T: What made them so difficult?
HS13: Some words that…we didn’t know them.
T: If I give you the meanings of the words could you do the exercise?
HS13: Yes I think I can do it.
T: But can’t you ignore these unknown words?
HS13: No
T: You cannot understand the sentence you need to know the meaning of the
HS13:                                                                                                        ⌊Because if I don’t know the words I cannot understand the meaning⌋
T: What about the context?
HS13: ah…sometimes
T: It depends on the sentence
HS13: Yes

An interesting picture emerged when students were asked whether they preferred the conventional way of grammar learning or concordance-based tasks (see Figure 9).

Figure 9: Preferences for conventional learning or DDL

Corpora were ranked marginally higher, but some participants of the experimental group suggested combining the concordance-based units with the grammar book, although it was not an option in the initial question, expressing their wish to include their previous conventional grammar learning experience in the corpus-based method. The following written extracts are a brief indication of the learners’ self-awareness (see Written Extracts 4, 5 and 6).

Written extract 4
HS5: I prefer the concordance units because they have all the exercises we need to do for our level. The grammar book has a lot of easy exercises and we don’t do them so it’s better to do exercises you give us, the most important exercises.
Written extract 5
HS13: I like the grammar book because it give us examples to understand. And I like the concordance too because it help us with the spoken language and is different from the lessons we do at class.
Written extract 6
MS11: I prefer the grammar book because I understand the units there more and because the concordances are more difficult

These extracts represent the three prevailing responses which are depicted in Figure 9. Students 5 and 13 were both high scorers and seemed to have appreciated the effectiveness of a corpus-based approach to grammar. In particular, the former considered the traditional grammar book and its mainly mechanical exercises simple, as opposed to the corpus-based activities which were appropriate for their level. The latter believed that the grammar book and the corpora can supplement one another, as the book provides thorough explanation, whereas the concordances are an innovative way of being introduced to spoken language. Student 11, on the other hand, made a straightforward comment and expressed his preference for the grammar book as a more understandable and less complicated approach to grammar.

The last question invited the participants of the experimental group to state whether they wished to access corpora the following year or not and, among the 14 participants, 10 of them chose to try it again. This is a rather interesting proportion considering the difficulties they experienced while being exposed to corpus-based activities for the first time. It certainly takes time to familiarise oneself with the nature of corpus data and be in the position to observe and make generalisations and convert input into output.

7. Test results

The tests represented a natural part of the learning and assessment experience for these young learners, who were accustomed to doing progressive tests during the ongoing terms. Table 4 presents the mean of both groups in each of the three tests.

CONTROL GROUP
MEAN (± SD)

EXPERIMENTAL GROUP
MEAN (± SD)

TEST 1

59.02   (± 28.41)

63.13   (± 27)

TEST 2

61.24   (± 21.70)

62.6   (± 23.38)

TEST 3

48.29   (± 29.59)

60.89   (± 21.74)

Table 4. Test results

The test results suggest that in test 1 the experimental group outperformed the control group, as the mean for the former was 63.13 out of 100 points as opposed to the latter, which reached 59.02. At the end of the first term the control group scored 61.24, while the mean of the experimental group was 62.6, and yet the experimental group performed slightly better than the control group. An unexpected picture emerges when comparing the results in test 3, which was based on almost all the grammatical items examined during the study, as the control group achieved 48.29 crossing the borderline of low scoring, while the experimental group scored 60.89.

7.1 Quantitative analysis of intra-group test performances

A comparison of performances of each group combining two tests at a time can offer further insights into the extent to which the input processed in each term had been taken up (see Table 5, 6, 7, 8, 9 and 10). The symbol ^ indicates the upward trend of performance of individual learners, while there is a separate column presenting the differences in scoring. The abbreviations (see notes 3 and 6) used in the students’ column are the same as those applied earlier in the written and interview extracts.

Students

Test 1

Test 2

Differences

Upward trend

MS1

63.6

66.6

3

^

LS2

45.4

47.9

2.5

^

LS3

45.4

66.6

21.2

^

LS4

45.4

52

6.6

^

MS5

62.5

31

-31.5

 

HS6

90

85.4

-4.6

 

LS7

18

52

34

^

HS8

100

91.5

-8.5

 

HS9

71

70.8

-0.2

 

HS10

85

77

-8

 

HS11

95

87.5

-7.5

 

MS12

69

62.5

-6.5

 

LS13

18

14.5

-3.5

 

LS14

18

52

34

^

Sum

826.3

857.3

31

 

Mean

59.02

61.24

2.21

 

Variation

806.87

470.84

308.62

 

Standard Deviation

28.41

21.70

17.57

 

Table 5. Performance of the control group in test 1 and test 2

Students

Test 1

Test 2

Difference

Upward trend

MS1

51

68.7

17.7

^

MS2

65.4

56.2

-9.2

 

HS3

94.5

95.8

1.3

^

LS4

18

18.7

0.7

^

HS5

100

83.3

-16.7

 

HS6

92

77

-15

 

LS7

18

20.8

2.8

^

HS8

83

70.8

-12.2

 

LS9

45.4

56.2

10.8

^

HS10

78

79

1

^

MS11

54.5

60.4

5.9

^

LS12

36

35.4

-0.6

 

HS13

81

83.3

2.3

^

MS14

67

70.8

3.8

^

Sum

883.8

876.4

-7.4

 

Mean

63.13

62.60

-0.53

 

Variation

729.06

546.79

94.56

 

Standard Deviation

27.00

23.38

9.72

 

Table 6. Performance of the experimental group in test 1 and test 2

The control group’s average was higher in test 2 than in test 1, while the experimental group’s was slightly lower. However, close observation of intra-group performances suggests that there is a marked upward trend in the experimental group, while the positive change is less obvious in the control group (see Table 5 and 6). The increase of the overall performance of the latter is therefore attributed to the greater scoring differences of individual students. What is of considerable interest is that the participants of the control group that raised their scores were mainly low scorers and particularly students 3, 4, 7 and 14 who managed to cross the borderline of medium scoring (see Table 5). On the other hand, the results of high competent students slightly decreased, which was expected as their initial marks were quite high, 90, 100, 85 and 95 points, with the exception of high scorer 9 whose initial performance was just above 70 points but managed to remain at about the same level in test 2. Similarly, two of the three medium scorers’ marks dropped, and particularly student 5, and only student 1 raised her score.

In the experimental group, there was no marked upward change in scoring differences, with the exception of medium scorer 1 with 17.7 points of difference and low scorer 9 with 10.8 (see Table 6). In the group of high scorers a small upward trend is observed in three out of six students, while the performance of the remaining three, students 5, 6 and 8, decreased with a difference of -16.7, -15 and -12.2. Their results in test 1 were high, 100, 92 and 83, respectively, and equivalent to those of the control group. Overall, nine students of different scoring groups raised their scores from test 1 to test 2 and six of the control group.

However, the scores of the participants of the control group dropped in test 3, with the exception of student 7 with a scoring difference of 27 points (see Table 7). When comparing performances in test 2 and test 3 an upward trend in two high scorers’ performance is observed, whereas students 3 and 14, low scorers, and student 9, a high scorer, performed badly with great scoring differences (see Table 9).

Students

Test 1

Test 3

Difference

Upward Trend

MS1

63.6

60

-3.6

 

LS2

45.4

40

-5.4

 

LS3

45.4

20

-25.4

 

LS4

45.4

30

-15.4

 

MS5

62.5

11

-51.5

 

HS6

90

85

-5

 

LS7

18

45

27

^

HS8

100

87.5

-12.5

 

HS9

71

45

-26

 

HS10

85

82.5

-2.5

 

HS11

95

90

-5

 

MS12

69

57.5

-11.5

 

LS13

18

10

-8

 

LS14

18

12.5

-5.5

 

Sum

826.3

676

-150.3

 

Mean

59.02

48.29

-10.74

 

Variation

806.87

875.37

292.85

 

Standard Deviation

28.41

29.59

17.11

 

Table 7. Performance of the control group in test 1 and test 3

Students

Test 1

Test 3

Difference

Upward Trend

MS1

51

65

14

^

MS2

65.4

60

-5.4

 

HS3

94.5

85

-9.5

 

LS4

18

67.5

49.5

^

HS5

100

85

-15

 

HS6

92

87.5

-4.5

 

LS7

18

25

7

^

HS8

83

75

-8

 

LS9

45.4

20

-25.4

 

HS10

78

62.5

-15.5

 

MS11

54.5

40

-14.5

 

LS12

36

52.5

16.5

^

HS13

81

80

-1

 

MS14

67

47.5

-19.5

 

Sum

883.8

852.5

-31.3

 

Mean

63.13

60.89

-2.24

 

Variation

729.06

472.70

369.47

 

Standard Deviation

27.00

21.74

19.22

 

Table 8. Performance of the experimental group in test 1 and test 3

Students

Test 2

Test 3

Difference

Upward trend

MS1

66.6

60

-6.6

 

LS2

47.9

40

-7.9

 

LS3

66.6

20

-46.6

 

LS4

52

30

-22

 

MS5

31

11

-20

 

HS6

85.4

85

-0.4

 

LS7

52

45

-7

 

HS8

91.5

87.5

-4

 

HS9

70.8

45

-25.8

 

HS10

77

82.5

5.5

^

HS11

87.5

90

2.5

^

MS12

62.5

57.5

-5

 

LS13

14.5

10

-4.5

 

LS14

52

12.5

-39.5

 

Sum

857.3

676

-181.3

 

Mean

61.24

48.29

-12.95

 

Variation

470.84

875.37

245.12

 

Standard Deviation

21.70

29.59

15.66

 

Table 9. Performance of the control group in test 2 and test 3

Students

Test 2

Test 3

Difference

Upward trend

MS1

68.7

65

-3.7

 

MS2

56.2

60

3.8

^

HS3

95.8

85

-10.8

 

LS4

18.7

67.5

48.8

^

HS5

83.3

85

1.7

^

HS6

77

87.5

10.5

^

LS7

20.8

25

4.2

^

HS8

70.8

75

4.2

^

LS9

56.2

20

-36.2

 

HS10

79

62.5

-16.5

 

MS11

60.4

40

-20.4

 

LS12

35.4

52.5

17.1

^

HS13

83.3

80

-3.3

 

MS14

70.8

47.5

-23.3

 

Sum

876.4

852.5

-23.9

 

Mean

62.60

60.89

-1.71

 

Variation

546.79

472.70

421.45

 

Standard Deviation

23.38

21.74

20.53

 

Table 10. Performance of the experimental group in test 2 and test 3

As mentioned in section 7, the average of the experimental group fell slightly in the last two tests. When comparing test 1 with test 3, in particular, a marked decline in the number of participants who raised their scores is observed (see Table 8). Only four students managed to score higher, one medium scorer and three low scorers, while student 4 in particular, who was considered a low scorer initially, managed to approach the scale of high scoring. Observing the scores from test 2 to test 3, on the other hand, which was designed to examine students in most of the grammatical aspects encountered during the study, the performance of three high scorers, along with one medium and three low scorers of the experimental group increased (see Table 10).

8. Discussion

The students’ preferences and evaluation of the DDL materials and method reported in the questionnaires and interviews can be summarised as follows: Some students were motivated to study grammar after the completion of the study and some others not, but most students remained neutral. The majority of the participants had acknowledged to a certain extent the contribution and potential of corpora but also pointed out the difficulties when involved in DDL, such as the KWIC format of concordances and unknown vocabulary. Most students chose to have access to corpora the following year and also expressed their preference for DDL tasks, but the need for teacher guidance and the combination of the traditional grammar book with the DDL tasks were also underlined.

According to the test results, great variance and sudden changes, both upward and downward, were observed in individual performances of both groups and of all three scoring groups, which consequently affected the mean greatly since it is a small-scale study. Such performances are not surprising, though, as adolescent learners are characterised by unpredictable behaviour and limited attention span and experience difficulty in recalling information (see Jensen 2005: 30, Leiguarda 2005); therefore unpredictable and unstable performances were expected. It is consequently difficult to tell whether the new and revised input had been taken up by students whose performance was greatly unstable, but perhaps it can be assumed that the moderate upward trend of performances and the scores that remained at about the same level could be a positive indication that some students had started to apprehend the function of some grammar features and benefited from the grammar sessions, given that each test was more demanding than the previous one. In this vein, the fact that the overall performance of the experimental group appeared to be steadier than the control group, as they did not present such severe decline, and the fluctuations of individual performances were more moderate, is an interesting observation after the completion of the DDL course with regard to the impact of corpora on the learning of grammar.

9. Conclusion

Based on the qualitative evidence further adjustments to corpus materials and course design are considered necessary. The use of grammar materials based on full concordances or conversational extracts, for example, for a certain amount of time would be more preferable, which could gradually lead to a smooth transition to the KWIC concordance format and to a more inductive approach to corpus data. In addition, a total abandonment of the conventional grammar book is perhaps not recommended with this age group, at least at the beginning of the study, as suggested by the participants, and thus DDL can initially be introduced as a complement to the already existent and approved teaching methods and materials (see Chambers 2005: 122, Meunier 2002: 135). The grammar book could be used initially for follow-up activities with the already induced grammar patterns. It could gradually be limited to a form of consultation and finally eliminated when learners become comfortable and familiar with the corpus material.

The students’ learning habits were not expected to change radically with the new corpus-based experience in such a limited time frame, but it was observed that students started getting a ‘feel’ for this new learning methodology and its potential as well as its difficulties. They may have not managed to become independent explorers of concordances but they started to ‘notice’ corpus input and tried to make some sense of it. Some students experienced more difficulty when coping with the complex format of concordances, while others were more comfortable with the inductive approach and yet the majority of students expressed their preference for concordance-based tasks and further access to corpora. Such tendencies were important gains and valuable feedback. Therefore, the present qualitative study could represent a preliminary step in the development of corpus-based grammar teaching to EFL learners within this age group, while further research could investigate the hypothesis that DDL is beneficial and provide evidence collected from empirical studies.

Acknowledgements

I would like to express my appreciation to Professor Michael McCarthy and Professor Ronald Carter for the continuous support and inspiration during the research study.  I would also like to thank Dr Luke Prodromou for his insightful comments.  Special thanks go to the students who participated in the research study and were willing to experiment with an innovative method despite the pressure of the official exams.  My gratitude to Dimitris Barlagiannis and Maria Pateinaki for their full support and advice.

This publication has made use of the Cambridge and Nottingham Corpus of Discourse in English (CANCODE). CANCODE was funded by Cambridge University Press and is a five-million word computerised corpus of spoken English, made up of recordings from a variety of settings in the countries of the United Kingdom and Ireland. The corpus is designed with a substantial organised database giving information on participants, settings and conversational goals. CANCODE was built by Cambridge University Press and the University of Nottingham and it forms part of the Cambridge International Corpus (CIC). It provides insights into language use, and offers a resource to supplement what is already known about English from other, non-corpus-based research, thereby providing valuable and accurate information for researchers and those preparing teaching materials. Sole copyright of the corpus resides with Cambridge University Press from whom all permission to reproduce material must be obtained.

Notes

[1] These students are second generation immigrants and are assimilated into the Greek culture and educational system. They were all learning English as a foreign language, while three of them were sparsely attending lessons of the language of their origin in one-to-one classes. They were competent students, mostly high-scorers, with the exception of two students who were medium-scorers according to their performance in test 1 (see section 7.1) and their learning progress observed in the previous years.

[2] M stands for male and F more female

[3] S represents each student with a unique identifier number, which is assigned alphabetically according to the initial of their last name.

[4] COBUILD was freely available online at the time the research was carried out (2003-2004). At present, a free trial is offered and subscriptions are available.

[5] Transcription conventions
? high rising intonation at the end of an utterance
falling intonation at the end of an utterance
rising intonation expecting the interlocutor(s) to complete an utterance
short pause or break of less than 1 second within a turn
<pause> pause or break of more than 1 second within a turn
Bold emphatic stress
Italic italicized words indicate reading aloud an extract from the DDL materials
⌊ ⌋ words in these brackets are utterances interrupting another speaker’s turn
+ + continuation of a speaker’s turn after being interrupted by another speaker
( ) italicized words in these parentheses interpret the speaker’s intended messages and indicate the use of mother tongue
/?/ indistinct utterance
T Teacher
Ss Students
S#? student not recognised

[6] H stands for high scorer (who achieves from 70 points out of 100 and above), M for medium (who achieves between 50 and 69 points) and L for low (who achieves from 49 points and below)

Sources

BNC = British National Corpus: http://www.natcorp.ox.ac.uk/

COBUILD = Collins Birmingham University International Language Database: https://www.collins.co.uk/page/Wordbanks+Online (The former URL address, which no longer works, but was available at the time of the research, is http://titania.cobuild.collins.co.uk/form.html)

CANCODE = Cambridge and Nottingham Corpus of Discourse in English © Cambridge University Press (restricted access).

References

Bernardini, Silvia. 2004. “Corpora in the classroom: An overview and some reflections on future developments.” How to Use Corpora in Language Teaching, ed. by John Sinclair, 15–36. Philadelphia: John Benjamins.

Boulton, Alex 2008. “Reaching the parts other teaching can’t reach”. Proceedings of the 8th Teaching and Language Corpora Conference, 2008, ed. by Ana Frankenberg-Garcia, 38–44.Lisbon Portugal: https://hal.archives-ouvertes.fr/hal-00326706/document

Boulton, Alex. 2009. “Data-driven Learning: Reasonable Fears and Rational Reassurance”. Indian Journal of Applied Linguistics 35(1): 81–106.

Boulton, Alex. 2010. “Data-driven learning: Taking the computer out of the equation”. Language Learning 60(3): 534–572.

Braun, Sabine. 2005. “From pedagogically relevant corpora to authentic language learning contents”. ReCALL 17(1): 47–64.

Braun, Sabine. 2007. “Integrating corpus work into secondary education: From data-driven learning to needs-driven corpora”. ReCALL 19(3): 307–328.

Carter, Ronald A. 1998. “Orders of Reality, CANCODE, Communication and Culture”. ELT Journal 52(1): 43–56.

Carter, Ronald. A. 2004. “Grammar and Spoken English”. Applying English Grammar: Functional and Corpus Approaches, ed. by Caroline Coffin, Ann Hewings & Kieran O’Halloran, 25–39. London: The Open University.

Chambers, Angela. 2005. “Integrating corpus consultation in language studies”. Language Learning and Technology 9(2): 111–125.

Cheng, Winnie, Martin Warren & Xu Xun-feng 2003. “The language learner as language researcher: putting corpus linguistics on the timetable”. System 31(2): 173–186.

Cobb, Tom M. 1997. “Is there any Measurable Learning from Hands-On Concordancing?”. System 25(3): 301–315.

Gabrielatos, Costas. 2004. “Teaching the expression of time: a concise framework”. IATEFL 2003 Brighton Conference sections, ed. by Alan Pulveress, 133–136. Canterbury, Kent: IATEFL.

Gabrielatos, Costas. 2005. “Corpora and language teaching: Just a fling, or wedding bells?”. TESL-EJ, 8(4), A-1, 1–37. http://tesl-ej.org/ej32/a1.html

Hunston, Susan. 2002. Corpora in Applied Linguistics. Cambridge: Cambridge University Press.

Jensen, Eric. 2005. Teaching with the brain in mind. 2nd edition. Virginia: ASCD books.

Johns, Tim 1991. “From printout to handout: Grammar and vocabulary teaching in the context of data-driven learning” ELR Journal Vol.4: Classroom Concordancing, ed. by Tim Johns & Philip King, 27–45. Birmingham: CELS, The University of Birmingham.

Johns, Tim & Philip King, ed. 1991. ELR Journal Vol.4: Classroom Concordancing. Birmingham: CELS, The University of Birmingham.

Kinsella, Kate & Kathy Sherak. 1998. “Designing ESL Classroom Collaboration to Accommodate Diverse Work Styles”. Understanding Learning Styles in the Second Language Classroom, ed. Joy M. Reid, 85–99. Upper Saddle River: Prentice Hall Regents.

Koosha, Mansour & Ali Akbar Jafarpour. 2006. “Data-driven Learning and Teaching Collocation of Prepositions: The case of Iranian EFL Adult Learners”. Asian EFL Journal 8(4): 192–209.

Leiguarda, Ana. 2005. “Motivating the Teenage Brain”. ESL MAGAZINE 44: 13–16.

McCarthy, Michael J. 2004. Touchstone – From Corpus to Course book. Cambridge: Cambridge University Press.

Meunier, Fanny. 2002. “The pedagogical value of native and learner corpora in EFL grammar teaching”. Computer Learner Corpora, Second Language Acquisition and Foreign Language Teaching, ed. by Sylviane Granger, Joseph Hung & Stephanie Petch-Tyson, 119–142. Philadelphia: John Benjamins.

Mukherjee, Joybrato. 2006. ‘Corpus linguistics and language pedagogy: The state of the art – and beyond’. Corpus Technology and Language Pedagogy: New Resources, New Toos, New Methods, ed. by Sabine Braun, Kurt Kohn & Joybrato Mukherjee, 5–24. Frankfurt am Main: Peter Lang.

O'Keeffe, Anne, Michael J. McCarthy & Ronald A. Carter. 2007. From Corpus to Classroom: Language Use and Language Teaching. Cambridge: Cambridge University Press.

Palaiologou, Nektaria. 2007. “School Adjustment Difficulties of Immigrant Children in Greece”. Intercultural Education 18(2): 99–110.

Passapong, Sripicharn. 2002. Evaluating Data-Driven Learning: The use of Classroom Concordancing by Thai Learners of English. Ph.D. dissertation, University of Birmingham.

Prodromou, Luke 2007. “Washback: Negative or Positive?”. Modern English Teacher 16(2): 26–34.

Römer, Ute. 2006. “Pedagogical Applications of Corpora: Some Reflections on the Current Scope and a Wish List for Future Developments”. Zeitschrift für Anglistik und Amerikanistik 54(2): 121–134.

Schmidt, Richard W. 1990. “The Role of Consciousness in Second Language Learning”. Applied Linguistics 11(2): 129–158.

Sifakis, Nicos C. 2009. “Challenges in teaching ELF in the periphery: the Greek context”. ELT Journal 63(3): 230–237.

Sinclair, John & Malcolm Coulthard. 1975. Towards an Analysis of Discourse. Oxford: Oxford University Press.

Sun, Yu-Chih & Li-Yuch Wang. 2003. “Concordancers in the EFL classroom: cognitive approaches and collocation difficulty”. CALL 16(1): 83–94.

Todd, Richard. 2001. “Induction from self-selected concordances and self-correction”. System 29: 91–102.

Vannestål, Maria & Hans Lindquist. 2007. “Learning English grammar with a corpus: Experimenting with concordancing in a university grammar course”. ReCALL 19(3): 329–250.

Vygotsky, L. S. 1978. Mind in society. The development of higher psychological processes. Cambridge, MA: Harvard University Press.

Walsh, Steve. 2006. Investigating classroom discourse. London & New York: Routledge.