Introduction

Concepts and practices in language assessments have undergone significant changes, especially in the area of teaching and learning. Many assessments in the field of English as a Foreign Language (EFL) tend to adopt a proficiency test which usually has no connection to the curriculum (Chapelle & Brindley, 2002). Therefore, the need for an approach that takes into consideration the close relationship between instruction, curriculum, testing, and outcomes is gaining popularity. This is reflected in the emergence of some alternative assessments that are constructed to support the close connection between the teaching and testing process.

In general, an assessment covers several practices to measure students’ skills and functions as an instrument for gathering information (Poehner, 2008). The main goal of assessment is to support the process of teaching and learning in the classroom. Additionally, it is also one of the tools to improve students’ motivation in achieving a learning outcome (Derakhshan & Kordjazi, 2015). Students’ skills are examined to find out their level of ability. This process is significant in determining the strategy to develop their skills as well as overcome the problem they encounter during the learning activity.

Assessments are traditionally divided into two types - summative and formative (Bechman, 1990). Summative assessments are conducted at the end of the teaching and learning process and assess students after they receive instruction. Their purpose is to measure the actual improvement of students’ understanding of the information that has been delivered. This assessment focuses on the product instead of the process. Summative assessments are usually administered as high-stakes tests that function as a decision-maker of students’ outcomes. While, formative assessments are administered before the learning process is completed, summative assessments are conducted during the teaching and learning process to give feedback to students to develop their skills. Unlike summative assessments, this assessment focuses on the process instead of the product.

Although both assessments have different functions, they are considered ineffective in improving students’ skills (Poehner & Lantolf, 2005). Summative assessment which only focuses on the results of learning, ignores the process of learning. On the other hand, formative assessment which focuses on the process of learning ignores the integration of instruction and testing practices during the teaching and learning process (Poehner, 2008). Since the aim of formative assessment is to measure students’ progress, the focus should be on finding out students’ weaknesses that need to be improved. In this case, interaction and mediation should be implemented during the test. The integration of teaching and testing is significant in enhancing students’ skills. Therefore, Dynamic Assessment (DA) has emerged as a kind of assessment that focuses on the process and integrates the learning and testing process.

The current study aimed to apply DA in assisting students of English for Specific Purpose (ESP) in writing descriptive texts. ESP classes focus on teaching language and communication skills specific to certain professions or disciplines of study (Basturkmen, 2010). The main challenge in teaching ESP classes at this public university was that students did not have much exposure to English communication. Writing, which is considered to be one of the most challenging skills for students, needs several stages and practice to master. Thus, DA was expected to be an alternative assessment to effectively help them master writing skills. As a result, the research question that guided the present study was:

How effective is DA in improving ESP students’ skills in writing descriptive texts?

Literature Review

Concept of Dynamic Assessment

DA is defined as a testing procedure that tries to transform performance using mediation so that the test taker’s potential can be identified (Swanson & Lussier, 2005). The main goal of this assessment is to learn the potential of the students (Dörfler et al., 2009). The concept of DA is highly influenced by Vygotsky’s theory (1964) which emphasized the importance of the Zone of Proximal Development (ZPD). This theory refers to the difference between the current developmental state of an individual without guidance and his level of potential development with guidance. DA incorporates the concept of ZPD which means that, during the process of testing, the students are given assistance to measure their achievement.

The framework of DA is conceptualized through the integration of the teaching process and assessment practice (Lantolf & Poehner, 2011; Poehner, 2008). In this case, the teacher acts as a guide who actively assists students’ development in performing the assessment. There is no strict model to implement a DA. Teachers have freedom to interpret the approach in guiding students according to their needs. Hence, it is not a pre-specified method of assessment that can be followed in a set manner, but it is rather a reconceptualization of teaching, testing, and development. In the process of designing such assessments, elements like competence domain, test format, and students’ characteristics need to be included (Dörfler et al, 2009).

The clear difference between dynamic and traditional assessments is that the former focuses on the process of learning and aims to guide students’ current skills to solve problems that appear during the testing process as well as to diagnose their potential (Shrestha & Coffin, 2012). Unlike traditional assessments which emphasize the already acquired skills and a product-oriented approach, DA does not consider an outcome based on product-oriented and preexisting skills (Sternberg & Grigorenko, 2002). If traditional assessment forbids any communication and interaction between examiner and examinee, DA considers learners as active participants, and the examiner as a motivator to provide students with a positive learning experience. Furthermore, DAs give the examiner a role to provide feedback during the testing process while traditional assessments forbid this activity. Examiners’ involvement during the testing process is called mediation.

Mediation in Dynamic Assessment

Mediation is an important feature of DA since it is a significant tool to identify students’ skill development. It aims to highlight the problem that appears to stimulate development (Shakki, 2016). In other words, it requires the communication and interaction between test administrators and test takers using hints, prompts, models, or leading questions to solve problems in instructional activities (Amiri & Saberi, 2016). This concept is quite different from a traditional assessment which views intervention or mediation as cheating or academic misconduct (Hessamy and Ghaderi, 2014).

In general, mediation is divided into two main formats: sandwich and cake (Dörfler et al., 2009; Lantolf & Poehner, 2011; Poehner, 2008). In the sandwich format, the stages involved are pretest-mediation-posttest. The activity starts with a pretest to find out students’ initial skills. Following the pretest activity, the intervention in the form of training is administered. Students are taught the most effective strategies to solve test items. The activity is followed by a posttest in which the parallel items reflect the scope of performance that students possess as the result of the intervention and their individual learning skills. Mediation can be conducted either in a group or individually. In this format, extensive mediation can be provided through separate sessions. However, the drawback of this format is that it is time consuming since the activity of pretest and posttest are conducted on different days. The cake format conducts the intervention during the process of assessment. If students solve the problems incorrectly, a specific kind of mediation is given during the test. Therefore, mediation is given every time a problem occurs. Mediation can be in the form of feedback, hints, or comments. Since there is extensive integration between the testing and mediation, this format takes a shorter amount of time. Thus, the mediation given should be concise and less complex than that of sandwich format. Some hints, feedback, or prompts are designed to be suitable for each item to cover the limited time.

In the process of mediation, two kinds of approaches are available: interventionist and interactionist (Poehner, 2008). The interventionist approach is considered more formal and standardized in terms of its procedure. In this case, pre-fabricated hints are given when students encounter difficulties. The emphasis is on the speed of learning as the aim is to assist students to solve the problem quickly and efficiently. On the other hand, the interactionist approach is focused on the dialogue between the examiner as mediator and students. In this case, the focus of mediation is to assess students’ development and understanding of the skills required in solving the problem.

Some components of DA are needed to support mediation. They are intentionality, reciprocity, and transcendence (Mehrnoosh & Rassaei, 2015). Intentionality refers to a situation in which the mediator directly communicates with the examinee by asking questions or giving clues. Reciprocity refers to the examinee’s responses toward the mediators’ support. This component relates to the examinee’s skills to react or respond to the mediator’s instructions.. Meanwhile, transcendence is the activity of expanding the collaboration between the mediator, the examinee, and the mediation.

Writing in ESP classes

There are several features that differentiate ESP classes from other EFL classes. Firstly, ESP uses more varieties of language (Basturkmen, 2006). This term refers to some registers of language use, such as English in Medicine, Accounting, Banking, etc. The term for specific purposes is divided into two perspectives. The first perspective is called common core plus which refers to specific purpose language as an extension of a basic core of a general language. Thus, students should master English for General Purposes as a basic core before learning ESP. Meanwhile, the second perspective suggests that all languages are considered to have only one variety, and basic core or general-purpose language varieties do not exist. Therefore, ESP classes can start at any level of English skills. Students do not need to learn the basic core of a general language. Learning language directly from a specific language purpose (for instance, English for Marketing, English for hospitality) is more effective since students immediately attain structures, styles, and vocabulary that are related to the usage in their target environment (Basturkmen, 2006).

The second feature is related to needs analysis. The ESP class is established to prepare English learners to use the language in a specific environment such as academic, professional, or workplace. Thus, its key feature is the syllabus which is based on the needs analysis of the learners (Basturkmen, 2006; 2010). ESP language is designed not to obtain general language education, but to assist the path to acquire greater language proficiency in a specific field. Furthermore, most ESP classes have time constraints, so time must be used effectively. The last feature is the type of syllabi. Different from general English classes, the most important question in the ESP class is what language structure and lexicon are to be learned (Basturkmen, 2010). Therefore, a specific syllabus is designed to specify the aspects of the language to be taught.

From the differences highlighted above it is clear that the focus of ESP is to learn a specific skill of language that can be applied in a target environment. In this study, the skill that was identified was writing. In the field of ESP, writing is considered the core activity of an institution (Paltridge & Starfield, 2013). Therefore, the concept of ESP writing is emphasized in the process of guiding students to comprehend a target text. ESP writing encourages the notion that not all writing is the same, and that language is used for certain specific purposes. Hence, the concept of purpose becomes the key feature of ESP writing. ESP writing covers a wide range of genres, meanings, and characteristics which are appropriate to each context. One specific feature that can be used to assist students in understanding the specific writing is through genre texts. The exploration of genre text guides students to find out relationships between audience, purpose, and organization of a text (Hyland, 2004; Paltridge, 2001). One specific kind of genre that is used in this study is descriptive text which refers to a kind of text that presents facts and instructions related to a product, like a brochure or instruction manual (Johns, 2011).

Previous Studies on Dynamic Assessment in Writing

Several studies have been conducted investigating the effectiveness of DA in the teaching and learning process. Shresta and Coffin (2012) investigated the influence of DA in academic writing development. Their study found that DA successfully identified and responded to students’ needs individually. Alemi (2015) found that DA is effective in building students’ awareness of writing assessment criteria. Writing assessment was important to help students understand the organization of texts. Composing a text was difficult for students who did not understand the organization of a text. Therefore, the instructor’s guidance through several interactions was helpful to overcome this gap. In the process of mediation, students were able to evaluate their own writing skills. This was conducted by instructing students to assess their own skills through self-evaluation. The activity was proven to assist low-level students achieve considerably more accuracy in their self-evaluation. In short, the activity helped the teacher and students to match their evaluation, so further assistance could be given to develop students’ skills.

Mauludin (2018) found that the application of DA helped the teacher to better understand the level of writing ability of students in an English Diploma Program (three-year university program). In addition, the mediation given reduced students’ anxiety so they felt more motivated in completing the writing task. Amiri and Saberi (2016) and Sadek (2015)also found that DAs were significantly more effective in composing a text. The process of mediation assisted the students to evaluate their own weaknesses which could be used during the guidance process. Miao and Lv (2013) also suggested that the mediation process between the teachers and students improved students’ potential in writing. By understanding students’ potential, instruction could be provided to overcome learning obstacles. Furthermore, Xiaoxiao and Yan (2010) mentioned that DA provided a dialogic way that could be an effective support to enhance students’ interest in writing.

The reviewed studies support the effectiveness of DAs in developing students’ language skills. However, there seems to be very limited research in DAs in ESP contexts, prompting the need for this study that focused on the role of DA in developing students’ skills in composing descriptive texts in an ESP class.

Method

The method used in the study was a mixed-method explanatory sequential design. The quantitative approach is first conducted and that is followed by a qualitative approach to describe the findings (Creswell, 2003). The quantitative approach used was quasi-experimental in design without random sampling. Seventy students participated in this study. They were divided into two classes. They were enrolled in the three-year diploma program of Librarian Technician at a public university in Indonesia and were in their second semester and learning level 2 of an ESP class. Level 2 was the continuation of the previous class. In the previous level of study, they had learned vocabulary used in the field of library science as well as basic writing skills, such as simple present tense, past tense, present continuous, etc. The class was conducted once a week for one and a half hour. In this level, they learned how to write several genres including descriptive, narrative, and expository texts. They were in the beginner to intermediate levels of English proficiency and ranged in age from 18 to 20 years old. The study was conducted for four weeks in five stages: pretest, mediation (group work), mediation (individual work), posttest, and data analysis. The intervention used in the study was cake format for both group work and individual work.

Stage 1

Since there were two equivalent groups taught in the semester, one class was assigned as a control group and the other as an experimental group. Each class consisted of 35 students. In the experimental group, the DA approach used was the sandwich format in which the mediation or intervention was conducted between pre-test and post-test. The control group received a traditional method of teaching without mediation activity. In the traditional method, the students received an explanation from the teacher and completed the tasks without the teachers’ intervention. Both the control and experimental group were given the pretest. They were given a series of pictures of a library and assigned to write a paragraph of a descriptive text in 200-250 words. This stage was conducted in the first week.

Stage 2

In the second week, both groups were given an explanation of a descriptive text and a task to compose a descriptive text in a group. In this stage, the first learning activity was building background knowledge. The students received an explanation of the definition and function of a descriptive text to build their background knowledge of the particular topic. Then, the teacher provided a sample of a descriptive text, and explained the skills of writing that students should master namely, content, organization, vocabulary, language use, and mechanics. In the modeling activity, the teacher demonstrated how to write a descriptive text. After modeling, the students were given a task to write a descriptive text from a series of library pictures. The series of pictures was different from that of the pretest. This task was considered as the first assessment after the teaching process. In this task, both groups composed a text in a group of five. The students were allowed to discuss and work together within the group. In the experimental group, DA was conducted. The teacher went around within a group and observed their work. When the teacher found the students made incorrect responses, the teacher directly offered corrections by providing hints, explanations, prompts, suggestions, or leading questions. The incorrect responses included the content (whether the description fit the pictures provided), organization, vocabulary, language use (including grammar), and mechanics (punctuation, spelling, etc.). The mediation was given within a group, so if there was a student who made a mistake in a group, the teacher gave hints for all the group members. Any student from the group was allowed to respond to mediation to find out the correct answer. Group work was conducted since time was limited. Providing peer guidance was supposed to facilitate students to finish their work faster. In the control group, no mediation was provided by the teacher. The students discussed the assignment by themselves.

Table 1: Learning activities

Sample of mediation 1

Below is the example of mediation conducted in the study. The teacher provided hints and leading questions for the students. Most of the conversations were conducted in Bahasa Indonesia, the native language, since most students were not fluent in English. These samples were translated by the first author.

T = Teacher

Ss = Students

S1 = Student 1

S2 = Student 2, etc.

Group 1.

(One student wrote a draft. The others tried to suggest some ideas. The student wrote “Public library in my city big and beautiful”)

T = Let me see, are you sure the first sentence is correct?

Ss = (no response)

T= Do you remember? When we write a sentence, we have to make sure that the sentence has a subject and a verb. What is the subject of this sentence?

S1= Public Library.

T = Correct. What is the verb?

S2 = is…was..?

T = Yes, you use “to be”. I don’t see it here. Is it ‘is’ or ‘was’? This is a descriptive text. What kind of tense should you use?

S1 = Present tense.

S2, S3, S4 = is….

T = Excellent.

Students received teacher guidance (or mediation) until they were able to correct all language errors in their writing.

Stage 3

The second assessment was given in both groups in the third week. In this stage, the students were asked to finish the task individually to find out their understanding of the topic. This was also conducted to find out each student’s weakness, so the teacher could help the student to improve it effectively. Similar to the second stage in the experimental group, the teacher walked around the class and observed each student. As the teacher found a student making mistakes, the teacher provided feedback until the student found the correct response. Similarly, hints, prompts, suggestions, and leading questions were provided as mediation. The control group did not receive any mediation; thus, they completed the task independently.

Sample of mediation 2

Below is the example of how the teacher provided students with suggestions to improve their writing.

T = What do you mean by the park?

S = The place to park the car.

T = Do you mean parking lot?

S = Oh … yes... [He crossed out the word ‘park’ and wrote ‘parking lot’. Then, he moved on observing the picture of the reading room of the library. In this situation, the teacher found out that the student did not describe ‘parking lot’ in detail, so the teacher interrupted him]

T = Let me remind you again. Descriptive text provides a detailed description of an object. I want you to make sure you have described an object in detail first; then, you can describe another object.

S = So, we describe the building first in detail, then go on to the other, sir? [He referred to the first picture. The picture can be seen in Task 2 in the Appendix]

T = Yes.

Students received teacher guidance (or mediation) until they were able to correct all language errors in their writing.

Stage 4

Posttests were conducted in both groups in the fourth week. The task was similar to the pretest, but a different series of pictures was provided. The purpose was to find out how much improvement the students had made during the two-week activities. For the experimental group, an open-ended questionnaire was distributed to obtain their views about the implementation of DA.

Stage 5

In this stage, the data obtained were analyzed. The posttest and pretest were analyzed using SPSS. A t-test was conducted to measure the statistical difference between the control group and the experimental group scores. The students’ writing was evaluated using the ESL Composition Profile (Jacobs et al., 1981) with two independent raters to avoid any subjectivity. The questionnaire was used to find out students’ opinions on improving the implementation of DA.

Results

In the ESP writing class, the experimental group received the DA approach. Before the treatment, the pretest was conducted by assigning them to compose a descriptive text. Their writings were then evaluated, and their scores were tabulated using paired sample statistics. The tabulated data are shown in Table 2. After the treatment, the posttest was conducted. Table 3 shows the results of the tabulated data.

Table 2: Paired sample statistics of pretest

Table 3: Paired sample statistics of posttest.

Table 4: T-test score

A T-test analysis of the scores for both the control group (p = .001) and the experimental group (p < .001) shows that students’ writing skills improved significantly. As the tabulated data shows, although both groups showed an improvement in mean scores, it was clear that the experimental group’s improvement was more significant. This finding supports the importance of mediation in improving students’ writing skills. The students who experienced the process of learning through intervention showed a significant improvement when compared to those who had limited intervention.

Following the posttest, the open-ended questionnaire was given to the experimental group. There were two main questions. The first question elicited their perceptions about the activity (whether they like it or not and why), and the second asked for suggestions to improve the activity. The analysis of students’ responses showed that out of 35 students, 30 of them had positive opinions about the activity, while five of them seemed unsatisfied. In general, students who had positive opinions mentioned that the activity helped them to find out their weakness, assisted them to find accurate vocabulary, and made them more relaxed in composing the text. The students also suggested implementing this method in other classroom activities.

Discussion

DA has a positive influence on students’ writing skills since it emphasizes the process of mediation which is an opportunity for students to interact and discuss a problem in the process of writing (Amiri & Saberi, 2016; Hessamy & Ghaderi, 2014; Shrestha & Coffin, 2012). The data of the current study showed that the experimental group performed better than the control group. This means that the interaction provided in the mediation assisted the students in finding accurate information to solve the problem when writing the descriptive text. The most challenging part of writing in a foreign language is to adjust the context of the target language. Since students have limited experience with the target language, the teacher should provide in-depth information about it. In this case, unlike English, Bahasa Indonesia does not emphasize the importance of ‘subject-verb agreement’. Thus, as mentioned in the sample of mediation 1 in the method section, the students had difficulty providing a ‘verb’ that agreed to the ‘subject’, such as providing ‘to be’ in sentences. The most effective way to deliver the material was by having direct interaction with the students. This way, the teacher could measure each student’s understanding.

Furthermore, it is important to create an activity that requires a dynamic evaluation process that assists students with suitable and interactive engagement. Some students who had no problem in grammatical form still found difficulties in describing an object using appropriate vocabulary. The sample of mediation 2 in the method section showed a student’s difficulty in describing “a parking lot” as he did not know the words in English. Furthermore, the student did not try to describe the parking lot in detail even though the picture showed a specific condition of the parking lot. In this case, the teacher provided suggestions and leading questions to help them with the task. Thus, DA provides an interactive learning environment that highly contributes to the development of writing skills (Davoudi & Ataie-Tabar, 2015).

In an ESP class, students may have limited time to develop English language skills because they may not receive enough explicit English vocabulary instruction. In ESP contexts, language plays a major role since it specifies the use of a word in a specific context. In this study, the students needed to use specific terms in the field of library science. The implementation of DA supported this area since it helped students identify and respond to the elements that they needed the most (Shrestha & Coffin, 2012). In this case, the students needed to use the proper vocabulary to describe a certain object.

Furthermore, the two types of DA methods that were applied also contributed significantly. In the second stage, the DA was conducted in a group. This offered more opportunities for students to have additional support from their peers (Davin & Donato, 2013). In a learning process, the peer has an important role as the source of information of the target language (Oliver, 2013). In a small group, peer assistance functions as behavior that supports the goal of achieving higher levels. In this study, the assistance by the teacher did not only focus on a certain individuals, but it spread across the group which maximized the function. In other words, besides teachers’ mediation, the support of peer mediation also improved students’ development skills in composing a descriptive text.

Aligned with the statistical data, students’ opinions also supported the effectiveness of DA. Students mentioned that DA helped them find their weaknesses. Most students did not realize that they made a mistake. During the mediation, the teacher gave them instant feedback. Some students’ mistakes were mostly about the organization of the text. In a descriptive text, the students needed to write the general description first before moving on to specific details. However, most students just wrote whatever they saw from the picture which made the text incoherent. This immediate feedback helped them realize their mistakes. This suggests that this activity improved students’ awareness of their current ability (Alemi, 2015).

Students also stated that the activity helped them improve their lack of vocabulary. In the writing process, students were often stuck in their writing, as they did not find the specific vocabulary to describe an object. Specific terms used in the field were needed as the description was about the library. Some students forgot the correct terms and some of them did not know the vocabulary. In this case, the mediation provided in DA offered an opportunity to track students’ development by providing information about their current writing skills, so the required amount and quality of support could be determined (Shresta and Coffin, 2012). In this case, the students’ level of vocabulary knowledge varied. Thus, appropriate types and amount of mediation could be given to students with different levels. By doing this, students with different levels of vocabulary skills could achieve a similar goal.

Finally, students reported that the DA activity was helpful, and they felt confident they could complete the task with the teacher’s guidance. This finding suggests that DAs allow the teacher to create a low-stress classroom environment that reduces student anxiety and fear of failure and increases motivation (Mehrnoosh & Rassaei, 2015). A comparable finding was reported by Mauludin (2018), who conducted a similar study for English Diploma program students. The teacher’s intervention of giving hints or leading questions reduced students’ stress so they felt less anxious and more comfortable in the writing process. By providing a comfortable environment, students could increase their motivation in composing a text. In short, the application of DA has proven to be effective in assisting students of this ESP class in developing their writing skills of descriptive texts.

Despite its success, a few students had unfavorable opinions of the activity. In general, they mentioned the unfairness of the teacher’s mediation as some students received more guidance than others did. They suggested that all students should receive equal time of mediation and assistance during the activity. This was considered a critical issue in the implementation of DAs. As there was limited time during the assessment process, the teacher found it difficult to give a fair amount of time for each student. This limited a majority of the students’ engagement with the task, as not all of them received the benefit of teacher assistance (Poehner, 2009). Hence, the ones that benefited the most from this activity were the ones who received more mediation from the teacher.

Conclusion

Based on the analysis of the statistical data and students’ opinions, DA had a positive influence on students’ development of writing skills, especially in enhancing their grammar and vocabulary skills. The T-test comparison between the control group and the experimental group showed that the DA effectively improved those skills. The mediation process of DA played a significant role as it assisted students to solve problems throughout the testing activity. The immediate feedback given during the assessment process helped students overcome the difficulty more effectively. Students’ opinions suggested that this activity helped them assess their own abilities, especially the use of grammar. It also helped them find specific words to describe specific objects. Moreover, this activity also provided a comfortable environment to compose texts.

Several limitations to the current study and suggestions for future studies deserve to be pointed out. Firstly, the data for statistical analyses were limited since the study included only a pretest and posttest. Students’ achievement in the classroom is influenced by many factors such as motivation, psychological needs, peer interaction, etc. Thus, other variables should be included in further research.

Secondly, even though the current study has demonstrated the effectiveness of DA, it only focused on one topic of writing and was conducted for a short duration of four weeks. Since students’ performance is unpredictable, it is important to provide longer time of DA intervention in different kinds of writing topics.

Thirdly, this study was conducted in an ESP class in an Indonesian University with Indonesian students. Findings may vary in other cultural settings. Thus, it is essential to replicate this study to determine the relevance of DA in different ESP educational settings and cultural backgrounds, as well as gain an understanding of how this approach influences students’ performance in other English skills like speaking and listening, for example.

Finally, the findings of this study are limited to the interpretation of quantitative experimental data. Further research adopting qualitative methods may provide better insights into students’ perceptions on the impact of DAs.