Assessing academic reading in students entering Higher Education

Main Article Content

Marcela Cabrera-Pommiez
Francia Lara-Inostroza
Juana Puga-Larraín

Abstract

This study presents an original instrument for evaluating reading comprehension, used in the Diagnostic Test of Communication Skills that was applied to students who entered a Chilean university, in the period 2017-2020, in order to determine their level of reading comprehension, applied to academic texts, at the time of beginning their higher education. This diagnostic test was developed according to a model that distinguishes three major reading skills, broken down into more specific learning outcomes, ranging from locating literal information to evaluating form and textual content. The results show that an important group of the evaluated youth does not have a satisfactory level of academic reading and presents difficulties in answering items that evaluated higher level implicit reading tasks, such as paragraph relationships, inferences, and interpretation of communicative purposes. It is concluded that measures are needed to encourage academic reading in students entering higher education, especially the extraction of non-explicit information through comprehensive reading, since this skill is directly linked to the learning and the acquisition of disciplinary discursive traditions.

Article Details

How to Cite
Cabrera-Pommiez, M., Lara-Inostroza, F., & Puga-Larraín, J. (2021). Assessing academic reading in students entering Higher Education. Ocnos. Journal of reading research, 20(3). https://doi.org/10.18239/ocnos_2021.20.3.2614
Section
Artículos
Assessing academic reading in students entering Higher Education

Marcela Cabrera-Pommiez, Francia Lara-Inostroza, Juana Puga-Larraín

Assessing academic reading in students entering Higher Education

Ocnos, vol. 20, no. 3, 2021

Universidad de Castilla-La Mancha

Evaluación de la lectura académica en estudiantes que ingresan a la Educación Superior

Marcela Cabrera-Pommiez *

Universidad de Las Américas, Chile


Francia Lara-Inostroza

Universidad de Las Américas, Chile


Juana Puga-Larraín

Universidad Diego Portales, Chile


Received: 30 November 2020

Accepted: 11 April 2021

Abstract: This study presents an original instrument for evaluating reading comprehension, used in the Diagnostic Test of Communication Skills that was applied to students who entered a Chilean university, in the period 2017-2020, in order to determine their level of reading comprehension, applied to academic texts, at the time of beginning their higher education. This diagnostic test was developed according to a model that distinguishes three major reading skills, broken down into more specific learning outcomes, ranging from locating literal information to evaluating form and textual content. The results show that an important group of the evaluated youth does not have a satisfactory level of academic reading and presents difficulties in answering items that evaluated higher level implicit reading tasks, such as paragraph relationships, inferences, and interpretation of communicative purposes. It is concluded that measures are needed to encourage academic reading in students entering higher education, especially the extraction of non-explicit information through comprehensive reading, since this skill is directly linked to the learning and the acquisition of disciplinary discursive traditions

Keywords: Academic literacy; reading comprehension; academic reading; communication skills; graduate students; Higher Education.

Resumen: Se presenta un instrumento original de evaluación de la comprensión lectora, utilizado en la Prueba de Diagnóstico de Habilidades Comunicativas que se aplicó a los estudiantes que ingresaron a una universidad chilena, en el periodo 2017-2020, con el fin de determinar su nivel de comprensión lectora, aplicada a textos académicos, al momento de comenzar los estudios superiores. Esta prueba de diagnóstico se elaboró de acuerdo con un modelo que distingue tres grandes habilidades de lectura, desglosadas en resultados de aprendizaje más específicos, que abarcan desde la localización de información literal hasta evaluación de forma y contenido textual. Los resultados muestran que un grupo importante de los jóvenes evaluados no tiene un nivel satisfactorio de lectura académica y presenta dificultades para responder ítems que evaluaban tareas de lectura implícita de nivel superior, como relaciones entre párrafos, inferencias e interpretación de propósitos comunicativos. Se concluye que son necesarias medidas que fomenten la lectura académica de los estudiantes que ingresan a la educación superior, especialmente la extracción de información no explícita a través de la lectura comprensiva, pues esta habilidad está directamente vinculada al aprendizaje y a la adquisición de las tradiciones discursivas disciplinares.

Palabras clave: Alfabetización académica; comprensión lectora; lectura académica; habilidades comunicativas; estudiantes universitarios; Educación Superior.

Introduction

Entering higher education presents, as the main challenge, the study of subjects and content specific to the chosen degree, for which students must mobilise a series of learning strategies, as they face greater cognitive demand than in secondary education. However, this challenge is not the only one from an academic point of view: an important secondary demand is constituted by the need to learn to read, write and express oneself orally in an academic context, within a specific disciplinary field, given that the student becomes part of a specialised discourse community (Swales, 1990; Hyland, 2009), which shares a defined communicative style and has traditional ways of communicating (Braslavsky, 2005), more complex discursively and different from the colloquial register used daily.

For much of the 19th and 20th centuries, it was assumed that this enculturation process developed spontaneously in the learner, who, by his or her means, had to learn to process the information in written texts and to incorporate into his or her expression an appropriate register that would enable him or her to perform well academically. At the end of the 20th century, this approach changed and was replaced by the notion of “academic literacy” (Estienne & Carlino, 2004; Lea & Street, 1998; Carlino, 2013), which implies the approach of a conscious and intentional teaching-learning process of the discursive traditions that students must know and manage for successful development in the academic and professional world.

Considering the above, and following Carlino's (2005) approach, academic literacy is understood as "the set of notions and strategies necessary to participate in the discursive culture of the disciplines, as well as in the activities of production and analysis of texts required for learning at university" (p. 410). In other words, it is the process by which the communicative codes and linguistic-discursive uses of each discipline are transmitted to students so that they become literate, that is to say, competent within the academic community linked to their discipline (Castelló, 2014).

The concept of academic literacy addressed in this article is defined based on five criteria: a) it is conceived as a set of social practices that takes place in literacy events, which are strongly influenced by the communicative context (those who communicate, how and what for). This position takes up the approaches of the New Literacy Studies (NEL), some of whose key works are from the 1990s (Street, 1984; Gee, 1991; Lea & Street, 1998; Barton & Hamilton, 1998); b) it is distinguished from a very close term -literacy-, as this "implies a way of using reading and writing in the framework of a specific social purpose" (Zavala, 2008, p. 71), thus, academic literacy is an acquired ability, which enables dialogue and interaction with experts in a discipline (Fang, 2012) and does not refer to the process of developing it in educational contexts; c) it implies the implementation of educational practices, executed by higher education institutions, as literacy is necessarily a formal instruction that "can be defined as the responsibility of teachers who are experts in a discipline to share with their students the ways of reading, writing, speaking, listening, researching and thinking of the members of that disciplinary community" (Montes & López, 2017, p. 166); d) it is focused on the academic literacy of the students, and it is not the process of developing it in educational contexts (Montes & López, 2017, p. 166); d) it focuses on a type of discourse - academic discourse - which is materialised both orally and in writing, and has distinctive features, such as complex syntactic structures, connectors infrequent in colloquial speech, discursive genres that privilege the transmission and reworking of knowledge, marks of epistemic positioning of the sender and lexical density (Uccelli & Meneses, 2015); and, finally, e) places the written text in a prominent place, particularly those that have an expository and/or argumentative communicative purpose; from which derives the strong epistemic value of the processes of comprehension and production of written texts.

The development of numerous studies on the subject shows the growing interest in seeking and sharing experiences on what deficiencies students show in their learning of academic reading and writing, what text sequences are managed in a better or worse manner and what teaching-learning strategies are useful, among other topics. In this panorama, writing seems to be a particularly attractive object for specialised publications, to the detriment of academic reading, which seems to be contained in the process of written production, like a stage that provides information and thus makes it possible to produce a good text.

In this study, research is carried out on academic reading, as it is considered to play an important role in the acquisition of knowledge and discursive models, given that it allows students to incorporate the way of communicating content and expressing themselves in academic language, typical of the university world. The proper development of academic reading has as its main effect an effective reading comprehension capacity, which must be carried out in a directed and permanent way, and not only as a function of the need to write. This statement becomes even more relevant when we see that many research studies carried out to assess reading comprehension skills in university students highlight the shortcomings detected, for example, in terms of the lack of application of comprehension strategies (Neira et al., 2015), lack of knowledge of the textual structure and discursive conventions of academic genres (Sologuren & Castillo, 2020) or difficulties in generating inferences from the textual surface (Guerra & Guevara, 2017).

The purpose of this article is to present an investigation into the academic reading skills of young people who entered a higher education institution -a private Chilean university-, an investigation that included the design, development and application of a diagnostic assessment model of academic language, considering a target population of university students belonging to four different cohorts and who, according to their socio-demographic characteristics, present difficulties for the development of academic and discursive competences typical of the tertiary level, such as working and studying, entering university several years after graduating from secondary education, as well as being older than the average age of a first-year student (Bordón et al., 2015). This background accentuates the importance of reading in academia, since, as Pérez and Natale (2016) point out, the development of academic literacy is a factor that collaborates with social inclusion in higher education, given that students from more disadvantaged classes often arrive less prepared than those from more affluent classes.

The guiding questions of the study are: does the assessment model fulfil the purpose of diagnosing initial academic reading skills? Do the young people assessed have a satisfactory level of comprehension of academic texts? Which reading levels are better developed? Which specific reading tasks are lowered and therefore require special instruction?

Reading: a cognitive and social process leading to learning

Reading is a particularly important skill in today's world. It enables citizens to function in society, to access content conveyed in written texts, for leisure and learning. Successful reading involves several factors: motivation (Muñoz et al., 2016), command of the lexicon used in the text (Perfetti & Hart, 2002), prior knowledge of the conceptual world represented (Kintsch, 1998), of the discursive genre (Bhatia, 2004) and textual structures (Pérez et al., 2016). All these factors are gradually incorporated in the reading subject, initially, thanks to the formal teaching provided at school, although their development continues throughout adulthood.

Academic reading is, therefore, one type of reading within a range of possibilities. Its main characteristic is that it is linked to academia, understood as the spaces where knowledge is disseminated, discussed and expanded, and where there are traditions, according to each discipline, which involve ways of discursively executing rhetorical operations and the exposition of information. From the students' point of view, entering higher education involves progressively assimilating and becoming assimilated to the discursive culture of the disciplines taught in their degree, so that they can understand (and then incorporate into their written production) rhetorical moves and textual patterns typical of scientific writing (Sabaj, 2012; Venegas et al., 2016), such as: alluding to previous research (by direct or indirect quotation), defining important concepts, including tables and/or graphs to show research results and explaining a theoretical and/or methodological gap.

Faced with this challenge of complex reading, the development of reading strategies becomes necessary to achieve meaningful learning (Escoriza, 2003; McNamara 2010), given that there is a strong implication between academic reading and learning, not only because reading is learning, but also because the discursive conventions of the discipline being studied are acquired, which should subsequently be reflected in a discursive and pragmatically appropriate expression.

From a cognitive point of view, Kintsch's (1998) model of construction and integration has been taken as the theoretical model of reference in this research, due to its acceptance in reading comprehension studies and the soundness of its proposal. According to this model, which takes up what was previously proposed by Van Dijk and Kintsch (1983), three levels of representation, elaborated by the reading subject, can be distinguished, namely: i) the surface code consists of the identification of words, phrases and sentences; ii) the text base, which is the semantic micro and macrostructure of the text, propositions obtained by abstraction from the textual surface; and, iii) the situation model, which is a mental representation of the text content, involving an updating of this content to the real world (Tijero, 2009) and which is obtained by integrating the text information with the reader's prior knowledge and reading objectives.

Assessment of academic reading

When considering the importance of academic reading in the field of university education, the need for adequate teaching arises. In this sense, a diagnostic assessment instrument is a good contribution, since it provides educational actors with a look at the initial state of students' reading skills. According to several studies on the subject (Solé, 2012; Neira et al., 2015; Coronado, 2017; Felipe & Villanueva, 2018; Cabrera & Caruman, 2020), there are deficiencies in the reading comprehension of students graduating from secondary education and those entering tertiary education, especially in the reading of expository and argumentative texts (Meyer & Ray, 2011), taking into acocount that in these types of texts the referential function prevails and they differ from the narration, which is a familiar and more easily comprehensible sequence. These reading deficiencies, if not improved, will have a negative impact on student learning, as academic success will be achieved to the extent that each subject can make up for them.

Concerning the diagnostic evaluation instrument, a specific objective of the research was to create a test, which would allow the student's reality to be analysed in a situated manner. The development of the test took into account the following background:

  • Given that the construct (comprehensive reading skills) is well defined and has been the subject of previous assessments, a review was made of the assessment frameworks of other instruments already validated, such as the PISA test (OECD, 2017); the ENLACE Media Superior test (Reyes et al., 2009) and the PIRLS test (Mullis &Martin, 2013).

  • Elements of the Chilean school curriculum for the subject of Language and Literature (Mineduc, 2009, 2015) were also considered, namely: i) the distinction between literary and non-literary texts and the identification of the communicative purposes of each type of text; ii) the presence of other semiotic codes besides the linguistic one, which transforms them into multimodal texts (O'Halloran, 2012); and iii) the learning objectives to be achieved through the Language and Literature curriculum.

  • We worked with fragments of real texts, drawn from typically academic discursive genres, such as academic papers, educational websites, textbooks and study texts.

  • The requirement for the stimulus texts selected was that they should be suitable for secondary school leavers, meaning that they should not involve too much-specialised knowledge.

The instructions instructed the student first to read these texts comprehensively and then to answer the items based on the literal content and the information that can be extracted from the textual surface. That is to say., the comprehension tasks involved obtaining information and performing implicit reading tasks such as making inferences, interpretations and evaluations of the content read (content and form).

To better specify the reading tasks requested of the students, the specifications defined for the reading comprehension axis are presented in table 1.

Table 1
Reading comprehension axis specifications

Reading comprehension axis specifications


Source: Cabrera, M. (2017). Diagnostic evaluation of Communication Skills year 2017.Universidad de las Americas. Work document.

Methods

Design

The research was non-experimental, ex post facto, descriptive and cross-sectional. The results obtained in four processes of application of the diagnostic instrument designed to evaluate the academic reading ability of students entering higher education at a Chilean university were analysed. In this way, it was possible to determine the level of development of academic reading. The variable under study is constituted by the communicative ability to understand written academic texts in students entering university.

Participants

Correspond to new students who took the Communication Skills Test -part of the Diagnostic Entrance Assessment test battery- during the years 2017, 2018, 2019 and 2020; reaching a figure of 21,465 students (see table 2).

Table 2
Coverage of the Communication Skills Test

Coverage of the Communication Skills Test


Table 3 presents a characterisation of the sample, based on the following socio-demographic indicators: number of women, average age, students who work and those who are the first generation of their family to access higher education.

Table 3
Sample distribution

Sample distribution


Variables and instruments

This is a written test, designed to measure students' initial academic literacy level. It is taken online from any computer at the beginning of the first semester. The time allotted for the test is ninety minutes.

It consists of forty multiple-choice items, with five options, with one correct answer. It is divided into two thematic areas: reading comprehension (twenty-five items) and written production (fifteen items). The latter is less representative because it consists of an indirect measurement of the process of written production: students do not write texts but must choose one option per item, which shows an appropriate selection of linguistic elements, intending to produce a text.

This article deals exclusively with the reading comprehension axis.

The instrument was designed taking into account two references: firstly, other assessment models of similar characteristics; and secondly, the theoretical framework developed previously, where the purpose of the test was defined, as well as the characteristics of the stimulus texts and the skills to be measured by the items; information that was systematised through a table of specifications. In the first version of the instrument, developed in 2016, experts from inside made a judgement and outside the university and a pilot test was applied. With the data from this first application, a second version was generated, in which items with poor statistics (difficulty, omission and discrimination) were eliminated and others were improved. This reformulated instrument was again reviewed by experts and implemented the following year (2017). The described sequence (implementation, statistician analysis, modification/deletion of items from the previous implementation and expert review of the modified version) was maintained during the following years, to ensure the validity and reliability of the test.

Regarding the achievement levels, two levels were established, according to the percentage of correct answers that each student could obtain, as presented in table 4:

Table 4
Achievement levels based on the number of correct answers obtained

Achievement levels based on the number of correct answers obtained


Procedure

Data collection was carried out using the Moodle platform and Power BI software, as the test was administered online. Students were invited by the Dirección General de Asuntos Académicos (DGAA) to take the Evaluación Diagnósticas de Ingreso, as part of their process of incorporation to the university. Once the application stage was completed, the submission platform allowed the delivery of the results both individually and grouped by degree, faculty and campus, as well as presenting socio-demographic indicators.

Data analysis

The items in the instrument are closed-response items, so the platform where the instrument was administered counted the number of correct, incorrect and omitted answers per student. These primary data allowed us to obtain statistical data by cohort such as Cronbach's alpha of the instrument, percentage of achievement by axis, skill and learning outcome, as well as the difficulty and discrimination index of each item (see table 5). Difficulty corresponds to the number of subjects who get the key right and was broken down into the following ranges: an index >=0.70 indicates low difficulty (easy item); 0.31 to 0.69, medium difficulty; and <0.30, high difficulty. Regarding discrimination, the ranges are>0.40, excellent quality of discrimination; 0.31-0.39, high quality; 0.20-0.30, medium; 0.00-0.19, insufficient; and, < -0.01, lousy. The last two categories imply discarding the item, as it does not adequately fulfil the purpose of differentiating good from bad students, according to their responses throughout the instrument (Díaz & Leyva, 2013).

Table 5
Stratigraphy of the Communication Skills Test

Stratigraphy of the Communication Skills Test


Results

The instrument designed to assess academic reading ability, and applied over four years to students entering the university, reached a total of 21,465 students, which shows that the coverage of the application was wide, as more than 80% of new students took it (88.1%, on average) since the test is applied on a census basis.

According to the socio-demographic characteristics presented (table 3), it can be seen that the sample is made up of a majority of women, the average age of entry to university is 24 years old, there is an important group of students who also work, and more than half come from families where the parents have not studied at university.

Concerning the statistics (table 5), there is a tendency over the years for the items to be easier for the population evaluated. In the case of items of medium difficulty, there is a fluctuation, with the lowest value in 2020. For difficult items, the tendency is not to exceed 10%. In the case of discrimination, good data is observed, as the ideal and average classification in all years remains above 70%. This indicates that the test items allow differentiating students who know from those who do not.

Regarding reliability, analysed through Cronbach's Alpha coefficient, the figures obtained show that the instrument is reliable in its evaluative purpose: 0.88 (2017), 0.84 (2018), 0.89 (2019) and 0.85 (2020).

Below are graphs showing the results obtained by each cohort, broken down by each learning outcome (LO) assessed.


Percentage of students at the achieved level
Figure 1
Percentage of students at the achieved level


Regarding the overall academic reading level, figure 1 shows that, although there is some disparity in the four groups assessed, a moderate proficiency can be seen, as in two cohorts (2017 and 2019) about half of the first-year students did not reach the achieved level, which means that their entry reading skills were deficient to comprehensively address the academic texts presented. The last group assessed (2020 cohort) shows a higher percentage of students (72.9%) who do possess basic academic reading skills. A chronological look shows an improvement over the period considered, even though an important group of young people is detected who do not have a satisfactory level of academic reading.

The design of the instrument contemplated the assessment of reading comprehension through the distribution of reading tasks in three different levels, according to Kintsch's (1998) model of mental representation. Thus, a communicative skill I, defined as "Extracting literal information from what has been read" is translated into LO 1: "Obtaining information at the literal level". The corresponding items assessed comprehension of information explicitly stated in the text, such as cause-effect relationships (e.g., According to the text, which element constitutes an effect of the application of pesticides on arable soils?), characterisations of places, objects and/or actors (Which of the following is a characteristic of societies with participatory democracies?), or events and associated dates (In which period of sleep is there an increase in eye activity?). Figure 2 shows the achievement of this LO.


Students at achieved level, learning outcome 1
Figure 2
Students at achieved level, learning outcome 1


It is observed that the students have an acceptable level of locating literal information, as the total of the sample is above 50% achievement, which means that at least half, and in some cases a little over 60%, of those assessed, answered the associated items correctly, thus demonstrating that they can account for the information expressed in the textual surface.

Communicative Skill II involves the first degree of abstraction, as the learning outcomes assessed pointed to the reader's ability to derive different implicit content: LO 2 was defined as: "Extracting the main idea from parts of the text or the whole text" and corresponds to statements such as The topic of the text/paragraph two is. LO 3 consists of "Relating information from different paragraphs of the same text", considering that such a relationship is not explicit and must be detected by the reader, as in the following statement: About the information presented in paragraph three, what function does paragraph five fulfil?. Finally, LO 4: "Draw valid inferences from literal information in a text", which asked students to infer information globally (considering the whole text) or locally (a part of the text, e.g.: From the characteristics of the national population in the colonial period, what can be inferred about the social classes?)


Students in achieved level, learning outcomes 2, 3 and 4
Figure 3
Students in achieved level, learning outcomes 2, 3 and 4


It can be observed that in number 4 less than half of the sample reaches achievement and two cohorts have only 30% of their students at the achieved level. This shows that the ability to infer information is not well established, which is worrying given that making inferences is a trait of good readers. What is more, establishing a relationship between paragraphs (LO 3) does not show a good stage of development since here we detect the lowest percentage of achievement of all (20% in the 2017 cohort). In the rest of the groups, approximately half achieve a good level of correct answers. Meanwhile, the ability to synthesise information (LO 2) shows higher results than the two previous ones, from which it can be deduced that the assessed population has better developed the ability to extract central ideas and/or themes from parts of the text and the full text.

Figure 4 shows the behaviour of the two learning outcomes of communicative skill III, which show a critical reading ability, i.e. interpretative and evaluative.


Students at achieved level, learning outcomes 5 and 6
Figure 4
Students at achieved level, learning outcomes 5 and 6


LO 5 shows a fairly good command of the ability to interpret the sender's communicative purposes (primary and secondary), which means that students were able to detect textual marks indicating the orientation given by the sender to both textual resources and text content. This can be observed when trying to persuade about the validity of a point of view (What is the sender's purpose of the text read?). This interpretation is clear from the achievement percentages, close to 70% of the sample, except for the 2017 cohort, which shows a low mastery of this skill. LO 6 shows lower results than the previous one, as around half of the students did not correctly answer the items associated with evaluating the substantive aspects (content) of the text and the form adopted by the text (or parts of it), according to its communicative purpose. The evaluation of this LO can be seen in statements that ask about the attitudes of the sender, for example, or the usefulness of textual elements (such as quotations), including in terms of a communicative purpose (e.g. What textual resource supports the sender's thesis?).

Discussion and conclusions

The research carried out considered, as material for analysis, the results obtained by four groups of students who entered a private higher education institution, in the Reading Comprehension axis of the Communication Skills Test applied to first-year students during each academic year. Given that this test was designed according to an original assessment model, the first research question asked about the usefulness of this model, based on the analysis of the results of the applications of the instrument developed. The data corroborated that the test manages to measure validly and reliably the students' abilities to interact with academic texts, since the expert validations implied improvements in the selection of fragments and the formulation of questions and, subsequently, thanks to the applications, it was seen that the reliability index (Cronbach's alpha) was satisfactory, as well as the difficulty and discrimination of each of the items.

The contribution of this research is twofold. On the one hand, an unpublished assessment model is presented, which, according to the data obtained, is defined as a satisfactory proposal for assessing academic reading; and, on the other hand, the results of its application are presented and analysed, to provide evidence of the reading skills applied to academic texts of students entering a university institution. The instrument developed was based on theoretical aspects of reading and writing skills at the higher education level, on contributions about academic literacy and the linguistic-discursive procedures of scientific writing. Its purpose was to evaluate, in a diagnostic way, the population of new students and thus to be a first step in the literacy education process that takes place in university education.

The research carried out offers two projections that point in two different directions. One is related to the evaluative aspect of the experience and consists of the idea of improving the written production axis - which is currently evaluated with multiple-choice items - through the incorporation of open-ended questions, which would allow students' textualisation to be evaluated and which would undoubtedly be a better tool for approaching their academic writing skills. A second aspect is didactic and can be synthesised in the following question: what didactic design would be effective in producing a significant improvement in the reading comprehension of expository-argumentative texts by higher-level students? Both projections are envisaged as future research in the field of academic literacy.

The two research questions formulated at the beginning were related to the expected results, which constitute evidence about the academic reading skills of the tested population. A first approach shows that academic reading is only moderately developed, since, depending on the cohort, between 27% and 51% of each group assessed did not achieve a satisfactory result, which is amplified in importance if one considers that the reading of texts with a low level of specialisation was involved so that they were expected to be accessible to students of any career. It was noted that not all of them were able to reach a minimum level of reading comprehension (given by the 60% of correct answers in the 25 items of the Reading Comprehension axis). Therefore, the first conclusion is constituted by the following inference: there is an important group of students for whom the reading of texts from the bibliography of their degree courses will represent a higher level of difficulty, bearing in mind that these texts are longer and have a greater conceptual and syntactic complexity than those of everyday reading, as well as having a specialised lexicon. This more deficient group will require well-targeted remedial action and accompaniment in their process of acquiring the academic register, a task which is mainly carried out by teachers of first-year subjects.

The results of each reading task assessed show different degrees of mastery. Thus, the elicitation of literal information (LO 1) shows good anchoring due to the fact that all cohorts having more than half of their students at the achieved level. This reveals an ability to decode the textual surface. However, the higher LOs show differences in implicit reading skills: synthesis of information (LO 2) has good achievement, which shows an ability to extract the main idea from parts of the text or the whole text (obtaining semantic macrostructure); while the ability to infer is quite low (LO 4), as well as the relationship between paragraphs (LO 3). These results indicate a lack of comprehension reading since the literature (León, 2003; McNamara, 2004) has pointed out the importance of making inferences in reading, since inferences are connections between the reader's previous knowledge and the textual content, and therefore, by making them, the reader is linking aspects of the text that are not explicitly related, as well as anticipating possible meanings, according to his/her knowledge of the subject and intertexts. On the other hand, the relationship between paragraphs is also important, since its recognition implies that the reader has been skilful in identifying the "path" of the text, in the sense that he/she has been able to establish the different discursive functions of the paragraphs and the rhetorical moves that follow one after the other, and which are typical of academic discourse.

Finally, the interpretation of the sender's communicative purposes exhibits a good achievement -superior to the inferences and relationship between paragraphs-, which reveals the ability to determine the positioning of the sender throughout the text. The evaluation of textual content and form has a lower achievement, but is nevertheless high (above 50% of the groups tested), considering that the same students had worse results in the previous skills. This is explained by the fact that the mental representations of Kintsch's (1998) model. That is to say, text-base and situation model are not successive and do not depend on each other "(...) because in reality it is not that one moves from one level of representation to another, but that researchers establish such a separation to study the complex phenomenon of textual comprehension" (Tijero, 2009, p. 126). In other words, and as corroborated by the results obtained, it is possible for a reader to obtain a good semantic macrostructure of the text read and to determine the sender's intentions, although his or her comprehension of the literal level is moderately achieved and his or her inferences are limited.

As a final idea of the research, there is still a need to apply academic literacy measures in students entering higher education, considering that the diagnostic assessment carried out showed deficiencies in academic reading.

References

Barton, D., & Hamilton, M. (1998). Local Literacies-Reading and Writing in One Community. Routledge.

Bhatia, V. (2004). Worlds of written discourse. A genre-based view. Continuum.

Bordón, P., Canals, C., & Rojas, S. (2015). Determinantes de la Retención en los programas e instituciones de Educación Superior: Nueva evidencia para Chile. Revista Estudios de Política Educativa, 2, 176-214. https://centroestudios.mineduc.cl/wp-content/uploads/sites/100/2017/06/EPE_N2.pdf

Braslavsky, B. (2005). El lenguaje escrito y los actos de leer y escribir. En B. Braslavsky (Ed.), Enseñar a entender lo que se lee. La alfabetización en la familia y en la escuela (pp. 41-47). Fondo de Cultura Económica.

Cabrera, M. (2017). Evaluación diagnóstica de Habilidades Comunicativas año 2017. Facultad de Educación, Universidad de Las Américas.

Cabrera, M., & Caruman, S. (2020). Habilidades de lectura de jóvenes al término de la secundaria: ¿qué se predice para su lectura en contextos universitarios? Álabe, 21, 1-17. https://doi.org/10.15645/Alabe2020.21.11

Carlino, P. (2005). Escribir, leer, y aprender en la universidad. Una introducción a la alfabetización académica. Fondo de Cultura Económica.

Carlino, P. (2013). Alfabetización académica diez años después. Revista Mexicana de Investigación Educativa, 18(57), 355-381. https://www.redalyc.org/pdf/140/14025774003.pdf

Castelló, M. (2014). Los retos actuales de la alfabetización académica: estado de la cuestión y últimas investigaciones. Enunciación, 19(2), 346-365. https://doi.org/10.14483/10.14483/udistrital.jour.enunc.2014.2.a13

Coronado, A. (2017). Escritura académica: retos y enfoques ante sus dificultades de aprendizaje. Magister, 29(1), 1-6. https://doi.org/10.17811/msg.29.1.2017.25-30

Díaz, P., & Leyva, E. (2013). Metodología para determinar la calidad de los instrumentos de evaluación. Educación Médica Superior, 27(2). http://www.ems.sld.cu/index.php/ems/article/view/173

Escoriza, J. (2003). Evaluación del conocimiento de estrategias de comprensión lectora. Edicions Universitat de Barcelona.

Estienne, V., & Carlino, P. (2004). Leer en la universidad: enseñar y aprender una cultura nueva. Uni-pluri/versidad, 4(3), 9-17. https://revistas.udea.edu.co/index.php/unip/article/view/12210

Fang, Z. (2012). Language correlates of Disciplinary Literacy. Topics in Language Disorders, 32(1), 19-34. https://doi.org/10.1097/TLD.0b013e31824501de

Felipe, A., & Villanueva, J. D. (2018). Diseño y validación de una prueba piloto para la evaluación de la competencia lectora de estudiantes universitarios. Investigaciones sobre Lectura, 10, 95-117. https://doi.org/10.37132/isl.v0i10.258

Gee, J. P. (1991). What is literacy? In C. Mitchell, & K. Weiler (Eds.). Rewriting literacy. culture and the discourse of the other (pp. 3-11). Bergin & Garvey.

Guerra, J., & Guevara, C. (2017). Variables académicas, comprensión lectora, estrategias y motivación en estudiantes universitarios. Revista electrónica de investigación educativa, 19(2), 78-90. https://doi.org/10.24320/redie.2017.19.2.1125

Hyland, K. (2009). Academic Discourse: English in a Global Context. Continuum.

Kintsch, W. (1998). Comprehension. A paradigm for cognition. Cambridge University Press.

Lea, M. R., & Street, V. (1998). Student writing in higher education: An academic literacies approach. Studies in Higher Education, 23(2), 157-172. https://doi.org/10.1080/03075079812331380364

León, J. A. (Coord.) (2003). Conocimiento y discurso. Claves para inferir y comprender. Pirámide.

McNamara, D. (2004). Aprender del texto: Efectos de la estructura textual y las estrategias del lector. Revista Signos, 37(55), 1-12. http://doi.org/10.4067/S0718-09342004005500002

McNamara, D. (2010). Strategies to read and learn: overcoming learning by consumption. Medical education, 44, 340-346. https://doi.org/10.1111/j.1365-2923.2009.03550.x

Meyer, B., & Ray, M. (2011). Structure strategy interventions: Increasing reading comprehension of expository text. International Electronic Journal of Elementary Education, 4(1), 127-152. https://files.eric.ed.gov/fulltext/EJ1070453.pdf

Ministerio de Educación (Chile) (MINEDUC). (2009). Objetivos Fundamentales y Contenidos Mínimos Obligatorios de la Educación Básica y Media. Actualización 2009. https://bibliotecadigital.mineduc.cl/bitstream/handle/20.500.12365/270/MONO-214.pdf?sequence=1&isAllowed=y

Ministerio de Educación (Chile) (MINEDUC). (2015). Bases Curriculares 7° básico a 2° medio. Lengua y Literatura. Unidad de Currículum y Evaluación. http://www.curriculumnacional.cl/inicio/

Montes, M. E., & López, G. (2017). Literacidad y alfabetización disciplinar: enfoques teóricos y propuestas pedagógicas. Perfiles Educativos, 39(155), 162-178. https://doi.org/10.22201/iisue.24486167e.2017.155.58062

Mullis, I., & Martin, M. O. (Eds.) (2013). PIRLS 2016. Assessment Framework, TIMSS & PIRLS International Study Center, Boston College.

Muñoz, C., Valenzuela, J., Avendaño, C., & Núñez, C. (2016). Mejora en la motivación por la Lectura Académica: la mirada de estudiantes motivados. Ocnos, 15(1), 52-68. http://doi.org/10.18239/ocnos_2016.15.1.941

Neira, A., Reyes, F., & Riffo, B. (2015). Experiencia académica y estrategias de comprensión lectora en estudiantes universitarios de primer año. Literatura y lingüística, 31, 221-244. https://doi.org/10.29344/0717621X.31.1536

O´Halloran, K. (2012). Análisis del discurso multimodal. Revista Latinoamericana de Estudios del Discurso, 12(1), 75-97. http://doi.org/10.35956/v.12.n1.2012.p.75-97

OCDE (2017). Marco de lectura de Pisa para el Desarrollo. En Marco de Evaluación y de Análisis de PISA para el Desarrollo: Lectura, matemáticas y ciencias. Versión preliminar (pp. 31-60). OECD Publishing. https://www.oecd.org/pisa/aboutpisa/ebook%20-%20PISA-D%20Framework_PRELIMINARY%20version_SPANISH.pdf

Pérez, M., Raído, M. D., Ovalle, M., González, L., Calero, E., Piedra, A., & Calero, A. (2016). El conocimiento de la estructura textual: una estrategia que ayuda al alumnado de Educación Primaria. Didáctica. Lengua Y Literatura, 28, 215-242. https://doi.org/10.5209/DIDA.54080

Pérez, I., & Natale, L. (2016). Inclusión en educación superior y alfabetización académica. In L. Natale y D. Stagnaro (Comps.). Alfabetización académica. Un camino hacia la inclusión en el nivel superior (pp. 13-46). Ediciones UNGS.

Perfetti, C., & Hart, L. (2002). The lexical quality hypothesis. Precursors of functional literacy, 11, 189-213. https://doi.org/10.1075/swll.11.14per

Reyes, S., Castillo, A., Zúñiga, A., & Llarena R. (2009). La evaluación objetiva de la habilidad lectora en la educación media superior. Marcos de Referencia. Centro Nacional de Evaluación para la Educación Superior, (CENEVAL). https://www.academia.edu/37624159/La_evaluaci%C3%B3n_de_la_Habilidad_Lectora_en_la_EMS

Sabaj, O. (2012). Uso de movidas retóricas y patrones léxico-gramaticales en artículos de investigación en español. Implicancias para la enseñanza de la escritura científica. Boletín de Filología, 47(1), 165-186. https://boletinfilologia.uchile.cl/index.php/BDF/article/view/22674/23991

Solé, I. (2012). Competencia lectora y aprendizaje. Revista Iberoamericana de Educación, 59, 43-61. https://rieoei.org/historico/documentos/rie59a02.pdf

Sologuren, E., & Castillo, M. N. (2020). Lectura en y a través de las disciplinas: la comprensión de géneros académicos escritos. Una exploración al ámbito desde algunas teorías y modelos de comprensión del discurso escrito. Investigaciones sobre Lectura, 14, 42-57. https://doi.org/10.37132/isl.v0i14.320

Street, B. (1984). Literacy in theory and practice. Cambridge University Press.

Swales, J. M. (1990). Genre Analysis. English in academic and research settings. Cambridge University Press.

Tijero, T. (2009). Representaciones mentales: discusión crítica del modelo de situación de Kintsch. Onomazein 19(2009/1), 111-138. http://onomazein.letras.uc.cl/Articulos/19/6_Tijero.pdf

Uccelli, P., & Meneses, A. (2015). Habilidades de lenguaje académico y su asociación con la comprensión de lectura en la escuela primaria y media: Un nuevo constructo operacional. Miríada Hispánica, 10, 179-206. http://www.miriadahispanica.com/revista/46bdee9011025a5f574e7933d9ba68983c038150.pdf

Van Dijk, T., & Kintsch, W. (1983). Strategies of discourse comprehension. Academic Press.

Venegas, R., Zamora, S., & Galdames, A. (2016). Hacia un modelo retórico-discursivo del macrogénero Trabajo Final de Grado en Licenciatura. Revista Signos, 49(S1), 247-279. http://doi.org/10.4067/S0718-09342016000400012

Zavala, V. (2008). La literacidad o lo que la gente “hace” con la lectura y la escritura. Textos de Didáctica de la Lengua y la Literatura, 47, 71-79. https://www.udea.edu.co/wps/wcm/connect/udea/28b1862b-59a7-4eb8-8d1a-bd3eff425b5c/doc-guia-lengua-literatura.pdf?MOD=AJPERES

Author notes

* Contact: mcabrerap@udla.cl

Secciones
Ocnos
ISSN: 2254-9099
Vol. 20
Num. 3
Year. 2021

Assessing academic reading in students entering Higher Education

MarcelaFranciaJuana Cabrera-PommiezLara-InostrozaPuga-Larraín
Universidad de Las AméricasUniversidad de Las AméricasUniversidad Diego Portales,ChileChileChile
Contexto
Descargar
Todas