Published Aug 27, 2022

Yaojing Xie  


This study seeks to investigate the relationship between text formality and writing quality by quantifying text formality in EFL English writing. In order to conduct the research, 548 English argumentation compositions were evaluated in terms of the five Coh-Metrix text formality dimensions: narrativity, syntactic simplicity, word concreteness, referential coherence, and deep cohesion. Based on the study, there is only a weak relationship between text formality and writing quality, and some facets of text formality, such as narrativity, word concreteness, syntactic simplicity, and deep cohesion, have varying degrees of influence on writing quality and, consequently, on English writing scores. As per statistics, narrativity has a detrimental effect on writing quality; word concreteness has a significant positive correlation with writing quality; syntactic simplicity shows a relatively small negative correlation with writing quality; deep cohesion has a modestly negative correlation with writing quality; and referential coherence has no correlation with writing quality. It is intended that the findings of this study will have some bearing on how English argumentative writing is taught and learned.


Download data is not yet available.



Text Formality, Writing Quality, Coh-Metrix

Andren, M., Sanne, J. M., & Linell, P. (2010). Striking the balance between formality and informality in safety-critical communication: Train traffic control calls. Journal of Pragmatic, 42(1):220-241. DOI: https://doi.org/10.1016/j.pragma.2009.05.022

Atkinson, J. M. (1982). Understanding formality: The categorization and production of ‘formal’ interaction. The British Journal of Sociology, 33(1):86-117. DOI: https://doi.org/10.2307/589338

Biber, D. (1988). Variation across Speech and Writing. New York: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511621024
Bussmann, H. (1996). Routledge Dictionary of Language and Linguistics. London: Routledge.

Chafe, W. L. (1982). Integration and involvement in speaking, writing, and oral literature. In D. Tannen (Ed.), Spoken and Written Language: Exploring Orality and Literacy. Norwood, NJ: Ablex.

Clark, H. H. (1996). Using Language [M]. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511620539

Crossley, S. A. & McNamara, D. S. (2011). Understanding expert ratings of essay quality: Coh-Metrix analyses of first and second language writing. International Journal of Continuing Engineering Education and Life-Long Learning, 21(2-3):170-191.

Diao, X. Y. (2019). The correlation between discourse coherence and writing quality of college students’ English compositions: A study based on Coh-Metrix. Beijing: Beijing University of Posts and Telecommunications

Du, H. Y. & Cai, J. T. (2013). The predictive model of argumentative writing quality of Chinese English learners based on Coh-Metrix. Modern Foreign Languages, 2013(3):293-300.

Fang, A. C. & Cao, J. (2009). Adjective density as a text formality characteristic for automatic text classification: A study based on the British National Corpus. In Proceedings of the PACLIC, 1:130-139.

Graesser, A. C., Jeon, M., Yan, Y., & Cai, Z. (2007). Discourse cohesion in text and tutorial dialogue. Information Design Journal, 15(3):199-213. DOI: https://doi.org/10.1075/idj.15.3.02gra

Graesser, A. C., McNamara, D. S., Cai, Z., Conley, M., Li, H. Y., & Pennebaker, J. (2014). Coh-Metrix measures text characteristics at multiple levels of language and discourse. The Elementary School Journal, 115(2):210-229. DOI: http://dx.doi.org/10.1086/678293

Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix: providing multilevel analyses of text characteristics. Educational Researcher, 40(5):223-234. DOI: https://doi.org/10.3102/0013189X11413260

Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, and Computers, 36(2):193-202. DOI: https://doi.org/10.3758/bf03195564

Gui, L. (2010). A Contrastive study of lexical proficiency between L1 and L2 compositions via computerized assessment. Foreign language Teaching and Research, 2010(6):445-450.

Heylighen, F. & Dewaele, J. M. (2002). Variation in the contextuality of language: An empirical measure. Foundations of Science, 2002(7):293-340. DOI: https://doi.org/10.1023/A:1019661126744

Labov, W. (1972). Sociolinguistic Patterns. Philadelphia: University of Pennsylvania Press.

Li, H. Y., Graesser, A. C., & Cai, Z. (2013). Comparing two measures of formality. In C. Boonthum-Denecke & G. M. Youngblood (eds.). Proceedings of the 26th International Florida Artificial Intelligence Research Society Conference (pp.220-225). Palo Alto, CA: AAAI Press.

Li, H. Y., Graesser, A. C., Conley, M., Cai, Z. Q., Pavlik, P. I., & Pennebaker, J. W. (2016). A new measure of text formality: An analysis of discourse of Mao Zedong. Discourse Processes, 53(3):205-232. DOI: https://doi.org/10.1080/0163853X.2015.1010191

Louwerse, M. M., McCarthy, P. M., McNamara, D. S., & Graesser, A. C. (2004). Variation in language and cohesion across written and spoken registers [A]. In K. D. Forbus, D. Gentner, & T. Regier (eds.). Proceedings of the 26th Annual Meeting of the Cognitive Science Society (pp. 843-848). Mahwah, NJ: Erlbaum.

McNamara, D. S., Graesser, A.C., McCarthy, P.M., & Cai, Z. (2014). Automated Evaluation of Text and Discourse with Coh-Metrix. Cambridge: Cambridge University Press. pp. 1, 84, 86, 87.

Olson, D. R. (1977). From utterance to text: The bias of language in speech and writing. Harvard Educational Review, 47(3):257-281. DOI: https://doi.org/10.17763/haer.47.3.8840364413869005

Richards, J., Platt, J., & Platt, H. (1997). Dictionary of Language Teaching and Applied Linguistics. London: Longman.

Wang, H. W. (2012). The relationship between text features and second language writing proficiency: An analysis based on Coh-Metrix [D]. Changsha: National University of Defense Technology.

Wen, Q. F, Liang, M. C., & Yan, X. Q. (2009). Spoken and Written English Corpus of Chinese Students (SWECCL 2.0). Beijing: Foreign Language Teaching and Research Press.
How to Cite
Xie, Y. (2022). The Relationship between Text Formality and Writing Quality: An Evaluation Based on Coh-Metrix. Science Insights Education Frontiers, 12(2), 1729–1739. https://doi.org/10.15354/sief.v12i2.824
Original Article