Critical thinking performance|Article review

    Critical thinking performance|Article review

    H

    E a

    b

    a

    A R R A A

    K C C C L L

    1

    c W i o i M c e

    1

    Thinking Skills and Creativity 21 (2016) 41–49

    Contents lists available at ScienceDirect

    Thinking Skills and Creativity

    j ourna l h o mepa ge: h t tp : / /www.e lsev ier .com/ locate / tsc

    ow might language affect critical thinking performance?

    mmanuel Manaloa,∗, Chris Sheppardb

    Graduate School of Education, Kyoto University, Kyoto, Japan Center for English Language Education, Faculty of Science and Engineering, Waseda University, Tokyo, Japan

    r t i c l e i n f o

    rticle history: eceived 27 July 2015 eceived in revised form 29 March 2016 ccepted 11 May 2016 vailable online 12 May 2016

    eywords: ritical thinking skills instruction ritical evaluation ognitive cost anguage proficiency anguage structure

    a b s t r a c t

    This study examined whether language structure or language proficiency might affect students’ critical thinking performance. Previous research has claimed that many non- Western students struggle with the demands of demonstrating critical thought. Two language-related causes have been suggested: one concerning structural limitations in the non-Western students’ first language, and the other concerning their second language pro- ficiency. In Study 1 described here, reports written by 110 Japanese second year university students, who had received instruction in academic discourse for critical evaluation (which is one aspect of critical thinking), were analyzed for use of evaluative statements. No dis- advantage was found for use of the Japanese language, which is considered as having a more indirect structure that may make critical evaluation more difficult. Measurements of language proficiency in English and Japanese, however, were found to correlate with pro- duction of evaluative statements in those respective languages suggesting that language proficiency could affect critical evaluation use. In Study 2, the same task was given to 43 first year students who had not yet received the same instruction. Analysis revealed similar patterns in their written work but at a lower level, suggesting that the second year students had benefitted from the skills instruction. Furthermore, unlike the second year students, the first year students evidenced no correlations between their language proficiency scores and their production of evaluative statements, suggesting that proficiency on its own is inadequate: students need instruction on the specific language forms and structures to use to demonstrate critical thinking in their written work.

    © 2016 Elsevier Ltd. All rights reserved.

    . Introduction

    In modern societies, the ability to evaluate the credibility of information that one encounters is an essential skill, especially onsidering the proliferation of unvetted information through the Internet and other forms of mass media (e.g., Glassner,

    einstock, & Neuman, 2005; Thomm & Bromme, 2011). However, educational development of critical thinking skills, which ncludes critical evaluation, is not a straightforward matter. There are, for example, disagreements about appropriate meth- ds for critical thinking skills instruction and assessment, and various individual and situational factors are believed to nfluence the development and use of such skills (e.g., Davies, 2006; Halpern, 1998; Ku, 2009; Manalo, Kusumi, Koyasu,

    ichita, & Tanaka, 2013, 2015; Ten Dam & Volman, 2004). One of the main areas of contention is whether cultural background ontributes to differences in critical thinking performance (e.g., Ennis, 1998; Manalo et al., 2013, 2015). Some authors, for xample, have portrayed Asian students as being deficient in critical thinking compared to Western students (e.g., Atkinson,

    ∗ Corresponding author at: Graduate School of Education, Kyoto University, Yoshida-Honmachi, Sakyo-ku, Kyoto 606-8501, Japan. E-mail address: manalo.emmanuel.3z@kyoto-u.ac.jp (E. Manalo).

    http://dx.doi.org/10.1016/j.tsc.2016.05.005 871-1871/© 2016 Elsevier Ltd. All rights reserved.

    dx.doi.org/10.1016/j.tsc.2016.05.005
    http://www.sciencedirect.com/science/journal/18711871
    http://www.elsevier.com/locate/tsc
    http://crossmark.crossref.org/dialog/?doi=10.1016/j.tsc.2016.05.005&domain=pdf
    mailto:manalo.emmanuel.3z@kyoto-u.ac.jp
    dx.doi.org/10.1016/j.tsc.2016.05.005
    42 E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49

    1997; Fox, 1994), and it has been found that many instructors at the tertiary level subscribe to such a view (e.g., Lee & Carrasquillo, 2006; Robertson, Line, Jones, & Thomas, 2000).

    Two language-related explanations have been proposed for the apparent differences in critical thinking performance manifested by students from different cultural backgrounds. One of those explanations concerns the structure of the student’s native language (L1). This explanation posits that, due to their structure, some languages may present constraints in the ease with which certain thinking skills can be undertaken or expressed. This explanation is sometimes referred to as the “Sapir- Whorf hypothesis” or the “linguistic relativity hypothesis” (see, e.g., Au, 1983; Hill & Mannheim, 1992; Hockett, 1954; Whorf, 1956). An example of a claim of this kind is Bloom’s (1981) proposal that counterfactual thinking (i.e., thinking about what might have been, contrary to facts) may be more difficult in Chinese compared to English.

    Some observations of linguistic differences, such as “indirectness” being a feature more prevalent in some languages, particularly Asian languages (e.g., Kong, 2005), would appear to support the idea that language structure could affect the ease with which certain modes of thinking could be undertaken or expressed. In a study by Itakura and Tsui (2011), for example, evidence was found that book reviewers use different strategies to convey critical evaluation when writing in Japanese compared to English: in Japanese, criticism is usually indirectly conveyed and is frequently preceded by an apology. Previous studies, however, have failed to provide convincing evidence that language structures could actually impose constraints in what users of the language can communicate. Although the earlier-mentioned study by Bloom (1981) claimed to have found evidence for such constraints where counterfactual thinking in the Chinese language is concerned, subsequent investigations failed to replicate or support Bloom’s results (Au, 1983; Liu, 1985; Takano, 1989; Vorster & Schuring, 1989). Even where the weaker version of the linguistic relativity hypothesis is concerned (i.e., that language can merely influence people’s thought patterns, rather than constrain their thought processes), the findings of the few studies that claim to have found supporting evidence (e.g., Boroditsky, 2001) have not been replicated in subsequent independent investigations (e.g., Chen, 2007; January & Kako, 2007; Tse & Altarriba, 2008).

    The other language-related explanation for the apparent culture-based differences in critical thinking performance con- cerns students’ second language proficiency (e.g., Floyd, 2011; Lun, Fischer, & Ward, 2010; Paton, 2005). This explanation is based on the observation that the students who are usually reported as having lower levels of critical thinking competence are international students in English speaking countries. It suggests that, because most of those students have to use a sec- ond language (L2, which is usually English) in their host environment, and they may lack adequate proficiency in that L2, they would likely manifest lower competence when performing tasks that require use of that L2. Tasks that are likely to get affected include cognitive tasks like critical thinking.

    One way of understanding this possible influence of language proficiency on critical thinking performance is in terms of cognitive cost (i.e., the mental resources cost associated with task performance). Language processing entails the use of cognitive resources in working memory (Baddeley, 1986, 1998), and lower proficiency in a language would require the use of more resources. Thinking critically would likewise require the use of working memory resources. There are, however, limited resources available in working memory (Baddeley, 1986, 1998) and, if a considerable amount of those resources has already been expended on utilizing a language in which proficiency is low, there may not be adequate resources remaining for the satisfactory execution of critical thinking.

    The negative impact of the higher cognitive cost entailed in using a language in which proficiency is low, on the execution of other cognitive tasks, has been demonstrated in previous research. Takano and Noda (1993, 1995) showed that the use of a foreign language detrimentally affects performance in concurrently undertaken non-linguistic tasks like arithmetic calculation and mental imagery, and Manalo and Uesaka (2012, 2014) reported evidence indicating that students’ lower proficiency in an L2 limits their ability to use diagrams when explaining information in that L2. Where critical thinking is concerned, both Lun et al. (2010) and Floyd (2011) reported indications that lower proficiency levels in English could detrimentally affect Asian students’ performance in critical thinking tests administered in English. However, neither of those studies used appropriate, objective measures of L2 proficiency to reliably confirm the connection between L2 proficiency and critical thinking skills performance.

    1.1. Overview of the present study

    The present study examined whether there might be evidence to support either or both (i) the language structure expla- nation, and/or (ii) the language proficiency explanation, in students’ manifestation of critical thinking in their written work. The study was not intended to be a comprehensive test of the language structure hypothesis: it examined only whether, in the written work of Japanese university students, there might be observable differences in the presence of critical thinking qualities depending on the language used, Japanese or English. Critical thinking was operationalized as students’ use of evaluative statements. This decision was based on the fact that use of evaluative statements comprises a salient expression of critical evaluation, which in turn is central to the notion of critical thinking application (e.g., Fisher & Scriven’s, 1997, p. 21, definition of critical thinking as “skilled and active interpretation and evaluation of observations and communications, information and argumentation” – italics added).

    In the present study, Japanese was deemed an appropriate language to examine because, like a number of other Asian languages, it employs patterns of expression that make it more indirect and inductive compared to English (e.g., Itakura & Tsui, 2011; Scollon & Wong-Scollon, 1991). Evaluation, however, requires precision and directness in conveying judgments about the quality or value of the subject being referred to. Thus, structural features of the Japanese language could make

    t l

    m O r t w a b t a h b

    e t – a

    h t p r l n t

    u i a l s l t a

    2

    E o

    2

    2

    F c d

    2

    e r c

    t w a t

    E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 43

    he production of evaluative language relatively more difficult. If so, it should be possible to detect lower rates of evaluative anguage use in the students’ written work in Japanese compared to English.

    It was equally important to consider whether using an L2 might detrimentally affect students’ critical evaluation perfor- ance. If this was the case, one would expect that critical evaluation performance in the L1 would exceed that in the L2. ne would also expect to find a relationship between the critical evaluation performance and L2 proficiency. Thus, possible

    elationships between students’ L2 proficiency and their production of evaluative statements were investigated. The ques- ion was whether L2 proficiency would manifest as a limiting factor because lower proficiency entails higher cognitive cost hen using the L2, leaving insufficient resources in working memory for critical evaluation. If this explanation is supported,

    relationship should be found between the students’ L2 proficiency and their evaluative statements production in the L2, ut not in the L1. A relationship in both the L1 and the L2 would suggest that general language or intellectual abilities – rather han L2 proficiency – affect critical evaluation performance. The reason is that language abilities, and intellectual abilities nd performance, are generally considered as being related (e.g., Ackerman, 1986; Neisser et al., 1996). Thus, a student with igher language and intellectual abilities could be expected to score higher in measurements of L2 proficiency, and evidence etter performance in tasks like critical evaluation – in both their L1 and L2.

    Furthermore, it is conceivable that this kind of relationship between language proficiency and critical evaluation also xists in the L1. In other words, that the more proficient or skilled a student might be in using his or her native language, he less cognitive resources such use might demand, and the more resources might be made available for critical evaluation

    which could then be better executed. Thus, in the present study, the possible existence of this relationship in the L1 was lso investigated.

    When considering the development of critical evaluation skills through instruction, if the language structure explanation olds, one would expect that skills instruction provided in the L2 would not result in any discernible skills improvement in he L1 (i.e., since the L1 has inherent structural constraints on the use or expression of those skills). In contrast, if the language roficiency explanation holds, one could expect that skills instruction provided in the L2 – where proficiency is lower – would esult in at least some discernable transfer to the L1, where proficiency is higher and more cognitive processing resources are ikely to be available for the application of those skills. Critical evaluation skills are thinking skills and its acquisition should ot be bound to the language of instruction. The important question is whether constraints or limitations in the language o use could detrimentally affect critical evaluation performance.

    The research reported here comprised two related studies. In Study 1, evaluative statements that second year Japanese niversity students produced in Japanese (their L1) and in English (their L2) were examined. These students had received

    nstruction on academic discourse. Thus, they were not naïve as to the requirements of expressing evaluative language, nd any differences in the writing they produced in L1 and L2 could be attributed to either the inherent structure of the anguage they were using or their proficiency in using that language. In Study 2, the same writing task was given to first year tudents who had received little instruction on academic discourse, and nothing explicit on the production of evaluative anguage. The purpose of this second study was to find out if the characteristics of L1 and L2 written work produced by he first year students, compared to their second year counterparts, differed. Hence, the aim was to find out whether the dditional instruction that had been received by the more advanced second year students might have made a difference.

    . Study 1

    Study 1 tested two hypotheses. The first was that students’ production of evaluative statements in Japanese and in nglish would differ. The second was that the students’ proficiency in a particular language would be related to the amount f evaluative statements they produce in that language.

    .1. Method

    .1.1. Participants The participants were 110 Japanese university students in their second year of study in science and engineering disciplines.

    or these students, Japanese is L1 and English is L2. They were taking a compulsory English communication skills development ourse that covers oral and written academic discourse using a task-based learning approach. The students came from four ifferent classes in that course.

    .1.2. Materials and procedure As part of the communication skills course, the students were provided class instruction, textbook explanations and

    xamples (Anthony, Rose, & Sheppard, 2010), and practice in the use of language appropriate for critical evaluation, including anking and debating different reasons and other forms of alternatives (e.g., clearly stating the premises, and then drawing onclusions). These materials and the instruction were all provided in English.

    For the purposes of the present investigation, the students were additionally provided with a single page Japanese

    ranslation of the part of the textbook dealing with how to make valid arguments. They were also supplied brief (one page) ritten examples (one in English and one in Japanese) of how alternative reasons could be ranked according to judgments
    bout their relative importance. The example texts conveyed someone’s opinion about the most important reason for learning he English language, among four possible reasons. The texts provided examples of evaluative statements and provision of

    44 E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49

    support for claims, although those were not labeled or overtly identified in any way in the texts. The equivalence and appropriate use of language in the English and Japanese versions were checked by several bilingual teachers of the course. Although all materials provided in the course are usually in English, the Japanese versions were supplied in this case to avoid possible disadvantage to the students’ production of evaluative language in Japanese (i.e., without the Japanese versions, it could be argued that the students might have simply been unfamiliar with the equivalent Japanese expressions for critical evaluation).

    During two consecutive 90-min class sessions of the course, the students were introduced to the Titanic and Space Shuttle Challenger disasters, including four basic causes that have been proposed for the occurrence of each of those disasters. During the class sessions, the students participated in guided exercises to explore and discuss the disasters and their corresponding possible causes.

    For homework, the students were asked to write two brief reports to explain what they considered to be the most important cause of each of the disasters. To avoid any possible misunderstandings about the requirements of the homework task, written instructions were provided in Japanese. The students were randomly assigned to write one report in English and the other in Japanese (i.e., if they were asked to write the Titanic report in English, they had to write the Challenger report in Japanese, and vice versa).

    2.1.3. Analyses The total number of sentences [Total], and the number of evaluative sentences (i.e., sentences where some evaluation

    of the relative value of the topic is made) [Evaluative], were counted and scored in the analysis of the students’ written work.

    Operational criteria were drawn up for determining what data counted as “evaluative sentences”. Examples of such requirements included: the sentence must explicitly say something about the worth or value of the subject, and that worth or value must be in comparison to something else. Conditional statements that explicitly convey a relative evaluation of the subject were counted. The following examples, in contrast, did not count: the use of simple adjectives or adverbs to describe something, prescriptive statements not explicitly expressing a relative evaluation or judgment, and conditional statements in general. Inter-rater reliability was checked by asking an independent coder to score a randomly selected sample of 25% of the data. Reliability coefficients obtained (Cronbach’s alphas) were deemed to be satisfactory: they were 0.922 in English, and 0.960 in Japanese.

    Language proficiency was measured (i) by using the students’ TOEIC-IP (Test of English for International Communication – Institution Program) scores, and (ii) in terms of the complexity of the sentences the students produced in English and in Japanese (explained below). TOEIC is a norm-referenced test of English listening comprehension and reading skills, widely used as a measure of students’ English language proficiency levels in Japan (http://www.ets.org/toeic). Although there is a speaking and writing test version, it is not as widely used as the listening and reading test version, which is what the TOEIC-IP contains. Students in the faculty where the students came from are required to sit the TOEIC-IP test at regular intervals during their period of enrolment, and their scores on that test are made available to their teachers. The TOEIC test has been used in numerous previous studies as a measure of L2 proficiency (e.g., Dornyei & Katona, 1992; Eggly, Musial, & Smulowitz, 1999), and various studies have reported that it has at least adequate reliability and validity (e.g., In’nami & Koizumi, 2011; Perkins, 1987; Powers, Kim, & Weng, 2008). The TOEIC is also considered a good measure of non-native English speakers’ ability to use English in real life situations rather than just in strictly academic contexts (Eggly et al., 1999). For these reasons, the authors of the present study decided to use the students’ TOEIC-IP scores as one measurement of their English proficiency.

    The other measure of proficiency used – complexity – was selected because, along with accuracy and fluency, it is generally considered as one of the key dimensions of proficiency in language production – both oral and written (e.g., Ellis, 2009; Ishikawa, 1995; Ortega, 2003; Skehan & Foster, 1999; Wolfe-Quintero, Inagaki, & Kim, 1998). However, unlike accuracy and fluency, for which it is difficult to come up with meaningful and equivalent/comparable measurements for English and Japanese, the authors of the present study were able to determine a viable way to measure complexity. Usually, analysis of production complexity in English entails calculating the ratio of words over sentences or terminable units (T-units; e.g., Ishikawa, 1995; Ortega, 2003; Richards, Platt, & Platt, 1992). However, what counts as a word in Japanese is not exactly the same as in English, and thus, such a method for calculating complexity was deemed inap- propriate. Instead, the number of verbs that the students used were counted and divided by their total number of sentences to generate a measure of complexity. Verbs were chosen as they are comparable in English and Japanese, and the use of more verbs can be considered a good indicator of more complex sentence structures (i.e., simple sen- tences usually contain only one verb, and inclusion of more verbs within one sentence usually requires the use of a more complex structure). It should also be noted that even though this measurement of complexity and the measure- ment of the students’ use of critical evaluation were derived from the same source (i.e., the writing that the students produced), these measurements were not dependent on each other. In other words, students can produce complex writing without being critically evaluative, and vice versa: they can write evaluative sentences that are low in complex-

    ity.
    Analyses of variance were conducted to compare the students’ production of sentences (Total, and Evaluative) in English and in Japanese. Correlational analyses were carried out to examine possible relationships with the students’ most recent TOEIC test scores and their complexity scores in English and in Japanese.

    http://www.ets.org/toeic
    E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 45

    Table 1 Means (with standard deviations in brackets) of total and evaluative sentences that the second year students produced in English and in Japanese.

    Total Evaluative

    English 20.31 (5.59) 3.46 (1.76) Japanese 18.68 (5.92) 3.75 (1.80)

    Table 2 Correlation coefficients between second year students’ TOEIC and writing complexity scores, and numbers and proportions of evaluative sentences they produced, according to the language used (effect sizes, R2 shown in brackets).

    Scores English Evaluative Sentences Japanese Evaluative Sentences

    Number Proportion Number Proportion

    TOEIC 0.23* (0.053) 0.14 (0.020) 0.18 (0.032) −0.06 (0.004) English Complexity 0.19* (0.036) 0.24* (0.058) 0.15 (0.023) 0.17 (0.029) Japanese Complexity 0.02 (0.000) 0.09 (0.008) −0.07 (0.005) 0.33*** (0.109)

    *

    *

    *

    2

    i

    c e t

    m J l 1 t

    p t i t

    E t b o c s o b t

    E e

    2

    E h w

    l i

    p < 0.05. *p < 0.01. **p < 0.001.

    .2. Results

    Table 1 shows the means, and standard deviations (in brackets), obtained for Total and Evaluative categories of sentences n the students’ written work in English and in Japanese.

    Repeated measures 2 × 2 ANOVAs demonstrated that there were no significant effects due to the task (i.e., the Titanic ompared to the Challenger reports), hence demonstrating task equivalence. The analysis however revealed significant ffects due to language in both the total number of sentences written [Total], F(1, 108) = 11.47, p = 0.001, �p2 = 0.020; and he number of evaluative sentences [Evaluative], F(1, 108) = 4.81, p = 0.030, �p2 = 0.006.

    These results indicate that the students wrote significantly more sentences in English compared to Japanese, but produced ore of the target evaluative language in Japanese. The ANOVA results also revealed that the students wrote more verbs in

    apanese (mean = 48.52, SD = 14.01) than in English (mean = 38.77, SD = 12.54), F(1, 109) = 84.56, p < 0.001, �p2 = 0.119. The anguage produced was also significantly more complex in Japanese as measured by the number of verbs per sentence (F(1, 09) = 100.8, p < 0.001, �p2 = 0.269), indicating that the reason for the production of fewer sentences in Japanese was because he students’ greater proficiency in that language enabled them to produce more complex sentences.

    Because the total number of sentences that the students produced differed considerably between the two languages, the roportions of evaluative sentences (i.e., Evaluative as a proportion of Total) were also calculated and compared according o the language used. The comparison revealed a significant difference: F(1, 109) = 20.32, p < 0.001, �p2 = 0.039. This result ndicates that the proportion of evaluative sentences was higher in the reports that the students wrote in Japanese compared o those they wrote in English.

    For the correlational analysis, the correlations between the students’ TOEIC scores and their writing complexity scores in nglish and in Japanese, and the evaluative sentences they produced in the respective languages (actual and proportions of otal), were examined. The results are shown in Table 2. The students’ TOEIC scores correlated significantly with the num- er of evaluative sentences they produced in English. The correlation between the students’ TOEIC scores and the number f evaluative sentences they produced in Japanese was not significant. Where the complexity scores were concerned, the omplexity of the students’ writing in English significantly correlated with both the number and proportion of evaluative entences they produced in English. The complexity of their writing in Japanese significantly correlated with the proportion f evaluative sentences they produced in Japanese. All cross language correlations between writing complexity and num- ers/proportions of evaluative sentences were not significant. Taken together, these results lend support to the argument hat language proficiency influences students’ ability to evidence critical evaluation in their writing.

    A significant correlation was also found between the proportions of evaluative sentences the students produced in nglish and in Japanese, r = 0.72, p < 0.001, R2 (effect size) = 0.518. This result suggests that students who were more critically valuative in one language also demonstrated greater critical evaluation in the other language.

    .3. Discussion

    Differences were found in both actual numbers and proportions of evaluative sentences that the students produced in nglish and in Japanese. The direction of the differences, however, was opposite to the language structure-based prediction: igher proportions of evaluative sentences were found in Japanese instead of English. This result suggests that the students

    ere better at producing evaluative language in their L1.
    The significant relationship found between the students’ language proficiency and their critical evaluation performance ends support to the notion that language proficiency could explain apparent cultural differences in students’ critical think- ng performance. When using an L2 (such as English) in which they may lack adequate proficiency (e.g., less automated

    46 E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49

    processing, inadequate linguistic knowledge), Asian students in Western countries may not be able to demonstrate critical thinking to the same degree as their native English-speaker counterparts because of the higher cognitive processing resource requirements of that L2, which depletes resources available in working memory for critical thinking. The important find- ing in the present study is that proficiency in both languages (English and Japanese) significantly correlated with critical evaluation performance in the respective language.

    There still remains the possibility that the significant correlations between the two languages in the proportions of the target language produced, are a result of other factors, which are correlated to both first and second language proficiency, and which might affect the production of evaluative language – irrespective of the language being used. In other words, in addition to language proficiency, people’s critical evaluation performance may also be influenced by some other individual factors – such as general intelligence, working memory capacity, or communication skills – that are independent of the language being used for expression. It would be useful to identify and examine those other factors in future research.

    As noted earlier, the student participants in this first study had already received instruction in academic discourse that included the use of evaluative language. Therefore, an important next question to address was, “To what extent had that instruction affected the relative production of evaluative language in English and in Japanese?” – which was pursued in Study 2.

    3. Study 2

    Study 2 tested three hypotheses. The first was that first year students would manifest lower use of the target evalua- tive language compared to the second year students from whom data were gathered in Study 1. The second was that the differences in evaluative language use would be consistent across English and Japanese. In other words, the prediction was that, like the second year students, the first year students would evidence greater use of evaluative statements in their L1 (Japanese) compared to their L2 (English). The third hypothesis was that the first year students’ proficiency in both languages would not be related to the proportions of evaluative language they produce. This hypothesis was based on the assumption that, without receiving instruction about how to undertake and convey critical evaluation, students’ language proficiency would not make a difference to their production of the target evaluative language as they would have very limited knowledge about the structures necessary for producing the required language.

    3.1. Method

    3.1.1. Participants Forty-three Japanese university students who were in their first year of studies in the same science and engineering

    faculty as the students in Study 1 were added to the participants of Study 1, for a total of 153 participants. The additional students came from two classes of a compulsory first year English communication skills course which deals with various aspects of oral and written academic discourse, but nothing explicit about evaluative language (which is not covered until the second year course).

    3.1.2. Materials, procedure, and analysis For one of their homework assignments, the students were given brief reading materials (in English and in Japanese)

    about the Titanic and Space Shuttle Challenger disasters, including the proposed causes of those disasters. These materials were drawn from the textbook used in the second year course. The Japanese translations were provided to these first year participants to ensure that their subsequent writing performance would not have been compromised by possible difficulties in understanding the English versions. The contents of those materials were not covered in class.

    The homework task that the students had to do was the same as that given to the second year students: to produce two brief reports to explain what they considered to be the most important cause of each of the disasters, after reading the materials provided. Like the second year students, they were randomly assigned to write one report in English and the other in Japanese. Also, like the second year students, they were provided with the one-page examples (one in English and one in Japanese) of how alternative reasons (for learning the English language) could be ranked according to judgments about their relative importance. The crucial difference was that the first year students were not provided class instruction and exercises on the use of academic discourse specifically pertaining to evaluative language.

    The written reports that the students produced were analyzed and scored in the same manner described in the first study. The first and second year students’ data were then compared using repeated measures ANOVAs. Correlational analyses were carried out to examine possible relationships between the students’ language proficiency scores and the proportions of evaluative language they produced.

    3.2. Results

    Table 3 shows the means, and standard deviations (in brackets), obtained for the Total and Evaluative categories of sentences in the first year students’ written work in English and in Japanese.

    Analyses of variance revealed significant effects due to year of enrolment (first year compared to second year) in the stu- dents’ scores for the total number of sentences [Total], F(1, 152) = 22.90, p < 0.001, �p2 = 0.113; and the number of evaluative

    E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 47

    Table 3 Means (with standard deviations in brackets) of total and evaluative sentences that the first year students produced in English and in Japanese.

    Total Evaluative

    English 15.70 (8.20) 2.09 (1.71) Japanese 12.84 (6.28) 2.26 (1.24)

    Table 4 Correlation coefficients between first year students’ TOEIC and writing complexity scores, and proportions of evaluative sentences they produced, according to the language used (effect sizes, R2 shown in brackets).

    Scores Proportions of Evaluative Sentences

    English Japanese

    s 1 s

    f m t a i fi E

    w t

    3

    o l c s t p

    i p i i

    4

    p s fi f a r

    l o r s t

    TOEIC 0.03 (0.000) −0.02 (0.000) English Complexity 0.16 (0.026) −0.18 (0.032) Japanese Complexity 0.01 (0.000) 0.14 (0.020)

    entences [Evaluative], F(1, 152) = 27.20, p < 0.001, �p2 = 0.130. A significant language effects were also found for Total, F(1, 52) = 26.67, p < 0.001, �p2 = 0.026. No significant interaction effects between language and year were found; nor were any ignificant effects found due to the task (Titanic versus Challenger).

    These results indicate that, compared to the second year students, the first year students wrote fewer sentences in total or their reports, and fewer evaluative statements. Like the second year students, the first year students wrote significantly

    ore sentences in English than in Japanese. Analysis of the complexity of the sentences they produced revealed that the otal number of verbs they used was likewise higher in Japanese compared to English, F(1, 43) = 14.80, p < 0.001, �p2 = 0.032; nd the index of complexity (total number of verbs divided by total number of sentences) was higher for what they wrote n Japanese compared to what they wrote in English, F(1, 43) = 75.58, p < 0.001, �p2 = 0.409. This finding indicates that the rst year students, like their second year counterparts, were producing more complex sentences in Japanese compared to nglish.

    The results of the correlational analysis are shown in Table 4. None of the correlations were significant – suggesting that, ithout instruction, the students’ proficiency in both languages generally did not make a difference to their production of

    he target evaluative language.

    .3. Discussion

    The results of Study 2 showed that the second year students wrote more sentences in their reports, and produced more f the target evaluative language, compared to the first year students. This finding suggests that instruction on appropriate anguage to use – which had been provided to the second year students – can improve students’ abilities in manifesting ritical evaluation in their written work. Although as noted the instruction was provided almost entirely in English, the ignificant language effects found were all in favor of the Japanese language, which suggests that there is transfer across he languages in skills acquisition. In other words, skills taught and learned in English also produce improvements in the roduction of evaluative language in Japanese.

    The results of the correlational analysis underline the importance of providing instruction to cultivate student competence n critical evaluation. No significant correlations were found between the proportions of evaluative language the students roduced and their proficiency scores in English and in Japanese. This therefore suggests that general proficiency on its own

    s not adequate: for students to evidence critical evaluation in their written work, it is necessary to also provide them with nstruction on the specific language forms and structures to use.

    . General discussion

    The findings of this study provide evidence that, at least for Japanese students, using the Japanese language (their L1) resents no disadvantage compared to English (their L2) in the production of evaluative language (i.e., the Japanese language tructure is not a limiting factor). How Japanese students’ evaluative language use might compare to that of students whose rst language is structured differently (e.g., native English speakers responding to the same tasks), or students who are

    ully bilingual in Japanese and English, would need to be examined in future research. However, in the present study, there ppeared to be no obvious deficits in evaluative language production in Japanese among the second year students who had eceived instruction in the necessary academic discourse.

    There is evidence in the present study, however, that language proficiency can influence the production of evaluative anguage. Although the conceptualization of critical thinking is likely to be independent of the language being used, the use

    f a language that one is not so proficient in requires greater cognitive processing resources and thus limits the remaining esources that could be utilized for the expression of critical thinking. In this study, the significant correlations between the tudents’ English proficiency scores and the amount of evaluative language they evidenced in their English writing indicate hat performance varied with L2 proficiency. This provides useful evidence to corroborate previously made claims (e.g., Floyd,
    48 E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49

    2011; Lun et al., 2010; Paton, 2005) that some of the shortcomings in critical thinking skills manifested by international students can be attributed to their having to use an L2 in which they may not be as proficient compared to their native speaker counterparts. Although those significant correlations were not high, results were also obtained in the present study showing that the students’ Japanese writing complexity significantly correlated with the proportion of evaluative language they produced in Japanese. This finding provides further support for the argument that language proficiency can influence the production of evaluative language, this time in the L1.

    4.1. Conclusion

    The finding that L2 proficiency is a potential limiting factor in students’ use of critical thinking skills suggests that educational strategies for improving non-native speaker students proficiencies in English (or the language used in the host country) would be beneficial. Successful implementation of such strategies would likely reduce the perceived deficiencies in Asian and other foreign students’ critical thinking competencies (cf. Lee & Carrasquillo, 2006; Robertson et al., 2000). The findings of this study also indicate that appropriate classroom instruction is essential for facilitating the development of students’ abilities to demonstrate critical thinking competencies in the work they produce. The second year students evidenced similar writing profiles to those of first year students; however, having received instructions in academic discourse relevant to critical evaluation, they also produced more of the target evaluative language. They did this in both languages, L1 and L2, even though academic discourse instruction was primarily provided in the L2 – suggesting some transfer of skills across languages. Furthermore, the lack of significant correlations between the first year students’ L1 and L2 proficiency and their use of the target evaluative language underline the necessity of providing critical evaluation instruction to students. Without receiving explicit instruction, most students − irrespective of their language proficiency – would not likely know how to effectively demonstrate critical evaluation in the work they produce.

    List of funder and grant number

    This research was supported by a grant-in-aid (23243071) received from the Japan Society for the Promotion of Science.

    Conflict of interest

    There is no conflict of interest known to the authors.

    Acknowledgements

    The authors would like to thank Kyoko Watanabe and Rongting Shen for their invaluable assistance during various stages of this study.

    References

    Ackerman, P. L. (1986). Individual differences in information processing: an investigation of intellectual abilities and task performance during practice. Intelligence, 10, 101–139.

    Anthony, L., Rose, R., & Sheppard, C. (2010). Concept building and discussion: foundations. Tokyo: DTP Publishing. Atkinson, D. (1997). A critical approach to critical thinking in TESOL. TESOL Quarterly, 31, 9–37. Au, T. K. F. (1983). Chinese and English counterfactuals: the Sapir-Whorf hypothesis revisited. Cognition, 15, 155–187. Baddeley, A. D. (1986). Working memory. Oxford: Oxford University Press. Baddeley, A. (1998). Human memory. Boston: Allyn & Bacon. Bloom, A. (1981). The linguistic shaping of thought. Hillsdale, NJ: Erlbaum. Boroditsky, L. (2001). Does language shape thought?: Mandarin and English speakers’ conceptions of time. Cognitive Psychology, 43, 1–22. Chen, J. Y. (2007). Do Chinese and English speakers think about time differently? Failure of replicating Boroditsky (2001). Cognition, 104, 427–436. Davies, W. M. (2006). An ‘infusion’ approach to critical thinking: Moore on the critical thinking debate. Higher Education Research & Development, 25,

    179–193. Dornyei, Z., & Katona, L. (1992). Validation of the C-test amongst Hungarian EFL learners. Language Testing, 9, 187–206. Eggly, S., Musial, J., & Smulowitz, J. (1999). The relationship between English language proficiency and success as a medical resident. English for Specific

    Purposes, 18, 201–208. Ellis, R. (2009). The differential effects of three types of task planning on the fluency, complexity, and accuracy in L2 oral production. Applied Linguistics,

    30, 474–509. Ennis, R. H. (1998). Is critical thinking culturally biased? Teaching Philosophy, 21, 15–33. Fisher, A., & Scriven, M. (1997). Critical thinking: its definition and assessment. Norwich, UK: Centre for Research in Critical Thinking. Floyd, C. B. (2011). Critical thinking in a second language. Higher Education Research and Development, 30, 289–302. Fox, H. (1994). Listening to the world. Urbana, IL: National Council of Teachers of English. Glassner, A., Weinstock, M., & Neuman, Y. (2005). Pupils’ evaluation and generation of evidence and explanation in argumentation. British Journal of

    Educational Psychology, 75, 105–118. Halpern, D. F. (1998). Teaching critical thinking for transfer across domains. American Psychologist, 53, 449–455.

    Hill, J. H., & Mannheim, B. (1992). Language and world view. Annual Review of Anthropology, 21, 381–406. Hockett, C. E. (1954). Chinese versus English: an exploration of the Whorfian thesis. In H. Hoijer (Ed.), Language in culture (pp. 106–123). Chicago:
    University of Chicago Press. In’nami, Y., & Koizumi, R. (2011). Factor structure of the revised TOEIC test: a multiple-sample analysis. Language Testing, 29, 131–152. Ishikawa, S. (1995). Objective measurement of low-proficiency EFL narrative writing. Journal of Second Language Writing, 4, 51–69.

    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0005
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0010
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0010
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0010
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0010
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0010
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0010
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0010
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0010
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0010
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0015
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0020
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0025
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0025
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0025
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0025
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0025
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0025
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0025
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0030
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0030
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0030
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0030
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0030
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0030
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0030
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0035
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0035
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0035
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0035
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0035
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0035
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0035
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0035
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0035
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0040
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0045
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0050
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0055
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0060
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0065
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0070
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0075
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0080
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0085
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0090
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0095
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0100
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0105
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0110
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0115
    I

    J K K

    L L L

    M

    M

    M

    M

    N

    O

    P

    P

    P

    R R

    S S T T

    T T T

    T

    V

    W

    W

    E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 49

    takura, H., & Tsui, A. B. M. (2011). Evaluation in academic discourse: managing criticism in Japanese and English book reviews. Journal of Pragmatics, 43, 1366–1379.

    anuary, D., & Kako, E. (2007). Re-evaluating evidence for the linguistic relativity hypothesis: response to Boroditsky (2001). Cognition, 104, 417–426. ong, K. C. C. (2005). Linguistic resources as evaluators in English and Chinese research articles. Multilingua, 24, 275–308. u, K. Y. L. (2009). Assessing students’ critical thinking performance: urging for measurements using multi-response format. Thinking Skills and Creativity,

    4, 70–76. ee, K. S., & Carrasquillo, A. (2006). Korean college students in United States: perceptions of professors and students. College Student Journal, 40, 442–456. iu, L. G. (1985). Reasoning counterfactually in Chinese: are there any obstacles? Cognition, 21, 239–270. un, V. M. C., Fischer, R., & Ward, C. (2010). Exploring cultural differences in critical thinking: is it about my thinking style or the language I speak?

    Learning and Individual Differences, 20, 604–616. analo, E., & Uesaka, Y. (2012). Elucidating the mechanism of spontaneous diagram use in explanations: how cognitive processing of text and

    diagrammatic representations is influenced by individual and task-related factors. In P. Cox, P. Rodgers, & B. Plimmer (Eds.), Diagrammatic representation and inference. Lecture notes in artificial intelligence (LNAI) 7352 (pp. 35–50). Berlin, Heidelberg: Springer-Verlag.

    analo, E., & Uesaka, Y. (2014). Students’ spontaneous use of diagrams in written communication: understanding variations according to purpose and cognitive cost entailed. In T. Dwyer, H. Purchase, & A. Delaney (Eds.), Diagrammatic representation and inference. Lecture notes in artificial intelligence (LNAI) 8578 (pp. 78–92). Berlin, Heidelberg: Springer-Verlag.

    analo, E., Kusumi, T., Koyasu, M., Michita, Y., & Tanaka, Y. (2013). To what extent do culture-related factors influence university students’ critical thinking use. Thinking Skills and Creativity, 10, 121–132.

    analo, E., Kusumi, T., Koyasu, M., Michita, Y., & Tanaka, Y. (2015). Do students from different cultures think differently about critical and other thinking skills? In M. Davies, & R. Barnett (Eds.), Palgrave handbook of critical thinking in higher education (pp. 299–316). New York: Palgrave Macmillan.

    eisser, U., Boodoo, G., Bouchard, T. J., Jr., Boykin, A. W., Brody, N., Ceci, S. J., & Urbina, S. (1996). Intelligence: knowns and unknowns. American Psychologist, 51, 77–101.

    rtega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: a research synthesis of college-level L2 writing. Applied Linguistics, 24, 492–518.

    aton, M. (2005). Is critical analysis foreign to Chinese students? In E. Manalo, & G. Wong-Toi (Eds.), Communication skills in university education: the international dimension (pp. 1–11). Auckland: Pearson Education.

    erkins, K. (1987). Review of the ‘test of English for international communication’. In J. C. Alderson, K. J. Krahnke, & C. W. Stansfield (Eds.), Reviews of English language proficiency tests (pp. 81–83). Washington, DC: TESOL.

    owers, D. E., Kim, H.-J., & Weng, V. Z. (2008). The redesigned TOEIC (listening and reading) test: relations to test-taker perceptions of proficiency in English (ETS Research Rep. No. RR-08-56). New Jersey: ETS; Princeton.

    ichards, J., Platt, J., & Platt, H. (1992). Longman dictionary of language teaching and applied linguistics. Harlow, UK: Longman. obertson, M., Line, M., Jones, S., & Thomas, S. (2000). International students, learning environments and perceptions: a case study using the Delphi

    technique. Higher Education Research & Development, 19, 89–102. collon, R., & Wong-Scollon, S. (1991). Topic confusion in English-Asian discourse. World Englishes, 10, 113–125. kehan, P., & Foster, P. (1999). The influence of task structure and processing conditions on narrative retellings. Language Learning, 49, 93–120. akano, Y., & Noda, A. (1993). A temporary decline of thinking ability during foreign language processing. Journal of Cross-Cultural Psychology, 24, 445–462. akano, Y., & Noda, A. (1995). Interlanguage dissimilarity enhances the decline of thinking ability during foreign language processing. Language Learning,

    45, 657–681. akano, Y. (1989). Methodological problems in cross-cultural studies of linguistic relativity. Cognition, 31, 141–162. en Dam, G., & Volman, M. (2004). Critical thinking as a citizenship competence: teaching strategies. Learning and Instruction, 14, 359–379. homm, E., & Bromme, R. (2011). It should at least seem scientific! Textual features of scientificness and their impact on lay assessments of online

    information. Science Education, 96, 187–211. se, C.-S., & Altarriba, J. (2008). Evidence against linguistic relativity in Chinese and English: a case study of spatial and temporal metaphors. Journal of

    Cognition and Culture, 8, 335–357. orster, J., & Schuring, G. (1989). Language and thought: developmental perspectives on counterfactual conditionals. South African Journal of Psychology,

    19, 34–38. horf, B. L. (1956). The relation of habitual thought and behavior to language. In J. B. Carroll (Ed.), Language, thought, and reality: selected writings of

    Benjamin Lee Whorf (pp. 134–159). Cambridge, MA: MIT Press. olfe-Quintero, K., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing: measures of fluency, accuracy, and complexity. Technical report

    no. 17. Honolulu, H awai’i: University of Hawai’i, Second Language Teaching and Curriculum Center.

    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0120
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0125
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0130
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0135
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0140
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0145
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0150
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0155
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0160
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0165
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0170
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0175
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0180
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0185
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0190
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0195
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0200
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0205
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0210
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0215
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0220
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0225
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0230
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0235
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0240
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0245
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0250
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0255
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    http://refhub.elsevier.com/S1871-1871(16)30025-6/sbref0260
    How might language affect critical thinking performance?
    1 Introduction
    1.1 Overview of the present study
    2 Study 1
    2.1 Method
    2.1.1 Participants
    2.1.2 Materials and procedure
    2.1.3 Analyses
    2.2 Results
    2.3 Discussion
    3 Study 2
    3.1 Method
    3.1.1 Participants
    3.1.2 Materials, procedure, and analysis
    3.2 Results
    3.3 Discussion
    4 General discussion
    4.1 Conclusion
    List of funder and grant number
    Conflict of interest
    Acknowledgements
    References

    Order for this paper or request for a similar assignment by clicking order now below

    Order Now

    Do NOT follow this link or you will be banned from the site!