Second Language Assessment and Technologies (1)
Traditional learning theories
How well the student can master small units of information and provide evidence of this learning on objective and decontextualised tests.
Constructivist views of learning
Emphasis not just the product of learning, but the importance of cognitive processing by the learner and the influence of affective, social, and cultural factors of learning:
What students can do in an intellectually supportive environment, not what they might score on a test taken after the solution to a problem is being memorized
To solve problems
CALL course?
How would you be assessed? How will you be assessed?
Traditional learning theories
Constructivist views of learning
Be tested whether you know (through memorization) how to use technology in L2.
Prove you know how to use technology in L2:
1. How to research your own CALL learning background to establish where you are, see the positive and negatives and take it from there. Whether you used New technologies for your own English learning.
2. How to research the teaching setting: school CALL profile
3. How to design and create a CALL environment where you can store a lot of your CALL good-idea material (website)
4. Have a pool of annotated Internet sources related to the different aspects of L2 at various levels for future immediate use.
5. Be able to evaluate Internet and CD ROM resources.
6. Keep truck of your own CALL learning in a reflective way: learn how to learn: reflective Journal
7. Learn IN DETAILS in details how two CALL software can be used in L2
8. Compare two L2 Testing software
9. Self Evaluation: Assess what you have learnt.
From what you have learnt so far, how could you assess your L2 students using technologies?
Traditional learning theories
Constructivist views of learning
Test them whether they know (through memorization) have mastered small units of information and provide evidence of this learning on objective and decontextualised tests.
Students prove they know how to use technology in L2:
1. How to research their own L2 learning background to establish where they are, see the positive and negatives and take it from there. Whether they used New Technologies for their own English learning.
2. How to design and create their own L2 electronic environment where they can store a lot of their L2 CALL learning material (website) and work, for example a personal website, power point presentations (my family, my pet, my friend, my house, etc) store their audio files, etc. E-Porfolio
3. Have a pool of Internet sources related to the different aspects of L2 learning at various levels for their own use.
4. Be able to use Internet and CD ROM resources for L2 learning.
5. Keep truck of their own CALL learning in a reflective way: learn how to learn L2 and learn how to learn L2 through the use of technology: reflective Journal
6. Learn in details how to use CALL software to enhance their L2 learning
7. Self Evaluation: Assess what they learn from the tasks they accomplish in L2 learning, using new technologies. For example, find authentic material on the Internet and complete tasks: traveling, cooking, music, going to the cinema or the theatre, finding information about social issues (drugs, teenage driving); chat with other people in the target language, collaborate, etc.
Current assessment movements reverse the emphasis on testing over learning.
ALTERNATIVE ASSESSMENT
Knapp and Glenn (1996): “assessment is alternative when the tasks used in testing are equal or similar to the best tasks found in instruction” and when students are engaged in ‘using the knowledge and skills to solve the kinds of problems and do the kinds of things students will face in the world outside school”
AUTHENTIC ASSESSMENT
“Assessment that both mirrors and measures students’ performance in ‘real’life’tasks and situations”
PERFORMANCE-BASED ASSESSMENT: synonym to those two above
GENERATE rather than choose a response

HOW CAN TECHNOLGOY SUPPORT ASSESSMENT?
Computer Based Testing Tools Review

TASK

AIMS
(a) Learn how two electronic testing tools work (through theory, hands-on practice and collaborative comparison)
(b) Collaborate using electronic tools such as MSN, email, chatting and conferencing.
(c) Use the target language in all four skills.

(A) REVIEW
Review the Computer Based Assessment (CBA) tool assigned to you, based on the review guidelines given below entitled Computer Based Assessment Review.

Work with your assigned partners and compare tools.

Group A:
Marios: QuestionMark http://www.questionmark.com/us/home.htmGeorgia: NEPTON http://nepton.engine.intercol.edu/
Maria: TOEFL www.ets.org/toefl/ [[http://toeflpractice.ets.orgEleni|http://toeflpractice.ets.org]]
Eleni: Quick Placement Test **http://www.oup.com/elt/catalogue/isbn/7162?cc=gb****
**http://www.digibuy.com/cgi-bin/product.html?98255077435**
Sally: NEPTON http://nepton.engine.intercol.edu/

Group B:
Vassos: QuestionMark http://www.questionmark.com/us/home.htm
Elena: TOEFL www.ets.org/toefl/ http://toeflpractice.ets.org
Andria Quick Placement Test http://www.oup.com/elt/catalogue/isbn/7162?cc=gb
http://www.digibuy.com/cgi-bin/product.html?98255077435
Beria: TOEIC: http://www.english-test.net/toeic/

Group C:
Andreas: TOEFL www.ets.org/toefl/ http://toeflpractice.ets.org
Elena:
TOEIC http://www.english-test.net/toeic/
Tom:
QuestionMark **http://www.questionmark.com/us/home.htm****


ESkill: http://www.eskill.com/english-language-testing.htm

Any other you would like to suggest. Please consult your lecturer first.


Computer Based Assessment Review - Guidelines

1. Test Identity: Title, Publisher, Provider (Institutional CBA developers / providers or commercial CBA providers), Author, Subject area (generic / specific, for example only for English), Electronic address.
2. Test overview
3. What is it?
4. Why take it?
5. What does it assess / measure?
6. Who should take it?
7. Where can one take it?
8. Who accepts it?
9. Test content
10. Duration
11. Types of questions
12. Sample activities
13. Demo available
14. Opinion


Work cooperatively using any of the following collaborative tools: MSN, email, chatting, conferencing, Skype, Wiki Discussion. Keep records of your electronic cooperation (for example save your emails, MSN, chat or conference notes) and attach them to your review as an appendix.)

Together with your partners, write a review of the tools. This review should include the following:
(a) Task Cover page
(b) Introduction to Assessment and testing in general and to assessment and testing using technologies in particular. (all collaborators, use your CALL textbook Butler-Pascoe M.E. and Wiburg K.(2003) M Technology and Teaching English Language Learners, Pearson Education, chapter 8, the notes you took in class during the lecture and any other written or electronic sources)
(c) Review of Tool 1 (one collaborator)
(d) Review of Tool 2 (one collaborator)
(e) Review of Tool 3 (one collaborator)
(f) Review of Tool 4 (one collaborator)
(g) Comparison of the three / four tools (all collaborators)
(h) Conclusions (all collaborators)
(i) References (all collaborators)
(j) Appendix (emails, chat or conference collaboration notes) (all collaborators)

IMPORTANT NOTE: include this review in your electronic portfolio, page Collaborative Evaluative Review of a Testing Tool.

WORK TIMELINE:
Two class session
Home time

Books

Alderson, J.C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge: Cambridge University Press.
Bachman, L.F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Bachman, L.F., & Palmer, A.S. (1996). Language testing in practice. Oxford: Oxford University Press.
Banerjee, J., Clapham, C., Clapham, P., & Wall, D. (1999.) Language testing update: ILTA language testing bibliography 1990-1999. (1st ed.). Lancaster (UK): The International Language Testing Association (ILTA).
Brown, J.D. (1996). Testing in language programs. Glenview, IL: Addison-Wesley Longman.
Clapham, C.M. & Corson, D. (Eds.). (1998). Language testing and assessment: Encyclopedia of language and education. (Vol. 7). Dordrecht, Netherlands: Kluwer.
Genesee, F., & Upshur, J.A. (1996). Classroom-based evaluation in second language education. Cambridge: Cambridge University Press.
Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press.
McNamara, T. (2000). Language testing. Oxford Introductions to Language Series. Oxford: Oxford University Press.
Spolsky, B. (Ed.) (1999). Concise encyclopedia of educational linguistics. New York: Elsevier. A compilation of 232 articles dealing with language and education topics, this encyclopedia includes eight articles about language testing by experts in the field.

Web Sites
Organizations

The International Language Testing Association (ILTA) is an independent international association of assessment professionals. ILTA seeks to promote the improvement of language testing throughout the world through workshops, conferences, publications, and professional services.
The American Council on the Teaching of Foreign Languages (ACTFL), a national professional association for foreign language teachers, has been instrumental in the development of proficiency assessment. The ACTFL Proficiency Guidelines for Speaking have recently been updated and can be downloaded at this site.
The Association of Language Testers in Europe (ALTE) provides definitions of proficiency levels as well as descriptions of tests developed by participating organizations.
The Center for Applied Linguistics (CAL) offers tests such as the Simulated Oral Proficiency Interview (SOPI), the Basic English Skills Test (BEST), and the Student Oral Proficiency Assessment (SOPA), and provides an online Foreign Language Assessment Resource Guide.
The Center for Equity and Excellence in Education Test Database, maintained by the George Washington University and ERIC/AE, includes almost 200 tests for use with limited English proficient students.
The Educational Testing Service (ETS) offers information about the Test of English as a Foreign Language as an online magazine.
The ERIC Clearinghouse on Assessment and Evaluation (ERIC/AE) offers a number of resources, including an assessment library and a test locator.
Collections
The Resources in Language Testing WWW Page includes an extensive list of links to and reviews of sites related to language testing, as well as a searchable database of articles from the journal Language Testing.
The University of Surrey: Dissertations in Language Testing page is a list of dissertations on language testing and assessment completed at the university since 1994.
The Language Testing Page of the National Foreign Language Resource Center (NFLRC) at the University of Hawaii offers materials relating to the assessment of less commonly taught languages, as well as the assessment of cross-cultural pragmatics.
The Foreign Language Test Database from the National Capital Language Resource Center (NCLRC) offers an online searchable database of tests that measure foreign language proficiency.
The online Foreign Language Assessment Resource Guide is part of an ongoing performance initiative of the Center for Applied Linguistics and the National K-12 Foreign Language Resource Center. This searchable database includes descriptions of language assessments that are currently being used in elementary, middle, and secondary school foreign language programs around the country.
The Cutting Edge CALL page has several demos that show what can be done with computer-designed tests.
The Language Testing page of Drs. Kenji Kitao and S. Kathleen Kitao has links to resources and articles on the web.
Research into Language Testing at the University of Duisburg is still under construction, but contains information about C-tests as well as an extensive bibliography

Proficiency Guidelines and Scales
The ACTFL Proficiency Guidelines for Speaking have recently been updated and can be downloaded from The American Council on the Teaching of Foreign Languages (ACTFL).
The Interagency Language Roundtable Scale grew out of the Foreign Service Institute Rating scale. This site includes a brief history of the scale, an annotated bibliography, and descriptions of the different levels.
The International Second Language Proficiency Ratings (ISLPR) (formerly the Australian Second Language Proficiency Ratings) site has descriptions of the proficiency levels along with information on registering for an ISLPR test (available only in Australia) and some references.

Other
TESOL & APPLIED LINGUISTICS: a) PROFESSIONAL ASSOCIATIONS
http://englishstudydirect.com/OSAC/TESOLprofass.htm

Resources in Language Testing page
http://www.le.ac.uk/education/testing/ltr.html

Software

http://caacentre.lboro.ac.uk/resources/web/onlres4.shtml

The following are names of institutional and commercial software developers and providers. Click on the links to find out more:

Institutional CBA developers/providers:
CASTLE (Computer ASsisted Teaching and LEarning) - An excellent place to begin with CAA for formative assessment, particularly if you are looking for free materials. CASTLE is a JISC/JTAP funded project which provides an authoring shell for the delivery of web-based MCQs. Users need only minimal IT skills ( i.e. being able to use a browser) and need not know any other CGI, HTML or other scripting or markup language.
Clyde Virtual University Assessment Engine - CVU offers free tools, which enable the delivery of online assessments and evaluation forms. The assessment engine allows users to design web-based tests and the assessment system displays tests to users, marks the responses, and offers immediate feedback. Question types available include multiple choice, multiple response, free text entry and list ordering.
TRIADS - (The University of Derby/Tripartite Interactive Assessment Delivery System) is a toolkit for users of Authorware Professional to develop CAA using a wide range of sophisticated question types. The site contains an impressive demonstration of TRIADS materials.
WWWAssign - A free tool for the delivery of web-based assessments. WebAssign is developed by Larry Martin of North Park University, Illinois.
WebTest - WebTest (Heriot Watt University) is a system for creating, delivering and marking web-based assessments. Features include the randomisation of questions, ability to display maths and scientific formulae and the creation of diagrams and graphs on the fly.
EXAMINE - enables users to develop CBA for delivery with Windows. A range of question types is available, including those which incorporate graphics and multimedia. The software, which is free to HEIs, was developed under the Information Training Technology Intitiative (ITTI), which has now ended.
WEBTEST at University of Waterloo a free web-based assessment system.

Commercial CBA providers:
QuestionMark - The most widely used commercial provider of CAA software in UK HE. Visit the site for a demonstration of QuestionMark Designer (Windows) and QuestionMark Perception (web-based).
WinAsks Professional - Smartlite Software - Software for the development of secure tests and questionnaires delivered using Windows 3.1 or higher.
WebMCQ - WebMCQ is a assessment service based in Australia, which hosts and maintains web-based tests.
EQL - Interactive Assessor - A system for creating question banks, generating customised tests, running tests and providing automatic results analysis.
Hot Potatoes Web Authoring Tools - The Hot Potatoes suite is a set of six authoring tools, created by the Research and Development team at the University of Victoria Humanities Computing and Media Centre.
CyberExam - CyberExam was originally constructed by Virtual Learning Technologies as an enabling software tool that allows schools, universities, and corporations to create on-line testing instruments that will automatically grade and analyse results.
Quiz Factory 2 - Quiz Factory 2 has been developed by the LearningWare Inc. Quiz Factory 2's powerful, easy-to-use interface allows tutors to quickly design and save a quiz with sounds, movies, and present formative assessment.
Test Pilot - Test Pilot has been designed to provide for the easy creation and deployment on online formative assessment using the latest Internet technologies. It has been vigorously time tested in the most demanding of corporate education, national university, primary and secondary educational environments.

Links to Commercial OMR Provider Websites:
DRS LTD
Speedwell
NCS
Trax UK