tlhIngan-Hol Archive: Sun Mar 30 17:25:11 2014
[Date Prev][Date Next][Thread Prev][Thread Next]
Re: [Tlhingan-hol] Certification Test Woes
<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml"; xmlns="http://www.w3.org/TR/REC-html40";><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-CA link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I second most of Elizabeth’s comments and want to apologize for the subject line of my e-mail possibly implying that there is a problem in the tests themselves. I know a lot of work went into them by the right people and I am proud of the level of commitment that they show of our bronze pach holders. Thank you d’Armond for writing and supporting the test. I’m not sure if it was in list e-mail or privately that I said recently that while at first I wanted to see the test made easier, when I looked at the requirements I changed my mind, and now simply want to see beginners better prepared so that more of them take it and more takers pass it.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>While the suffix number question has been unexpected and tripped people up in the past, it’s <b>not that hard to teach to. I’m sure I won’t be the first person to tell people this:</b><br>Remember the sample sentence in TKD, “Due to your apparent minor errors”? Write it like this:<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>QaghHommeyHeylIjmo’<br> 1 2 3 4 5<br><br>Now you know the numbers for all the noun suffixes. Similar sentences (just one would be kind of ridiculous) can be constructed to fix the sequence for the verb suffixes, and what was once a stumbling block becomes a free point.<br><br><o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I would like to see simpler levels to reward the achievement of people who are above the pointing and grunting level without the level required for the claws. I always worry that the people who come to qep’a’ knowing nothing and work really hard for three days, making amazing progress, go away not knowing the extent of their achievement because they “failed the lowest test.” I’m going to make sure that those who want to and have advance preparation time can pass the test, and see that it is made clear at qep’a’ that while this is the lowest level test, it represents a significant understanding, and not one that can be gained in a long weekend. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I’ve never heard anyone really complain about the tests, just watched them be disappointed, and had them admit they were blindsided by the suffix number question. I would have to stop and count on my fingers to answer them, myself, but I wouldn’t argue that they had no place on the tests.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>- Qov<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> d'Armond Speers, Ph.D. [mailto:[email protected]] <br><b>Sent:</b> March 30, 2014 15:48<br><b>To:</b> tlhIngan-Hol List<br><b>Subject:</b> Re: [Tlhingan-hol] Certification Test Woes<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><div><div><div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>I have tried to be consistent, fair and permissive in grading of KLCP tests. The goal is not to make someone discouraged by grading hyper-critically, and to allow room for creativity (language is not a 1:1 thing).<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>For each question in the test bank there is an explanation of the purpose of the question, what is being tested by the question. These are printed in the answer keys with the expected answer(s), which the proctor uses when grading. When I'm grading tests, I am specifically looking for mistakes that pertain to the purpose of the question. If someone answers the question correctly but is not strictly adhering to the vocabulary or grammar of their test level, I do not count that as an error. (I would not count off if someone used {-'e'} in an answer in the level 1 test.) If there are vocabulary or grammatical errors that do not pertain directly to the purpose of the question I count off a single point rather than marking the whole question wrong. If they completely miss the point of the question, or the number of errors exceeds the available points per question (5 points per question for levels 1 & 2), then I just count the question wrong. (If someone is making that many errors per question, then they really aren't at that level yet.)</span><o:p></o:p></p><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Whenever I distribute tests to someone else to administer and grade, I provide these instructions. With multiple test administrators over a period of many years, it's easy to see how these guidelines may not be followed consistently. I have not personally attended a qep'a' or qepHom in several years, or administered tests, though I do still regularly generate new tests from the test banks for qep'a'mey and qepHommey. It sounds like we may need to get some written guidelines for test administrators and strive for greater consistency in grading.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Creation of the KLCP was a group effort. I designed the structure with Lawrence (3 levels with 3 pins, because of the cultural importance of the numeral 3). There are 100 questions for each level in the test bank (plus the reading comprehension questions for Level 3), and generating tests means selecting questions from the test bank at random. Level 1 is a sub-set of TKD; level 2 is all of TKD; and level 3 is open-season on all available materials (mainly including additional materials from KGT). We produced written guidelines for each level, describing the vocabulary and grammar that was in scope. Questions were written to be distributed evenly across the topics for each level. Each question in the test bank was reviewed by multiple highly-skilled speakers when we were creating the program, following the written guidelines for the level. I have all of this in a MS Access database, which allows me to manage the randomization and generation of tests and keys quickly and easily. We put a ton of thought and effort into it, to make it as fair, correct and relevant as we could. And ultimately, it is intended to encourage people to achieve certification; it is supposed to be a positive experience.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>All of that being said, we now have the benefit of many years of experience, which we didn't have when we were creating the program. And if it's not achieving its goals of encouraging learning and rewarding progress, then we should be open to that feedback and willing to make adjustments. The challenge is that the more significant the adjustments (severely limiting the vocabulary for level 1, for example), the more effort will be involved in re-developing the test bank. If we make Level 1 too simple, then the gap between Level 1 and Level 2 becomes huge. And to be honest, I would not be able to do this re-development work, so someone else would have to take it over completely. Taking out the questions about the suffix number would be easier, and I agree that it's probably too pedantic for the level 1 test. These questions would have to be replaced with something else, maintaining balance across the topics for the level. I'm not sure how many questions there are like this in Level 1, but if anyone has interest in writing some new replacement questions, I'd be happy to work with them on improving the test bank.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Another thing to consider is that if we simplify level 1, then people who worked hard to accomplish that achievement might feel slighted because we lowered the bar. Not sure if that's a real thing, it just occurred to me.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Sorry for the wall of text, just lots ideas bouncing around.<o:p></o:p></span></p></div></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>-- Holtej<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div></div></div></div></body></html>_______________________________________________
Tlhingan-hol mailing list
[email protected]
http://mail.kli.org/mailman/listinfo/tlhingan-hol
![[KLI Member]](/pics/KLImemberbutton.gif "[KLI Member]"){kind=small}
![[Hol po'wI']](/pics/level3_big.png "[Hol po'wI']"){kind=small}