land butcher wrote:With the proliferation of these top secret multiple guess exams, does our board ever review the exams for vaguely worded questions or questions that are constantly answered wrong, probably because the question and/or answers are just wrong?
Yes, at least with the exam in the previous format, if a question "performs poorly", that is a significant majority of examinees answers it wrong according to the test plan, it is usually excluded from the scoring (this happens in Standards setting).
The question is still in the pool for use on future exams. But (again, in the old format), the question will undergo some amount of rewriting anyway so as not to be identical to a question on a previous exam. If the question performed poorly, that info is available to the exam development team for the upcoming exam. If they still wanted to use the basic question, they would look at it to determine if it was worded poorly or if there was a problem on the grading plan. Usually it is because the question was worded ambiguously. Occasionally, the grading plan may have not included an answer that may not have been the best answer, but was correct nonetheless. This can happen when a question addresses matters which can be dependent upon differences in local jurisdictions. The grading teams normally have LSs from a wide variety of jurisdictions, so if there are one or two that are outside of the norm, there is usually one or more LSs on the grading team who can attest to that and adjust the grading plan accordingly. Since the grading of the new format is totally different, I don't know how that circumstance would be handled.
As to the calculator thing, I don't understand the prohibition against taking in extra batteries, but 2 calculators would seem to be a reasonable rule to follow. If you forgot to put fresh batteries in your calculators before the exam, you probably deserve to fail. If both calculators die of some non-battery electronic version of a stroke, I guess it just wasn't meant to be for you this year. Bummer.
IMO, the LSIT/EIT is where the candidate should have to prove their calculating abilities. The professional level exams should be testing professional level skills and knowledge. The upper level exams should be testing analysis, management, decision making ability, communication skills, etc. Calculating, and thus the need for a calculator should be minimized on the 2nd division exams. Many of the exam development team members I worked with in the past attempted to develop questions with this philosophy in mind. One year we even developed a complete construction problem that had only calculations that were simple enough that one should have been able to quickly do them in one's head, even under exam stress. The rest were process and decision questions.
Obviously, if the problem is involving something like SPCs, it would be mostly about calculations. But for most other topics, calculations could and should be minimized on the LS, and accordingly be more robust on the LSIT.
The CA exam has a reputation as being the toughest, or one of the toughest among the states. I don't think that reputation is or was accurate. When I took the exam in 1996, when I worked on exam development and grading from 2006 to 2010, and of the past exams which have been released by BPELS, none have questions that are unreasonably difficult.
The reputation, I think comes as a result of 2 factors. 1, the amount of content on the exam is such that a person near the minimally competent level is going to use every bit of time available to fully complete the exam. Some may barely complete it, some may not quite complete it. A few may have time to review their answers.
If one is unaware that most years the cut score ends up somewhere near or a little above 50% and that there have been no must-pass portions, then one probably has the idea that they must complete all portions of the exam to have a hope of passing, and that the cut score will be somewhere around 70% (that was my assumption when I took it). To complete the CA exam (in it's previous format) and have a high degree of confidence that a high percentage of your answers are correct after rushing through them and without time to have checked them is a very difficult task.
2nd, the minimal qualifications to take the exam in CA are among the lowest in the nation. Most states require a combination of education and experience of 8 years or more. Some states specify minimum amounts of experience in certain types of surveying. Thus, the candidate whose 6 years is 80% construction staking would not qualify.
CA also lacks a 3-strikes rule that many other states have. The 3-strikes rule is if you fail the exam 3 times, you must re-apply to take it and show that you have gained additional training and experience in order to be re-accepted. The low entrance bar and the lack of a mechanism to weed out those who perrenially fail the exam pulls down the performance of the overall candidate pool. It also necessitates that the cut score be lowered in order to pass an "acceptable" percentage of the candidates.
The high amount of underqualified and/or underprepared examinees who inevitably fail can either admit that they were underqualified or underprepared, or they can beach about how difficult, and probably how unfair the exam is.
If CA had minimum qualifications that were on par with the average of other states, and had a mechanism to weed out those who have proven that they are not yet ready by failing 3 or more times, I believe that the performance of the overall candidate pool would be at least on par with that of other states, and perhaps better given the training opportunities that now abound through distance learning and CLSA Chapter review courses.
But, it ain't likely to happen in the foreseeable future.
