California Surveyors Forums

NCEES Launches Computer-Based Testing

See attached press release

Does anyone have knowledge of the format of the current LS exam? I have heard that it is all multiple choice and administered on a computer. Please tell me that this is not true!

Gary Winglovitz

It is true Gary. It's also being offered in April and October now.

Just curious, but why would the computer based exam & multiple choice questions be a bad idea?

ALL multiple guess tests have been dumbed down to make the questions fit the multiple guess format.
Plus the computer centers are worse than a TSA full search and the personnel are even more clueless.

My line of thinking is that if we want to be respected as a profession, one of the criteria is that we have similar threshold requirements as other professions. I don't necessarily know the answer to this but....are the licensing exams for Doctors, Lawyers, Architects, Engineers etc... computer based and multiple choice? We are already one of the few states that does not require a degree for licensure and it seems that this new exam might not be moving us in the right direction.

Gary Winglovitz
(951) 303-3062

Agreed, Gary. The new format is certainly convenient for NCEES.

The plus side of the current exam format. More testing sites, grading is timely. More than one test per year. the goal is to make it easier for the candidates to get to a testing site with a variety of test dates. Yes other professions are moving to CBT.

If the test has been "dumbed down", why didn't the pass rate jump up?

Having taken (and failed) both, I prefer the old style. I almost passed based on my knowledge and experience in the real world. The new style releys much more heavily on memorization and is therefore easier in my opinion. Let's not forget though that if applicants and their references are being honest the greatest hurdle to passing is qualifying to even take the test so the procedure still maintains some credibility and integrity. I fear that applicants and those references are not being completely forthright though which is allowing unqualified applicants to take and possibly pass the test by studying and memorizing the material rather than by real world experience.
Oops, my bad, I was alluding to the State specific exam. I did take and pass the National in the previous format and felt it was quite easy actually.

If the test has been "dumbed down", why didn't the pass rate jump up?

Look at what the schools are putting out.
A quote from Joesph Sobran
In 100 years we have gone from teaching Latin and Greek in high schools to teaching remedial english in college.

Multiple guess tests are a test against time, how many questions can you answer in XX time.
The old tests were a test of knowledge.

The old tests were given 2 times a year and could be done in one day.

That's a great quote!

The State specific under the old format has only been given once a year for the past 20 years. the Current CTB state specific is given twice a year.

My license is over 20 years old.

land butcher wrote:My license is over 20 years old.

mine too

I took CA test in 2010 for the first time and passed. My feelings are that the NCEES test was easy. The state specific test was well written and was a decent shot at what I believe to be minimum competency. I did not, however, find it to be especially difficult. If prepared it was easily passable. The most difficult test I have had to take was the LSIT.

The above being said, I have been taking other state exams lately. I must say, there is no way for a multiple guess exam to be near as comprehensive as a written one. Multiple guess exams are purely the product of funding. It has been very upsetting to me how much easier some of the out of state exams are from the CA exam I very much respected.

Honestly, as a surveyor without a 4 year degree, if our exam is moving /has moved to multiple guess, we need a 4 year degree requirement. A 4 hour multiple choice test does not even come close to examining minimum competency. At least with a degree requirement we would ensure each applicant can answer 4 years of multiple choice testing!

Honestly, as a surveyor without a 4 year degree, if our exam is moving /has moved to multiple guess, we need a 4 year degree requirement. A 4 hour multiple choice test does not even come close to examining minimum competency. At least with a degree requirement we would ensure each applicant can answer 4 years of multiple choice testing!

LOL

Nationaly that is direction things are going. CA is still a holdout at this point.

RAM wrote:Nationaly that is direction things are going. CA is still a holdout at this point.

The political power of academia.

That's one thing CA is doing right.

As stated above by others, CBT multiple guess is more cost effective, easier to grade, probably easier to defend (all subjectivity effectively eliminated), and is more convenient for the examinees.

But what is the purpose of examining applicants and what is the purpose of licensing? It is for the protection of the public. It is not for the edification of those already licensed or those seeking to become licensed. I know that the primary driving factor toward CBT was cost, with the ease and speed of grading also being high on the list.

BPELSG did seek input and buy-in from representatives among CA's licensed surveyors. When the proposal received a luke warm (at best) response from those it was presented to, BPELSG apparently decided that such buy-in was no longer important. The idea was going to move forward regardless.

These are the main problems that I have with an all multiple guess format:

1. I have never worked on a project in which not only were all the questions posed to me, but also where I was presented a short list of possible solutions from which to pick the best.

My experience, as I expect is as most of yours, is that not only is the surveyor expected to come up with an appropriate answer without it being provided to him beforehand, but also has the responsibility of determining what the questions to be answered for each project are as well.

2. Multiple guess format allows the not quite qualified a very good opportunity to guess their way into a license, and also provides the totally unqualified a chance to guess their way into licensure.

Statistically, it is not likely that a totally unqualified person would be able to guess their way to a license on one exam, but statistically, winning more than the amount you spent on a lottery ticket is not very likely either yet every week, thousands of people win anywhere from enough for a nice dinner for two on up to several million.

Likewise, statistically, many who are not quite qualified will be issued a license each year, and a few, or perhaps more than a few who are totally unqualified will be issued licenses over the course of several years. And the more times an unqualified person takes the exam, even if their knowledge does not increase, their statistical probability of passing increases.

Maybe we should refer to it as the "Lottery Format" rather than by some cool sounding acronym (CBT) or even the cynical "multiple guess".

3. Multiple guess, in it's current form, does not provide an effective means of testing one's reasoning abilities. Correct answers to a series of good related questions may require a thread of good logic based on knowledge and experience, but it is also possible to get through with a little knowledge and a general ability to use deductive reasoning, especially if one has the ability to go back and change an answer of a previous question. It is difficult to have succesive multiple guess questions of the same subject matter (i.e. one boundary survey scenario providing the basis for several questions) in which successive questions and the provided answer options won't clue in, or lead a moderatley astute examinee into the correct answers. It is also possible to choose enough correct answers to pass on plain dumb luck (Woo Hoo! I got 4 outta 5 numbers and the Mega number!)

I have been given several assurances that a multiple guess, or Lottery Format exam, using interactive technology, which provides subsequent questions based upon the examinees previous answers, can effectively test an examinee's reasoning abilities.

I concede that a multiple guess format using such interactive technology would more effectively test for threads of reasoning, and would be a great format... for a significant portion of the exam. The ability for someone to guess their way to a license would likely be greatly reduced, but not eliminated by using an interactive multiple guess exam format.

The main problem there is that the exam is probably many years away from an effective interactive format for the LS exam. How many not-quite qualified and totally unqualified people will receive licenses in the meantime because of a cost effective exam (from an in-agency view which is a short term view when considering the potential harm to the public), but an inadequate one to ensure minimal competence for the protection of the public.

4. Multiple guess provides no way for the candidate to demonstrate an ability to effectively communicate. A professional's ability to arrive at a correct answer, whether by reasoning based on knowledge or by guessing one's way there, is useless if that professional cannot communicate the answer, it's significance, and the reasoning behind it to those outside of the profession who require the answer.


There is already a definite problem of licensees signing for candidates who are not qualified by knowledge, and in many cases, probably not fully qualified by type and amount of experience. In many cases, the licensee who signs for such a person may feel that the person has put in their 6+ years, so they deserve their shot - and besides, if their really not qualified, the exam will weed them out, right?

California already has one of the lowest statutory requirements to sit for the exam in the US. Licensees signing for the unqualified merely compounds an already deficient standard.

The exam may not weed them out. A multiple guess exam of questions of otherwise equal difficulty to those of essay, short answer, or even fill in the blank format are inherently easier. But prior to that, with the old format, we saw each year an exam of reasonable difficulty to measure minimal competence routinely end up with a cut score somewhere near 50% in order to result in a pass rate in the 20s or 30s. At any level of school, anything below 70% was unsatisfactory and anything below 60% was a failure. For the years that I was either involved with development or grading, I saw no reason why the cut score should have not been 70%. If the pass rate was alarmingly low at that cut score, that may reflect the exam, or it may reflect the preparedness of the overall population of examinees. I think that the release of the 2005 exam should have put that argument to bed. We've been lowering the bar into the failure range in order to boost the numbers.

Not only is this a failure to adequately protect the general public from incompetent practice, it is also very unfair to the new licensees who should not have yet received a license. In issuing licenses to these people, the State has told them that they are adequately competent to practice without the supervision or guidance of a more experienced licensee. Unless the examinee had th eopportunity to review the portions they missed in a grading appeal (an opportunity which is no longer available under the new format), these new licensees have no way of knowing what areas of practice they are clearly not competent in, which they are clearly competent in, and which they are hovering right near minial competence in. If passing in with a score in the 50s or 60s, they will certainly be under the impression that they are competent in areas where they are clearly not competent, or perhaps only competent as long as the questions requiring answers in a project are relatively simple.

These people will be completely shocked when a disgruntled client or a concerned licensee files a complaint with BPELSG. What are they to think when they are cited for doing exactly in practice as they answered on the exam they passed?

I understand some of th edriving force behind all of this, that bigger numbers of surveyors is good for the profession and that waning numbers can cause surveyors to lose important levels of influence at the Board, in the legislature, and in other important arenas. But I think the focus just on the numbers is as shortsighted as is focusing purely on deed dimensions while ignoring more pertinent evidence is in a survey.

Growing the population of surveyors with those unprepared to practice independently will eventually weaken the profession as related professions and the public at large begin to recognize that the knowledge surveyors have is no longer all that specialized, that perhaps a civil engineer is adequately trained to do the job. If our qualifications are to be so low, what's the point of having a license? Why not a simple technical certification that anyone with high school trigonometry and a 6 month course at the community college can take?

Nope. We need the license. But we also need to ensure adequate minimal qualifications and adequate testing to ensure a minimal level of competence that reflects a professional level of knowledge as a reasonable basis for protection of the public.

Why are we licensed?

Evan brings up two good points, comparing passing a multiple guess test to the lottery and "Why are we licensed".

Multiple guess tests vs. the lottery
On any given question the odds are 1 in 4 or 5 of guessing the correct answer and usually one or two are such that they can be eliminated leaving the examinee with a 1 in 2 or 3 chance of guessing right. I heard of an examinee that took a multiple guess test and was able to answer about 50% of the questions, the rest he picked one of the answers (A,B,C,orD) and used that one choice for all the questions he didn't know. He passed. I have a daughter that when in grade school could read about a subject and absorb the material like a sponge. When required to write about a subject, whether a paragraph or 3 pages, what she wrote was interesting and held the readers attention. I would bet 5 figures that she could read 2 or 3 survey books and pass the multiple guess LS exams. Does that mean she is a competent enough surveyor to hold a license - no way.

Why are we licensed?
Good question. Every license is government issued. Every license requires a government issued test that must be passed to receive that license. Almost every license requires some course of study to learn the material to pass that test. All three of the above require money and, with the exception of private schools, that money all ends up in government coffers.
My Father, living in a major eastern city, bought his first car in ~1920. No driver's license required, the salesman showed him how the car worked and they went around the block a few times and he was on his way. His first driver's license was a "Chauffeurs" license so he could drive city buses.
"As automobile-related fatalities soared in North America, public outcry provoked legislators to begin studying the French and German statutes as models.[5] On August 1, 1910, North America's first licensing law for motor vehicles went into effect in the U.S. state of New York, though it initially applied only to professional chauffeurs.[6] In July 1913, the state of New Jersey became the first to require all drivers to pass a mandatory examination before receiving a license."
Have automobile related fatalities gone down due to licensing or due to vehicles designed for better passenger safety?
I find the above quite interesting as to how licensing of professionals evolved into licensing the general public. I would like to know how this came about as the Motor Vehicle code is vague on this issue. On the same subject, the agricultural inspection stations at the California borders are authorized under the "Business and Professions Code" which applies only to those involved in the commercial business of agricultural products. So, where did they determine they can inspect private vehicles?

How many pilots in the barnstorming days had licenses? The first pilot's license was not issued until 1927.

Did G. Washington or A. Lincoln hold Land Surveyor licenses?

Government requires licenses to "protect the public". Is that really the reason?

As stated before, to receive a license you pay academia, a government testing fee and a periodic government fee, and in many States academia again, to keep that license. Anyone see a trend here?

As for protecting the public, just how is this accomplished? Does anyone from government go out and check your field work? A 1ft miss-set rear corner in a high dollar R1 area cost a local surveyor his business. Is government capable of checking our work? Builders are forced to pay government agencies for building inspection fees. What were the building inspectors doing when hundreds of condos were constructed without firelocks in the walls and lack of proper fire protection between units? So after paying thousands of dollars to the government for public safety inspection these builders paid out millions to settle construction defect lawsuits. I once asked a construction defect attorney why the local agency was not named in the lawsuits, his reply "government agencies are immune".

So besides revenue what else does licensing do? It provides the government with a lot of your personal information; DOB, SSN, currant address, mothers maiden name. This information is used to track a licensee should the government think they have a reason to do so. Think NSA and Edward Snowden here. When our personal information was on paper and microfiche it was pretty much accessible only by those authorized to do so. Now, as those who shopped at Target, Neiman-Marcus and other stores now know, a 17 year old in the Ukraine can access your "secured" computer stored data. With your name, DOB and SSN how hard would it be for someone to locate your birth certificate and get your parents info and track down your mothers maiden name?

I think it's time to rethink the whole licensing thing. Our personal information is no longer secure and does the level of public safety provided, if any, justify providing that information?

It would be interesting to see the cut scores of the new state exam. Under the old system, as an LS Review instructor told me, all you have to do is get an 'F' and you get a license. Since the pass rate has gone down, perhaps the cut score has risen.

Full disclosure: I'm a multiple guess LS.

I sit here reading these replies as someone who has been holding off taking the LS for a couple of years. I did not hold out due to thinking the test would be difficult to pass, instead I felt the responsibility of my own "stamp" was still too much for my skill level. I do not see how having a complete multiple guess test will help protect the public or even remotely demonstrate that one is competent to be trusted as a Land Surveyor.

The point seems to be well made here that academia and cost play a great role in the decision to "standardize" the test. However without the capacity to have a professional overlook the work performed in the test, is it possible to see what actual work or what the tested individual is capable of? The public at large already seems to think we are only as good as a GIS map until they actually us. Do we really need to cut costs so bad that we put the profession at risk?

I learned pretty quick in engineering school that knowledge of the subject I was being tested on often had little to do with passing my exams. I had to learn each professor's testing style, I had to learn how to take the exam. The PE and LS exams were no different, nor was the EIT.

I remember one professor, he was of course department head of my emphasis, and I had to take three of his classes. I could not get better than a C in his classes. But my pal, who struggled through every other class during his entire college career aced this particular professor's class each time, he got an A in each of the three classes we took together. He knew how to take the test.

Remember, each candidate needs four references. If we take this mentoring and reference providing portion seriously, then maybe we wouldn't even need a licensing exam.

When asked my opinions on the multiple choice exam format for the state specific portion of the exam I stated that I felt that this type of test did not adequately evaluate the examinee.

I suggested that if a MP test was implemented then I thought that grading the exam with 2 points for a correct answer, 0 points for no answer, and -1 (or -2) for an incorrect answer would minimize guessing. This thought was met with derision and rejected out of hand.

I also think that under my format that a candidate that doesn't answer a certain amount would be disqualified. Apparently I like shouting at the rain.

My personal opinion is that the exam should be a written exam showing thought processes and followed with a "practical" component. I.e. personal interview with real world problems being discussed.

I've heard the suggestion that the exam should be given on the hood of a truck, with wind blowing, scrapers buzzing by, and a proctor acting as a job super coming by every few minutes to shout at you that some stakes were wrong, that a pipe crew is waiting and there's going to be a backcharge, and to just peer over the examinee's shoulder from time to time.

I think there's validity to that idea.


Tom, my experience was similar when asked for input on the all multi-guess (lottery) format. The group of surveyors gathered were told that their input was very important. One in the group was quite entusiastic, a couple were cautiously open to more info, and two didn't like the idea at all. The overall consensus was (with me and the enthusiastic guy as the outliers) to refine the idea with some suggestions (I think a grading scheme penalizing for incorrect answers was among them) and present it to us again. We were told that since our input was so important, a follow up meeting would occur a couple months later. That was in early 2010. I checked with all the other guys in 2011 or 2012. None of us were contacted about the exam format again.

Great idea for a exam test. THAT'S real world surveying. Compound that with the plans being in inches.

SOP in govt. If you make a ripple in their pond expect to be shunned. Govt does not want free thinking it only wants agreement