Exam Opinion/Rant/Musings...

[E_Page]

The header is sure to get Ric's attention. Don't worry, Ric, no confidentiality violations forthcoming.

This is a spinoff based on the last few postings of the TX Board thread. Jason Camit asked about relative exam difficulty between states. I started with that, and much like the nature of the TX thread, started meandering about on the topic. Since it really has nothing to do with the TX Board, or possible threats to the continued existence of our own, I decided to cut this out of that one and start this one.

These are just my ramblings and not likely to ever go beyond that. Agree or disagree, I'd be interested in your thoughts.




This is also outside of your (Jason's) 5 year window, but I took the OR exam in 95. The state specific portion was two hours long, choose and answer 4 out of 7 essay questions.

Somewhat similar to what Tom described as the analytical portion of the TX exam, you had to evaluate the given information, come to a conclusion, and explain your reasoning. Subjective but truly a measure of one's professional abilities as opposed to one's technical abilities. Contrary to what some may wish to believe, effective communication is a professional ability and a valid area of testing.

With the ever increasing litigious nature of our society, many people don't merely grumble and figure out where they fell short when they don't make the cut, they accept no other conclusion than that the test is unfair and sue.

So the states have been moving in the direction of making their exams ever more defensible. The most defensible exam is one with binary or very limited choices. T/F and multiple guess. One answer is right, the others, not right. No professional judgment required to grade the exam, it's done by scantron. The flip side of that is that no demonstration of professional judgment is required to pass the exam either.

Exams like that are perfectly fine for testing and certifying technicians. Remember or look up the formula, crunch the numbers, look up the code section and find the answer, look up the name of the procedure asked for and find it on a list of 5 choices, mark the appropriate bubble. A person can literally gues their way into a license.

If we are going to do that, go to a T/F and multiple guess exam format, and members/employees of BPELS have stated publicly that the exam is going that way, then let's take what we now call a surveyor's license and call it a surveyor's technical certificate. Let's identify the activities in §8726 that are purely technical in nature and allow the certified survey technician to do those things without direct supervision. It would be the perfect certification requirement for party chiefs, chiefs of parties, and project managers overseeing survey projects which are purely technical in nature.

But let's then make a more professional level exam, model it after the Bar exam, but specific to survey issues. Some of the exam will look like what we have seen in the state specific PLS exam over the past 10 or 15 years: mostly short answer and some fill in the blank. There will also be essay type questions. The grading key for these will focus on issue spotting and appropriate reasoning based upon accepted standards of practice and law.

This would be the licensed professional level of surveying. Those areas within §8726 which go beyond the purely technical may only be performed by or under the direct supervision of this level.

This split level license could accomplish two things: 1) allow more LSIT level people to move on to a level which allows them to perform certain functions under their own certification, and 2) separate out the more abstract, process, and legal functions of the surveyor and restrict them to those who actually have an interest and study those aspects of surveying. Both of those things would serve to improve the standards of practice at each level of certification/license, IMO.

But maybe I'm just out in left field.

[JasonCamit]

since the PE exam in CA is all multiple choice, as I understand it (how many PEs do we have in CA now?). And I think the CA exam (state specific) as is, does a decent job in testing your professional decision making. Its difficult to fit every scenario into a (4) exam, along with technical testing as well. And our exam seems to be more curriculum based (technically based), sprinkled with some of these survey scenarios that you find in real life. This format, along with the criteria for allowing a person to sit for the PLS exam seems to be an issue. I am not sure how this happens, but I know people you have sit for the exam that I suspect (cannot prove) did not have a minimum of one year of responsible charge in the field. I sat on my first exam, next to a PE testing for her PLS. She said that this was her 12 time (yes 12) that she was taking the exam. I was stunned and scared, being a first time examinee. I asked her a little bit about her survey work history. She said that she worked on a field crew where she was currently employed, but did not appear to be responsible charge, IMO. I suspect also that she focuses on answering all of the corriculum based questions, since they are technical in nature and easier to study. Who signed off on her. How are we policing ourselves. I am sure this practice is not just limited to my PE table partner. Mandatory (4) year degree and corriculum based exams does not seem to be the right avenue, without focusing on someone's professional decision making abilities based on real world situations.

[pls7809]

RCE's may sit for the exam with 2 years of broad based land surveying experience that is satisfactory to the board. They are not required to have responsible experience or field experience for that matter. Scary.

See PLS Act 8742 (a)(3).

[LS_8750]

Well fellas, what's so wrong with the system as it is? Are we saying after getting licensure as the system is set up currently that we are not really qualified to make professional judgements and opinions? I hope not.

My gripe is and always will be that the LS component of the Board is a "good old boys network". I got my PE in 2003 and spent the next 7 years among my engineering duties tromping through the forrest, falling into creeks, covered in poison oak, blackberry thorns, stingy nettles, jumping over rattle snakes, dodging wild pigs, and applying for my LS. I finally got to sit for the test. And I thought I might have failed that damned test after the proctor said "times up" and I walked out of the room. The board should have let me take the test when I first applied, three years prior to my sitting, I was ready. My PE alone demonstrates my ability to make professional judgements and render professional opinions. But no, they made me work for it. I hold that damn LS license dear though, perhaps moreso than my PE. And yes I understand a lot of PEs that never had even close to the amount of field experience I had upon sitting for the exam that got to take the exam because they could doctor up the application. I call bollocks on that based upon my own circumstances at the time.

After all that I don't have any problem with the system as it is. I never took the LSIT so I cannot comment, but I can say that the NCEES portion of the exam was a joke. I cannot believe that NCEES test is the standard for licensure in other states. As for the state specific exam I thought it was tough, calculation intensive, cutting no slack for number crunching mistakes, it was legit.

Education and continuing education? Yes. Educate yourselves. Suck it up. I bet each and every LS out there would agree.

The PE was hard, very hard. The EIT was perhaps in another universe by comparison, it was difficult. I joke that the reason for more PEs by number than LSs and the pass rate being higher is that 99.5% of all candidates who sit for the PE exams are graduates of an accredited engineering curriculum.

Educate yourself, nevermind the exams, it's professional practice that counts.

Just a gool old boy whose earned it.

[E_Page]

Jason,

You've hit on a few different things here. Some of which I can comment on, and others I cannot because I have worked on the exam within the last two years and am bound by a confidentiality agreement.

This is a good time to remind anyone who has taken the exam within the past two years as they may not be fully aware of the fact:

IF YOU HAVE WORKED ON OR TAKEN THE EXAM WITHIN THE PAST TWO YEARS, YOU ARE BOUND BY THE CONFIDENTIALITY AGREEMENT YOU SIGNED AT THE TIME YOU TOOK OR WORKED ON THE EXAM. DO NOT DISCUSS PARTICULARS OF EXAM CONTENT.

When I took the exam, I barely remembered what I signed beforehand, so if you had asked me what I signed, I may not have remembered the confidentiality agreement. Like just about everyone else there, I had more immediate concerns.

OK, with that out of the way, I will say that the exam development teams strive to create an exam that is more professional in the issues tested, with the technical being less emphasised. Having said that, there are some practice areas which are almost purely technical in nature, and so questions in those areas will reflect that.

Issues presented in any of the questions, while obviously fictional, are designed to represent issues that you will face in real life practice. Sometimes entire scenarios are simplified in order to present a workable problem within the allotted timeframe, and sometimes sets of issues are included in one problem where they are unlikely to all occur on one real project, in order to test on those issues without having to create several more realistic scenarios in order to test on each issue. In any case, the scenarios are not so unrealistic as to require off the wall, overly complicated, or unrealistic solutions. It is an exam designed to measure minimum competency as a professional, not advanced competency or abilities.

And that's as close to the line any of us should step with regard to actual content.


A good professional exam should provide opportunity for the examinee to show reasoning and logical analytical process. It's not easy to write questions that avoid leading an examinee through the proper reasoning in a fill in the blank and short answer format.


I share your concerns about experience. First, engineers are not required to have "responsible training" in their experience, just broad based and progressive experience. In fact, they don't even need to split their time between field and office. It can be all one or the other. They need only to have worked enough different tasks or types of projects for it to be considered broad based, and they only need to have progressed from whatever point they started at.

That you met someone who took and failed 12 times does not surprise me. The most I've heard of was 23 times, and that may not even be a record.

I am of the strong opinion that once one fails three times, that person should have to re-apply, showing additional education and experience and supplying additional references.

I am also of the opinion that the Board should contact each of the original references and ascertain from them just why they thought this person was qualified. A little pressure like that may make some who are too free with their references consider it a little more carefully.

Along those lines, I am certain that some references go along with at least bending the truth when it comes to amount of or nature of a candidates experience.

IMO, each year there is a certain percentage of examinees who not only have no business taking the PLS exam, but should have their LSITs revoked for demonstrated incompetence. But I have to stop there before I venture into why that bothers me so much.


Here is my "exam veteran" story: When I showed up to take the exam the first time (1st in CA, already passed OR), near the exam hall doors, I witnessed a portion of a conversation between an exam "rookie" and an exam "veteran" of several years. The veteren was all set for the 8 hours ahead (this was in the mid 90s - 8 hr exam). He had his rolling cart loaded with all kinds of books, a cushion to ease the pain of his ample rear on the hard folding chair, and a thermos of coffee. I don't know why he had that, maybe for lunch break. The rookie had, as I recall, a small backpack commonly used for carrying books at school, much as I had.

As I approached, the rookie's face was ashen and had a worried expression as he listened to the veteran tell him, "Nobody passes this on their first try. It almost always takes at least 3 or 4 before you get lucky. Don't worry about it, just use this time to get an idea of what to expect....blah, blah, blah..."

I had no intention of ever becoming an exam veteran, so I walked well out of earshot of this knucklehead and his underconfident audience of 1. Poor rookie was probably so freaked out by the time he looked at the first problem that he drew a blank. Both of them were probably right back at the same spot the following April.

For 1st time takers: Avoid advice from multi-year exam veterans. They obviously have no idea wht it takes to pass. There are plenty who have passed on their first attempt. Take your advice from those who passed on the 1st or 2nd taking. Avoid conversations about the exam just prior to exam time. Those invariably focus on peoples fears and can only hurt your frame of mind.



I'm not really sure what you mean by "curriculum based exams", Jason. In the time that I've been involved with the exam, no faculty or other representatives of academia interest were involved in developing the state specific exam.

It is my understanding that the NCEES has a lot of academia representation. But I don't know how that might change content or focus. At the time I took the NCEES portions in OR in 1995, the questions did not reflect some "curriculum based" methodology that is different from methodology properly used in practice.

In fact, the college curriculum, in what they teach, teach the same methodologies that one should be using in practice. So I don't know what you mean by this. Perhaps you can clarify.

Surveying is a very broad field of practice. The types of problems seen by some on a regular basis may be encountered very rarely or not at all by other practitioners. But if it is an area of practice covered under §8726, it will be tested sooner or later. The content of the exam is based upon what practicing professionals reported to the Board as being included in their practice. The Board broke all that down identifying percentages of types of projects that make up the actual practice of surveying across the state. The content of the exam roughly reflects those percentages.

So if you saw a problem on the exam that you've never encountered in real life, it's not that it is something that one would only see in a 4 year curriculum. It's just something that you've never seen, but also something that some other surveyor(s) deal with on a regular basis.

[JasonCamit]

I guess I saw a few questions that were in reference to procedure that are not practiced anymore by a surveyor in 2011, that I learned in school, but I could be wrong.

[BeachBum]

The California State Specific LS exam is not perfect … which makes it perfect! How many boundary surveys has anyone completed that fit perfectly with all the evidence? I would venture to guess the number is close to zero. Personally, I would be very disappointed with a multiple guess examination, and if we go that route, then I would not have a problem with allowing PE’s to practice Land Surveying again (I can feel the daggers already!).

Here is my reason; some people have suggested to me that we test procedures that are really not in practice today. I say hogwash! Pick up any ancient textbook. My grandfather, who was an Engineer, gave me his textbooks from his college days (sometime in the 40’s) from all his surveying classes. The math and theories were exactly the same as my textbooks from the 90’s. Even to this day, we are still utilizing the exact same theories and principles. What has changed … technology! And what does this technology help us do … measure, that’s it! I have yet to find that magic instrument that surveys for me, all it has done is make it easier to obtain large amounts of data in a shorter period of time. This is only a small portion of what we do. If this is ok, then we are all just number crunchers and data collectors, which I think most engineers would be ok with.

Along with the all science, math, and technology that is in everything we do, the one key component that pieces all the data together is the ART of surveying. We make that square peg fit perfectly in the round hole. We take all the imperfect measurements (by today’s standards), along with all the conflicting evidence, and we make it work harmoniously together. And there we have our Picasso, Rembrandt, or whatever. Why do I call it art, well are any of you completely confident when you render you Professional Opinion? I would venture satisfied, but COMPLETELY confident? Just like an artist, never satisfied and a little worried that someone else will find an imperfection.

This test, as is, tells me what I would like to know. Does the candidate possess the logic to follow an imperfect survey, interpret correctly the intent (follow in the footsteps), and apply a conclusion that would include the least impact to the established surroundings according to state law and allow another surveyor to logically follow in the “footstepsâ€￾ of the candidate. Yes, the test is not perfect, but there is an infinite amount of ways to go from point A to point B. If a candidate were to come up with the wrong solution, the graders will take into consideration and see if your path is just another way to get to point B, and apply points as necessary. But that is the key, if other surveyors can’t follow your logic, then it would not be beneficial to the public to give the candidate a license and cause chaos.

Just my humble opinion!

[waterworks]

"I am of the strong opinion that once one fails three times, that person should have to re-apply, showing additional education and experience and supplying additional references.

I am also of the opinion that the Board should contact each of the original references and ascertain from them just why they thought this person was qualified. A little pressure like that may make some who are too free with their references consider it a little more carefully.

Along those lines, I am certain that some references go along with at least bending the truth when it comes to amount of or nature of a candidates experience.

IMO, each year there is a certain percentage of examinees who not only have no business taking the PLS exam, but should have their LSITs revoked for demonstrated incompetence. But I have to stop there before I venture into why that bothers me so much."

I could not agree more Evan! I've often wondered if there is a way to contact the board and challenge someone's exam application, meaning I know for a fact their application was falsified based on the level of their experience...an employee you've inherited and such? Is there a mechanism for that?

[E_Page]

WW,

Basically, without solid proof of a fraudulent application, that amounts to a bad reference, which doesn't count against the candidate in any concrete way as long as they have a sufficient number of positive references. it may cause the board application examiner to scrutinize the rest of the app a little more closely to see if there are irregularities.

Without saying whether or not I fully agree with the reasons, there is a logical reason for that. If a candidate has four good references and one really bad one, how much should that bad one count? Isn't it possible that the one bad reference is just someone who may have harbored some not so obvious resentments or dislike for the candidate?

Should the candidate be rejected if 4 other LSs state he or she is ready for the exam? In all likelihood, knowing nothing else of the reputations of the references, is it more likely that the one is givng the Board a true picture of the candidates experience and abilities, or that the other 4 are?

So if all you have is your statement with no documentation or other strong evidence to back up your claim, then no, there really is no mechanism to challenge an application. If you did have strong evidence of a falsified application and/or references, BPELS would most likely be very interested in it.

My suggestion: if you have proof, take it to the Board. If you have only your observations, gut feelings, or something similar without stonger evidence to back it up, let it go and hope the exam weeds them out.

making such an accusation without proof will reflect more poorly on you than it will on whoever you are reporting.