ACTUARIAL TESTING
By Jerry Tuttle, FCAS, CPCU, ARe
Society of Actuaries 2007 Actuarial Speculative Fiction Contest

(The events and persons in this story are fictional. Any resemblance to real people is purely unintentional.)

    I asked everyone but Woody to stand as we raised our beer mugs. “Congratulations to our newest Fellow, Woody Fishman! People who aren’t actuaries have no idea how hard our tests are.” As the senior actuary in the company, I had gathered all the company’s actuaries for a celebratory beer fest at the nearby pub.

    “So what will you be doing with all your free time?” asked Wendy Chang, one of the younger actuarial students.

    “I know one thing he will be doing,” Jay Thompson chimed in. “A little mandatory new Fellow volunteer work.” Everyone broke out in hysterical laughter – except Woody.

    “I don’t get it. What’s so funny?” asked Woody.

    “You become a new Fellow, and then how soon you forget. Oh man, I would hate to have to do that. How did that tradition ever get started?” asked Wendy.

    “Well, funny you should ask,” I replied, stroking my beard thoughtfully. “We’re out of the office, right? Whatever is said out of the office stays out of the office?” Everyone nodded in agreement. “I guess it’s OK to tell you. I have to admit that this quaint little custom is really attributable to something I discovered about ten years ago.”

* * *

    In 1982, I was working for a large insurance company with a big actuarial department. I was reporting to the chief actuary, Vince Lentini. I doubt if any of you remember those days, but that’s when there were three big insurance companies in town, and we were all sort of friendly rivals. The three actuarial departments were about the same size, and we generally knew our peers from actuarial meetings and softball games and such.

    One day Vince pulled me aside and asked me if I was aware of the chief actuary’s bet. It turns out that he and the other two chief actuaries had a standing bet every six months on the total actuarial exam pass ratios of their respective departments. Each of them would put $333.33 into a pool, with the winner taking the pot.

    It turns out all three of them decided the total exam pass ratio formula was too simple and not necessarily equitable. In fact Vince seemed to lose considerably more often than two-thirds of the time. The three chief actuaries agreed to nominate one actuary per company to a little task force, with the goal of deriving a more equitable pass ratio formula – solely for this chief actuary bet. Vince nominated me.

    Our task force met. We had experts in automobile merit rating, workers compensation experience rating – and baseball. Individual historical experience is used in all three of these areas to forecast future experience, and we thought we could do the same with people’s actuarial exam passing statistics. We found people’s age and years of experience had some nonlinear relationships with exam passing. We studied the pass ratios by exam – which are much different for the lower exams than the higher ones, by the way – and we investigated the probabilities of passing each exam at the first attempt, the second attempt, and so on, and how these probabilities change when someone enrolls in one of the actuarial cram courses or uses one of the supplementary study manuals.

    We put all of this into some regression software, and we came up with a preliminary version of an experience rating actuarial exam pass formula. We tested it retrospectively on past exam sittings for each of the three companies, we calibrated it, and came up with a final version that we considered equitable for each company. Now for each upcoming exam sitting, each company provided its own exam history for each student, and we produced a predicted pass ratio from the formula for each company. The winner of the chief actuary bet was the company that performed best, relative to its own predicted pass ratio.

    This formula worked fine for a good ten years. Each of the three actuaries was winning the bet about one-third of the time because the formula did a pretty good job of equalizing the three companies. It was as if we had created a betting point spread that had equalized the odds among the companies.

    Suddenly in 1994 the formula started to weaken. We began to seriously underestimate pass ratios, and in fact the company with the lowest expected pass score was outperforming the two stronger companies in the formula. This happened for four consecutive exam sittings, so we wondered what had changed. We reassembled the original task force to delve into the problem.

    There were numerous possibilities. Maybe the exams had gotten easier. Maybe the students were smarter or better prepared. Maybe something had changed affecting the mix of students or the mix of exams. We knew we had some investigating to do.

    We used our contacts on the Examination Committee, and they insisted that the exams and the grading were as difficult as ever – no surprise there. There had been no major change in syllabus readings. None of the three companies had increased the study hours. We spoke to the people running the actuarial cram courses and writing the study manuals, and none of them seemed to think anything had changed. We studied the mix of students and exams – always a good answer when there are unexplainable actuarial results is that the mix has changed - had there been a shift so that for example, 60% of the students were taking the first exam for the third time, and was that causing the formula to crumble? We could find no explanation, and we were running out of ideas.

    It occurred to me that since our three companies were all in the same town, most of the students would take their exams at the same exam site. Maybe the exam proctors had some insight. We interviewed the proctors for all of the exams from the last sitting. Only one of them had any useful thoughts.

    “To tell you the truth, I have been noticing something over the past few sittings,” began James Leech, a fellow from one of the other companies. “Now, you won’t quote me on this, right? Generally I assign the seating somewhat randomly, to reduce the chance of friends sitting together and cheating. But, my seating is not entirely random. I assign the best looking women to the front row. Hey, they have to sit somewhere, right?”

    “Other than watching for cheating, I really don’t pay much attention to any particular person, except I might sort of casually gaze at the people in the front row from time to time.”

    “What I have observed is after the exam is over, a number of students, both females and males, have been much more animated and chatty than they have been in previous years. People have had sort of a wild look in their eyes. If I didn’t know better, I would have thought some of the students might have been on drugs,” continued James.

    “Now that I think about it, something else has increased in recent years. During exams, people have become more compulsive than ever. They constantly straighten their pile of scrap paper, they arrange their pencils in a precise order – I find it odd, but I guess it doesn’t bother anyone,” concluded James.

    I’m no expert, but this smacked of drugs. I discussed this possibility with my task force, but they thought the idea was ridiculous. I have a colleague in another company, and if anyone would know about actuaries and drugs, he would. I called Sammy Finch and invited him to lunch.

    Sammy was not at all shocked at my suggestion. “There is a guy,” he began quietly, looking around to make sure nobody was listening. “I understand he was a bio-chemistry major who took a few actuarial exams and then decided the exams weren’t for him. He then started selling some kind of product online that claims to help people with the exams. I’m not saying it really works. But I have heard that some actuaries swear by it.” Sammy wrote down the address for a web page. The address looked like a random sequence of letters and numbers; it was not a web address you would find by yourself. “That’s all I know about it. And don’t use my name, OK?”

    I thanked Sammy for his information, picked up the check, and returned to the office. I carefully typed in the web address he had given me.

    The website that came up was full of new-age gobbledygook about spiritualism, the connection between metaphysical and medication, and other topics that I considered nonsense. I followed the alternative health remedies link and found a statement that “many of these remedies have not been formally validated by scientific research.” What a surprise! But there it was buried several pages into the website – a web page devoted to the actuarial exams.

    That web page had a good understanding of the actuarial exams – their mathematical nature, the enormous quantity of material that needs to be mastered, the need to both memorize numerous formulas and lists and then apply them creatively. The page went on to discuss the relationship between nutritional deficiencies and disorders such as hyperactivity and attention deficit disorder. Dietary supplements that treat those disorders could also be used in small doses for actuarial students. The supplements contained nutrients that worked with brain chemistry that improve focus, specifically appropriate to actuarial exam study.

    Finally the web page got to the real ad. There was a single pill for sale for $2,000 called Mathophosphatidylcholine. I thought the “Math” part of the name was a nice touch. The pill was specifically for actuarial exam-takers who had failed the same exam before with a grade of five. It was to be taken precisely 18 hours before the exam. There was no guarantee, and there were some temporary side effects. The pill was described as being not illegal, but the Federal Drug Administration had not evaluated this particular use. There was a phone number for more information.

    I called the phone number. The man on the other end had a deep hypnotic voice, and he would not identify himself. He clearly had an understanding of the actuarial exams. He was matter-of-fact about the idea of improving exam performance with a pill, although he kept referring to it as a dietary supplement. “You know, better living through chemistry.” He insisted it was perfectly legal.

    I tried to order a pill for myself. The man discussed the side effects, which were similar to what the exam proctor had observed. The pill was only available to people who had recently failed the same exam with a grade of five, and that an official Society exam grade and a notarized signature were required before he would sell it. The five was necessary, he explained, because this was not a magic miracle drug for students who had no hope of passing; rather, it was for students who had proved they were close to passing and merely needed a little extra help in memorizing and retaining information. He asked me if I were willing to provide my notarized proof of my grade of five. Although I had failed a few exams with fives, obviously I had not done so recently, so I declined, thanked him, and hung up.

    I wanted to discuss all this with my boss Vince, but I decided it was probably a good idea not to involve my own boss with anything about drugs. I decided to call the chairperson of the Examination Committee, Margaret Bradley, an old friend of mine. Margaret was incredulous when we discussed this, but she knew me well enough so that she would take my report seriously and do her own investigation.

    I never heard back from Margaret directly, but I did notice about a year later that the Society changed its exam instructions to the effect that actuarial students are not permitted to sit for the exams while under the influence of performance-enhancing substances. Students would be tested at the exam site, and their exam scores could be excluded at the sole discretion of the Society. Subsequent to that, our exam pass ratio formula reverted back to its old ways, and the three chief actuaries were happy. I’m not sure if they ever knew this story. Then one of the three companies was sold off, one of the chief actuaries changed jobs, and we retired the formula.

* * *

    There was silence at the pub as everyone considered my story. I could see some skepticism in people’s eyes. Did I make up the whole thing? I am known as somewhat of a prankster, so it would not be beneath me to make up a totally outrageous story about ancient actuarial history that nobody else could verify.

    That’s quite a story,” said Woody Fishman, the new Fellow, “but I still don’t get what it has to do with me and why everyone laughed at the idea of mandatory new Fellow volunteer work.”

    “You may be a Fellow, but they still don’t test for common sense on the exams, do they?” I joked. “Who do you think the Society gets to collect the urine samples of the students before the exam?”

END


Copyright 2007, Jerry Tuttle



The author would like to thank the many friends who helped with early drafts and with some of the details of this story.

My prior stories are Proof from 2001, 1 + 1 = 0 from 2003, and Actuarial Rating from 2005 . Thanks for visiting.


eXTReMe Tracker