02-001928 Ana C. Rivero vs. Department Of Health Status: Closed Recommended Order on Friday, December 20, 2002.

Summary: Petitioner failed to prove that she should have received a passing score on the Florida Medical Licensure Examination administered November 15 and 16, 2001, and her petition should be dismissed.

STATE OF FLORIDA

DIVISION OF ADMINISTRATIVE HEARINGS

ANA C. RIVERO, )

)

Petitioner, )

)

vs. ) Case No. 02 - 1928

)

DEPARTMENT OF HEALTH, )

)

Respondent. )

_________________________________)

RECOMMENDED ORDER

Pursuant to notice, a formal hearing was held in this case

on Septe mber 20, 2002, by video teleconference, with the

Petitioner appearing in Miami, Florida, and the Respondent

appearing in Tallahassee, Florida, before Patricia Hart Malono,

a duly - designated Administrative Law Judge of the Division of

Administrative Hearing s, who presided in Tallahassee, Florida.

APPEARANCES

For Petitioner: Dr. Ana Rivero, pro se

5787 Southwest 88 Court

Miami, Florida 33173

For Respondent: E. Renee Alsobrook, Esquire

Cassandra Pasley, Esquire

Department of Health

4052 Bald Cypress Way

Bin A02

Tallahassee, Florida 32399 - 1703

STATEMENT OF THE ISSUE

Whether the Petitioner shou ld receive a passing grade on

the Clinical Application of Medical Knowledge and Basic Science

& Disease Process portions of the Florida Medical Licensure

Examination ("FMLE") administered November 15 and 16, 2001.

PRELIMINARY STATEMENT

In Examination Grade Reports dated March 1, 2002, the

Department of Health ("Department") notified Dr. Ana Rivero that

she had failed to achieve passing scores on the Clinical

Application of Medical Knowledge and Basic Science & Disease

Process portions of the FMLE administer ed November 15 and 16,

2001. The Department reported that Dr. Rivero achieved a score

of 331 points on the Clinical Application of Medical Knowledge

portion of the examination, which had a minimum passing score of

350 points, and that Dr. Rivero achieved a score of 332 points

on the Basic Science & Disease Process portion of the

examination, which had a minimum passing score of 350 points.

Dr. Rivero timely requested an administrative hearing, and the

Department transmitted the matter to the Division of

A dministrative Hearings for assignment of an administrative law

judge.

In her request for an administrative hearing, Dr. Rivero

challenged the fairness of the administration of the November 15

and 16, 2001, FMLE on four grounds, as follows:

(1) On the firs t day of the FMLE, the

answer sheets were not distributed on time.

They were distributed 45 minutes after the

exam began; consequently not only causing a

lot of stress on myself and other test

takers, but also prolonging the termination

of the exam.

(2) At random, I observed as I took the

exam, some questions whose correct answer

was not the same on the English version as

on the Spanish version, yet there is only

one answer sheet for both English and

Spanish exams.

(3) Under Section 458.3115 1(d). . . States

"Every person eligible for restricted

licensure under this section may sit for the

USMLE [United States Medical Licensure

Examination] or the agency and board -

developed examination five times within five

calendar years." I have only taken the FMLE

four times in a three - year period beginning

in 1999.

(4) I also question the Angoff method,

which is the chosen method of scoring the

FMLE, because it differs from the method

used to score the USMLE.

At the hearing, Dr. Rivero testified in her own behal f, and

Petitioner's Composite Exhibit 1 was offered and received into

evidence without objection; this exhibit consists of six

separate documents, including the Examination Grade Reports for

the Clinical Application of Medical Knowledge and Basic Science

& Disease Process portions of the FMLE administered November 15

and 16, 2001; a Department of Health Election of Rights Form,

together with a notice that the final FMLE examination was

administered November 15, 2001, and that it cannot be given

again; a cop y of Section 458.3115, Florida Statutes (2002); a

copy of Dr. Rivero's request for an administrative hearing, to

which is attached a memorandum from Department employee Gregory

Stone; a letter to Dr. Rivero from David Paulson, Manager of the

Department's T esting Services, in which he explains the scoring

system for the FMLE; and a copy of "Scoring and Score Reporting"

dated September 16, 2002, from the 2002 USMLE Bulletin. The

Department presented the testimony of Eunice Filer, a

psychometrician employed b y the Department; Margaret Anglin, a

Program Administrator with the Department; and Jeanne Smith, an

Examination Supervisor with the Department. Respondent's

Exhibits 1 through 3 were offered and received into evidence. 1

One of the bases for Dr. Rivero's challenge to the

examination is that there were discrepancies between the English

and the Spanish versions in the order in which the answers were

listed for some questions in the examination. Dr. Rivero wanted

access to the Spanish version of the examinat ion so that she

could compare the Spanish version with the English version to

identify the discrepancies. The Department objected to

production of the entire examination and requested that

Dr. Rivero's review be limited to those questions and answers

that the Department scored as incorrect on Dr. Rivero's

examination. The undersigned limited the production as

requested by the Department, and the Department agreed to

provide the English and Spanish versions of the selected

questions and answers to Dr. Rive ro for her review. The parties

were advised that Dr. Rivero could request that the final

hearing be reconvened if she wished to present evidence to

establish a discrepancy between the English and Spanish versions

of the examination.

A telephone conference was held on November 4, 2002, with

counsel for the Department and Dr. Rivero in attendance. At

that time, Dr. Rivero stated that she had not been given access

to the English and Spanish versions of all of the examination

questions and answers and that sh e had refused to review the

examination questions and answers produced for her review by the

Department in accordance with the instructions of the

undersigned. Dr. Rivero stated during the telephone conference

that she did not wish to present any addition al evidence in

support of her challenge to the examination, and the hearing was

closed.

The transcript of the proceedings conducted on

September 20, 2002, was filed with the Division of

Administrative Hearings on October 18, 2002; during the

November 4, 20 02, telephone conference, it was agreed that the

parties would file their proposed findings of fact and

conclusions of law on or before November 22, 2002. Both parties

timely filed their proposals, which have been considered in the

preparation of this Rec ommended Order.

FINDINGS OF FACT

Based on the oral and documentary evidence presented at the

final hearing and on the entire record of this proceeding, the

following findings of fact are made:

1. The Department is the state agency responsible for

licensi ng and regulating physicians practicing medicine in

Florida, including foreign - licensed physicians.

Sections 458.311 and 458.3115, Florida Statutes (2001);

Rule 64B8 - 5.002, Florida Administrative Code. The Department is

also authorized to administer lice nsing examinations to

physicians seeking to practice medicine in Florida.

Section 456.017, Florida Statutes (2002).

2. Dr. Rivero was accepted as a candidate for the FMLE and

sat for the examination on November 15 and 16, 2001. Dr. Rivero

attained a sca led score of 332 points on the Basic Sciences &

Disease portion of the examination and 331 points on the

Clinical Application of Medical Knowledge portion of the

examination. Each of these portions of the FMLE administered

November 15 and 16, 2001, had a minimum passing score (also

known as "cut score") of 350 points.

3. On November 15, 2001, the first day of the examination,

the Department staff who were to administer the examination were

notified that the Federal Express shipment of examination

material s was incomplete and did not include the laboratory

value sheets and the answer sheets to be used for each portion

of the examination. The supervisor of the examination

administration arranged to have copies of the appropriate

documents available that mor ning.

4. The examination candidates, who had been told to arrive

at the examination site at 7:30 a.m., were not admitted to the

examination room until 8:30 a.m. as a result of the problem with

the laboratory value and answer sheets. The candidates were

t old to skip the questions that required use of the laboratory

value sheets and to write the answers in the examination

booklets for the questions that required use of the answer

sheets. The candidates were advised prior to beginning the

examination that t hey would be allowed additional time to

transfer their answers from the booklet to the answer sheet.

5. The examination began at 9:30 a.m. on November 15,

2001, after a delay of one hour. The administration supervisor

made an error calculating the time a nd gave the candidates four

hours and ten minutes to complete the examination, rather than

the prescribed four hours. In addition, all candidates who

wanted additional time to transfer their answers from the

examination booklet to their answer sheets were given as much

additional time as necessary.

6. Dr. Rivero experienced stress and nervousness as a

result of the delay and confusion in the administration of the

examination that might have affected her performance on the

examination. She did, however, h ave sufficient time to complete

the examination on November 15, 2001, and to transfer her

answers to the answer sheet.

7. The minimum passing score on both portions of the

examination was 350 points. These "cut scores" were developed

for the November 200 1 FMLE using the Angoff method of scoring.

The Angoff Method is a widely used method for selecting the "cut

score" for an examination. For each administration of the FMLE,

a group of physicians are chosen to review the examination and

determine, question by question, the percentage of minimally

competent people who would answer each question correctly. The

"cut score" for each portion of the examination is developed by

averaging the responses of the physicians.

8. The Angoff method was a valid methodolo gy for

ascertaining the "cut scores" for the November 2001

administration of the FMLE.

9. After the examination was scored, a group of physicians

and a psychometrician met to review all of the questions that

were the subject of a complaint by examination candidates and

all of the questions that a statistically significant number of

candidates answered incorrectly. The group also conducted a

Point by Serial review of the examination, which involves

establishing that the candidates scoring highest on the

ex amination answered a particular question correctly, while

candidates scoring lowest on the examination answered the same

question incorrectly.

10. As part of this post - examination review, the

November 2001 FMLE was reviewed for discrepancies between the

o rder of the answers to questions in the English version of the

examination and the order of the answers to questions in the

Spanish version of the examination. No discrepancies were

found. 2 In addition, Dr. Rivero conceded that there were no

discrepancie s between the English and Spanish versions of the

questions she answered incorrectly.

11. The results of the review of the November 2001 FMLE

established that the examination was fair, reliable, and valid.

12. The November 2001 FMLE was developed, scored , and

reviewed in accordance with the procedures normally used by the

Department. Dr. Rivero has failed to establish that she should

be awarded additional credit for any question the Department

scored as incorrect on the Basic Sciences & Disease and on th e

Clinical Application of Medical Knowledge portion of the

examination. 3

CONCLUSIONS OF LAW

13. The Division of Administrative Hearings has

jurisdiction over the subject matter of this proceeding and of

the parties thereto pursuant to Sections 120.569 a nd 120.57(1),

Florida Statutes (2002).

14. Dr. Rivero has the burden of proving by a

preponderance of the evidence that the Department's

administration and scoring of the November 15 and 16, 2001, FMLE

was arbitrary or capricious or constituted an abuse o f

discretion. See State ex rel. Glasser v. J. M. Pepper , 155 So.

2d 383 (Fla. 1st DCA 1963); State ex rel. Topp v. Board of

Electrical Examiners , 101 So. 2d 583 (Fla. 1st DCA 1958). Based

on the facts found herein, Dr. Rivero has failed to meet this

burd en.

15. The record reflects that the Department made

accommodations to the examination candidates on November 15,

2001, to allow them to compensate for the late arrival of the

laboratory value sheets and answer sheets. Dr. Rivero did not

establish that s he suffered any adverse effects of the delay

except for an increase in her level of stress and nervousness.

This is insufficient to establish that she should be awarded a

passing score on the Clinical Application of Medical Knowledge

and Basic Science & D isease Process portions of the FMLE

administered November 15, and 16, 2001.

16. Dr. Rivero conceded that, even if there had been

discrepancies between the English and Spanish versions of the

examination in the listing of answers to several questions,

ther e were no such discrepancies with respect to the questions

that the Department scored as incorrect on her examination.

Therefore, any such discrepancies would not result in

Dr. Rivero's being awarded a passing score on the Clinical

Application of Medical Knowledge and Basic Science & Disease

Process portions of the FMLE administered November 15, and 16,

2001.

17. Finally, the evidence presented by Dr. Rivero is not

sufficient to establish that the Department's use of the Angoff

Method for developing the " cut scores" for the November 15 and

16, 2001, administration of the FMLE was arbitrary or

capricious. Rather, the evidence establishes that the Angoff

Method is widely used and produces an accurate "cut score."

Consequently, Dr. Rivero has failed to esta blish that the "cut

scores" for the examination should be lowered to a level that

would result in her achieving a passing score of the Clinical

Application of Medical Knowledge and Basic Science & Disease

Process portions of the FMLE administered November 15, and 16,

2001.

RECOMMENDATION

Based on the foregoing Findings of Fact and Conclusions of

Law, it is RECOMMENDED that the Department of Health enter a

final order dismissing the petition of Ana Rivero challenging

her failing scores on the Clinical Appl ication of Medical

Knowledge and Basic Science & Disease Process portions of the

FMLE administered November 15, and 16, 2001.

DONE AND ENTERED this 20th day of December, 2002, in

Tallahassee, Leon County, Florida.

_____________ ______________________

PATRICIA HART MALONO

Administrative Law Judge

Division of Administrative Hearings

The DeSoto Building

1230 Apalachee Parkway

Tallahassee, Florida 32399 - 3060

(850) 488 - 9675 SUNCOM 278 - 9675

Fax Filing (850) 921 - 6847

www.doah.st ate.fl.us

Filed with the Clerk of the

Division of Administrative Hearings

this 20th day of December, 2002.

ENDNOTES

1 / Respondent's Exhibits 2 and 3 are the Examination Grade

Reports at issue herein, and the se two documents are also

included in Petitioner's Composite Exhibit 1 and were

transmitted by the Department to the Division of Administrative

Hearings with Dr. Rivero's request for an administrative

hearing. The Department claims that these documents ar e

confidential pursuant to Section 456.014(2), Florida Statutes;

this contention is rejected, however, because that statutory

section applies only to "examination questions and answers."

The documents identified by the Department do not contain any

examin ation questions or answers.

2 / One of the exhibits introduced into evidence by Dr. Rivero is

a memorandum dated April 1, 2002, from Gregory Stone, a Senior

Management Analyst, to Cherry Shaw, the Department's former

attorney in this case, in which he add resses Dr. Rivero's

complaints regarding the November 2001 examination. In the

memorandum, Mr. Stone stated that "two items were translated

incorrectly on the Spanish version of the examination" and that

"all candidates were given credit for the questions ." Eunice

Filer, the Department's psychometrician, testified that this

information is incorrect, that Mr. Stone had confused the

November 2001 FMLE with the FMLE administered on another date.

3 / A final point raised by Dr. Rivero in her request for an

a dministrative hearing is that she has taken the FMLE only four

times within the past five years. Dr. Rivero is apparently

referring to the notification she received advising her that,

pursuant to Section 456.017(1)(c)2., Florida Statutes (2002),

the final administration of the FMLE was November 15, 2001, and

that the examination cannot be given again.

Dr. Rivero cites Section 548.3115(1)(d), Florida Statutes

(2002), as providing her the right to sit for the USMLE or the

FMLE five times within five ye ars. The question of whether

Dr. Rivero can sit for the FMLE or the USMLE again is not within

the purview of this administrative hearing. Nonetheless, it is

noted that the evidence reflects that Dr. Rivero sat for the

FMLE in May 1999, May 2000, May 2001 , and November 2001; she

applied for the November 1999 administration of the FMLE, but

she did not show up to take the test. Dr. Rivero also sat for

the USMLE in March 1998 and June 1998. Dr. Rivero has been

unsuccessful in her efforts to pass either the FMLE or the

USMLE.

COPIES FURNISHED:

E. Renee Alsobrook, Esquire

D epartment of Health

4052 Bald Cypress Way

Bin A02

Tallahassee, Florida 32399 - 1703

Ana C. Rivero

5787 Southwest 88 Court

Miami, Florida 33173

William W. Large, General Counsel

Department of Health

4052 Bald Cypress Way

Bin A02

Tallahassee, Florida 3239 9 - 1701

R.S. Power, Agency Clerk

Department of Health

4052 Bald Cypress Way

Bin A02

Tallahassee, Florida 32399 - 1701

NOTICE OF RIGHT TO SUBMIT EXCEPTIONS

All parties have the right to submit written exceptions within

15 days from the date of this recomm ended order. Any exceptions

to this recommended order should be filed with the agency that

will issue the final order in this case.