IBCD 2016 - Innovation and biomarkers in cancer drug development
When is an assay ready for the clinic?
Dr Erasmus Schneider - Wadsworth Center, Albany, USA
21 years ago I joined the New York City Department of Health with the express mission to set up a cancer testing regulatory programme. It has grown tremendously, obviously; when I started we were really looking at soluble tumour markers, your standard PSA, CA and so on. We developed a proficiency test programme, or EQA as it’s called in Europe, but over the years, probably the late ‘90s, the lab developed testing, overview became really important and has grown tremendously since then and we get probably in the order of 200 submissions per year that myself and an associate review for clinical and analytical validity.
Could you tell us about what you presented at the conference?
The question I was asked to address by the organisers was when is a test ready for clinical use from the healthcare system’s management perspective as well as who decides and what is the basis for a decision. So what I focussed on is the evolution of a test which starts with the discovery in usually a research/academic lab. People develop an assay that works, it’s not very well standardised but they get some initial results, usually it leads to publication. The question really is how do you translate this research assay into a clinical assay where, of course, requirements for reproducibility are much higher because we all want to be sure that when we have our sample tested that the result is trustworthy.
So that’s sort of the question I was trying to address and I think the main issues or main points are that once you translate from research to clinical you have to do all these validation steps pre-analytical where you have to make sure the reagents do what the manufacturers claim they do. You have to make sure that your samples are stable and robust; there is data out there that suggests that a lot of research studies that are irreproducible are not because the assay was bad but because the input sample was bad and they don’t have the original samples any more. Then the analytical validation obviously is you have to make sure it’s reproducible, limit of detection, it’s precise, it’s accurate and once you have that you what’s called lock down your assay. You’re no longer going to change your reagents, how you do it.
From there you go to a clinical validation which really wants to determine how many false positive and how many false negatives you have. Basically, can you separate a patient population into two groups – those affected with the disease that have the marker and those that are not affected that don’t have the marker? That’s the sensitivity and specificity at the clinical level. Once you’ve achieved that the assay is basically ready for clinical use. A second level validation is then clinical utility which essentially is does the assay provide, or the test provide, information that helps the physician decide how to treat the patient? Does it affect patient management? When they are little bit into an expanded definition of clinical utility that was just published in The Journal of Molecular Diagnostics in the September issue that not only looks at the patient but also the family and how you make decisions in a larger context rather than just treating the patient.
Could you give any tips or advice to people looking for a valid assay?
The most important step you have to do is decide what you want to do with the assay, what patients you want to target, what disease, what should be the outcome. What information do you want to gain from that assay? Once you do that then you can start designing your validation protocol. So there is no one size fits all recipe but starting with the intended use and what information you want to gain is the best thing to do.
Whose call is it, ultimately?
Who decides? Right now in the US we have obviously the FDA, the Food and Drug Administration, that deals with in vitro diagnostic devices which are tested, are manufactured and validated by a company then are sold to a clinical lab for their performance but the manufacturer has no control over how it’s performed. So even a well-validated manufactured assay that’s sold to a lab can be done poorly if the lab doesn’t know what they’re doing.
On the other hand we have the lab-developed tests, LDTs, that are developed and manufactured in the lab that actually performs it and that is actually where New York State’s clinical evaluation programme, which I work for, comes in. We review all the assays that are offered to New York State patients. If we say it’s ready then obviously a lab can offer it commercially, if you want, or clinically. Other people that are involved are organisations like obviously the College of American Pathologists’ joint commission, various insurance companies. But essentially everybody with slightly different weighting of the factors look - is an assay clinical valid, is it analytically valid? That’s the baseline that you have to meet.
Did any questions come up following your presentation?
No direct questions came up; there was a lot of discussion on sample quality. There was another presentation in my session where somebody talked about the quality of sample, how you collect them, how you store them and that is very important obviously. Especially if you go beyond DNA testing, you go into more RNA and protein testing. So, yes, that’s part of the pre-analytical validation is how do you collect your sample, how do you store your sample, how do you ship them to the lab, what do you do with them; that is very important for reproducible assays. So you don’t want to have a false negative just because your sample was degraded.
