Introduction to Clinical Study Design: Where to Start Part 1

My name is Dr Laura Lee Johnson. I'm the associate director of the Division of Biometrics III at the United States Food and Drg Administration. I'm also one of the co-directors for IPPCR.

This is our disclaimer, courtesy of my working at the FDA, you know, they hire me, they pay me, but they only want to take account of anything that they decide they like and disown anything that they don't like. Part of what's really important here is in fact there are a lot of different people who take this course from all around the world. And for some of us, it's the first introduction and for others we have a pretty advanced understanding of clinical research.

So part of what I aim for in my lectures is to give some of the tricks, and the tips, and the concepts that I've learned over the years from the various investigators I've worked with. So my general objectives here, I want you all to be better consumers of the medical and scientific literature because while this course is focused on principles and practice of clinical research, those principles and practices are true for non-clinical research, too. There's a huge push right now also from the NIH and from many other groups.

You'll see many journals that have written their editorials saying, "We are no longer going to accept really bad research just because it's pre-clinical. " And all of these study designs, many of them, they have laboratory components. Not just because you do.

But my biggest randomization problem happened in somebody who did not randomize on her 96 well plate, and it caused a huge problem for her study, and her interpretation. So I want you to be better consumers. I want you to be better users.

And I want to enhance the conversation inside research teams. And it might be with your study statisticians and epidemiologists, but it also will be across and between a lot of different folks. Realistically, what we really want is better science.

With the information you learn, you're not going to be able to do your own statistical analyses. Like, some of you already can do your statistical analyses based on when you were raising your hands. But realistically, if you want to learn how to do stats, you want to learn how to be a data manager, you need to go take direct coursework in how to do that.

But this should aid you to improve your abilities to critically evaluate grant applications, protocols, and the literature. No one's an expert in any given area, but we do all try to combine our expertise, and in this world, everything really is a team science work. Because it's really easy to write that your study's going to use a randomized double-blind control, parallel-arm design, and do an intent-to-treat analysis.

It's really easy to say that subjects and participants are going to be consented. Those which you've already heard from the first two lectures, and you're going to learn throughout the entire course, it's really not easy to do it. And it's not easy to implement and maintain the integrity of your randomization.

And Pamela [spelled phonetically], Shawn [spelled phonetically], and some other folks have written in the chapter on randomization, a lot of the threats to that integrity that show up, very well-meaning people, trying to do good studies, and how they were undermined. It's hard to maintain blinding and masking. One of the tricks one of my investigators taught me, she actually made badges for her study staff, and they had blinded with a little person with a little mask, almost like a racoon-looking thing, and then unblinded.

And all of her staff wore these badges because she was doing a study where it was a physical intervention. And people knew that they were doing yoga, or they weren't doing yoga, but she didn't want them to talk about it to the people who were actually taking them through all of their study measurements. That was an innovative way to try to protect the blinding.

It helped her participants remember who they couldn't talk to but then who they needed to complain to. Multiple study arms. How do you make sure your study arms aren't bleeding into each other?

Data collection. How do you actually make sure that you're standardized your data collection process? We'll talk a lot about that throughout the several lectures.

And also, how do you transfer data to regulatory and other groups? It might be that your studies don't fall under FDA purview, but they might. There are a lot of different regulatory organizations around the world and a lot of different rules that you have to follow.

But many times, different people want your data and even if none of the regulatory groups do, this is a time and world of data sharing, and how do you adequately share that data, and make sure it's useful to other people? There're a lot of data standards you're going to hear about later in the winter and those will be interesting, useful for you, too. But that's a long view.

Now just tonight, I'm going to talk about how to identify study designs that are used in clinical studies, epidemiology public health research. We're actually going to cover most of the epidemiology next Monday. I'm also going to talk about masking and blinding, different types of interventions, and comparison groups.

So if you want to know what chapters in the book this is covered under, it's chapters 19 and 29. So what is your question of interest? Are you trying to interpret work in some new population?

Are you trying to make a decision about an individual case? How many people in the room are actively, clinically doing work? A handful.

Okay. So what we find out comparing two groups of people, or multiple groups of people may be very different than when you need to make a decision about that individual patient in front of you. And for each of us, like, I talk to my dad.

What drug is he going to use? His doctor talks about like the pluses and minuses of all these different therapies. What in the end is the decision?

And how does it work for him? We're looking at changing a population. Diabetes management, a large portion, has tried to shift the curve of a population.

Sometimes, classically we look at those differences of groups in a study. But sometimes we're trying to do biomarker development, we have to figure out exactly what type of biomarker. People love biomarkers, but they forget to figure out exactly what the biomarker is for sometimes.

Are you trying to develop a whole new outcome? Part of my job is it's patient reported outcome liaison. I help people develop new endpoints for clinical trials that involve the patient voice.

The level of evidence. What is it that we're trying to establish? Are we trying to figure out what the current level of evidence is?

You're going to hear about meta-analyses and other types of secondary data reviews. Is that what we're trying to do instead? Regardless of what you're doing, always remember the analysis follows the design.

Your question will always come first. And we may have to edit it because it may not be directly answerable, but your question comes first. If at the end of the day, they're answering something that does not address your question, you need to say, "Hold up, people designing this study and analyzing the data.

That is not what we need to do. " Because your question is going to drive the hypotheses. We're going to actually design that experimental design for your study is in order to make sure we can test the hypotheses.

We're going to do all of our sampling, all of that data collection, in-line with the experimental design. Your data comes from the samples. We analyze the data.

We draw conclusions. That generally leads to more questions, and we start the whole process again. But sometimes I look at data analyses, and I read papers, and I'm like who cares?

Like you answered a question that was answerable, but it wasn't the actual question of interest. So always strive -- like the reason statisticians have jobs, it's not just because we like to analyze data, but to say you have a new, cool question, and we don't have a method to do it. So we need to develop the methodology to answer the actual pertinent question.

The other problem, though, is you need to take all your design information to a statistician early and often because part of our job is to give guidance about some of the assumptions for the methods. And to try to help make sure that we're going to do the best job possible with the fewest subjects possible to answer a question. Because, of course, you ask a statistician how they see research study, they say everything impacts the statistical analysis.

But I will also say that that's not just my job security. That's because sometimes at the end of the day, investigators bring the information to me, and I say, well, we've undermined the integrity of the study by making the following decisions. So I can't help you.

You collected all that data and it's basically worthless. You don't want that to happen. It's not good for you.

It's not good for your study team. And it's not good for the human beings that agreed to participate in your study. So we're going to go through a little bit of vocabulary.

But, of course, none of you will want that to happen and you sure don't want it to happen because you're here late at night listening to me talk. So when I go through vocabulary, part of this is to get us on similar footing. We will talk about arms.

I do not mean my appendage. But in clinical research, we talk about study arms, or samples, or groups. We use these words fairly interchangeably.

A lot of times we talk about wanting to demonstrate superiority. So remember John powers talked about this a little bit last night. When you want to demonstrate superiority, you're talking about detecting a difference between either groups, or between treatments, or study arms.

The idea is that there's a difference in some way, shape, or form. Sometimes we say we want to demonstrate that the different arms are equally or similarly effective. So that is an equivalence trial.

Sometimes we want to demonstrate that things are non-inferior. You got to be careful with non-inferiority because sometimes they start stepping away, so I have one non-inferiority study, then they say, well, now I have a new compound, so I need to show that it's non-inferior to the first thing you showed was non-inferior. I got to make sure it's not inferior to group one, right.

So non-inferiority. You can also think of this while it's not exactly the same kind of like generics, right. When you think about equivalence, I sometimes think about generics, I have a generic cough syrup, I want to cough about the same amount plus or minus a little bit, that's equivalence.

Non-inferiority is [negative] it may be a little bit worse but not enough that it really matters. Figuring out that margin, big, big difficult problem. Also, I'm a very bad lady and interact and use patient versus participant versus subject.

Truth be told, what you're supposed to do is say a study subject. Why? That helps kind of differentiate, like, really when you're in clinical research, you are a guinea pig.

I sign up for clinical trials, I recommend anybody who does work in clinical research, sign up for trials. Because you should understand what it means to have your data possibly out there and breached. You should understand what it -- pain in the neck it is to fill out all these forms.

Understand what the burden is. But we also sometimes, in the literature especially, more of the behavioral social science, are talking about participants. We want people to feel like they're participating in research and actively engaged.

I do a lot of my work now, though, with patient medical records, so they are literally patients that we are working with. But because of that, I have a tendency to flip in between all of these three words but what I should always be saying is participant and subject. So shame on me.

Don't make my mistakes. A little bit about study design taxonomy. I kind of break the world into interventional and observational studies.

Interventional means I do something to you. Observational means I watch the film of your life, or the photograph, as the case may be. We also break the world into longitudinal versus cross-sectional.

So longitudinal is the film. So when I look at you at baseline six months, seven months, eight months. Cross-sectional is I give you all one survey right now and we're done.

It's kind of like a census. We ask you something once, walk away. Prospective versus retrospective.

Prospective means I'm going to follow you into the future. I'm going to kind of take my data real time. Retrospective means I look back in the past.

So I may be looking back at employee health records that were gathered over the last 30 years by a Department of Defense or an army. So retrospective is looking backwards at data that was already collected, may or may not have been collected the way you wanted it, or the data you wanted, but you're using what's already there. Prospective you are moving forward and collecting data.

Don't get too hung up on those, but it is a little bit important. So like if you're prospectively collecting data, and storing it, and you go back and analyze your stored data, it's still a prospective study. Blinded or masked.

So this is when the investigator, the people running the study, the study participants do not know what intervention they're on. Sometimes we actually mask them instead to the hypothesis that we're testing. Sometimes we do not blind or mask a study and it's called open label study for some of the interventional trials.

But depending on who all is blinded or not, single blind, double blind, unblinded. I used to do some work for the National Eye Institute. They do not like blinded.

They prefer masked. You can understand why. Depending on where you train, you will see different names for each.

Randomized or nonrandomized. We're going to have a whole lecture on randomization. Paul Wakim gives a great lecture on this.

The idea is how am I allocating subjects. Is there a random element to it? Or can I figure out who's going to be going into which treatment group?