Study design: from a purpose to research questions to the study design. So how do you come to a study design that suits your questions and your basic idea of research? It's basically five main components you have to take into account.
That's the purpose. So why do you do this research? What is your interest?
Also then the conceptual model and both these components are essential for formulating your research questions, which are really central. They define your variables. They define the relationship between these variables.
Also, where are your main questions in the conceptual model, considering your purpose and based on these research questions, then you can define your methods. Do you go for a qualitative, a quantitative study, a mixed method study? And where do you sample and where do you collect your data?
And I will talk about these components in this session. So, there are different design considerations. First of all, what is your focus and why do you do this study?
What do you want to evaluate? Do you want to focus on outcomes? We will, in that sense you can also speak of a kind of summative study, when you're interested in the outcomes.
Then you typically collect quantitative data and the research field speaks of a fixed design or a quantitative study. If you're more interested in processes, how is it that things work? Why do they work a certain way?
Explanations on that? You want to understand or shape also a product, which variation works best. And why is that?
You often use flexible designs. And of course, depending on the focus, you also can combine these two strategies. Other consideration is the impact you want to achieve.
So, let's say you starting from a very concrete context and a very concrete question that comes out of practice that often tends itself to connect directly to practitioners, to work more with qualitative methods and use flexible designs. Flexible also in the sense that you probably get more iterations back to your conceptual models and to your variables, because you probably don't understand completely at the beginning of your study or what you're doing. You need to first also explore the field and understand your variables.
This is very important if your research really focuses also on initializing change in practice, because then the link to your actual practice is very important. Another is the purpose of your research, so flexible or qualitative strategies are more appropriate if you want to explore what are the possibilities, or if you want to shape a product. You typically do that by talking to your end users, by taking into account different variations possible.
On the other hand, you can use non-experimental fixed strategies when you want to do descriptive studies. So, you want to describe what users do with your product, how they interact with it. But you can do that still by collecting quantitative data.
And last but not least, if you really can go for an experimental variation of your treatments, this would be then a classical fixed design. So, you really look for testing your hypothesis and collecting data for testing these hypotheses. And last but not least, collecting data in the sense so.
. . when you're going for a fixed design, either a non-experimental, quasi-experimental or experimental design, you really should be sure what kind of measurements you need to take and how you collect these samples and from which user group or where you get the data from.
On the other hand, if you first need to explore and collect data and you're not 100 percent sure how to measure the concepts in your models and first need to understand some part of that and explore that, so you should go for flexible designs. So, again, just to repeat that, and what I want to stress here again is depending of. .
. on your design, if you go for a flexible or fixed design or mixed design, there are iterations possible and a fixed design typically is more streamlined that you go from the purpose and the conceptual model to your research questions. And these clearly define your methods and sampling strategy.
On the other hand, with qualitative studies, this is more of an iterative process and adaptation. So, some examples of typical fixed designs, fixed designs are experimental studies. What is characteristic for these studies?
Typically, you have an intervention you do in the study. So also, the golden standard for so-called experiment. You do random sampling so you don't need to afterwards control and it seems probably that you have control variables that are modifying variables, but you have specified your dependent and independent variables in advance and you want to test a hypothesis about them.
In non-experimental studies, what we do a lot in the field of learning technologies, for example: it's often not possible to have a systematic variation, to have a real control group and experimental or treatment group. But this also has to do with ethical considerations you should take into account. So, in some cases it's simply not possible to work with control and treatment groups.
And in that sense, you probably have to work with so-called post-hoc splits. So, you give all the users the same treatment or the same opportunity and you see how they use this opportunity and you're interested in the effects of when users use this opportunity. So basically, afterwards, after your intervention or your study, you can split the users in users that really made use of a new feature you want to introduce to a product and the other group that didn't use it.
And you look at the effects on your variables that you're interested in. In a lot of cases, the variables here are not fully specified, or you can apply a splitting of the group. On the other hand, flexible designs, and here you get basic methods like a case study, ethnographic study or a grounded theory study.
And case studies are typically you select one case and try to understand the context. So, it's a lot about the application context. And you can use a range of data collection methods like interviews, observations and others.
Ethnographic studies, on the contrary, more focus on understanding a certain group, community or organization. And you typically immerse as a researcher in this community and try to understand via participant observation. And thirdly, a grounded theory study: It generates theory from the data collected.
So, you have data collections, you do several iterations and work on the methods. So, in summary, this is the core takeaway you should remember. Ask yourself about the purpose.
Consider your conceptual model, how are your research questions formulated and that clearly then either leads to iterations and flexible designs or structuring of your variables in fixed designs. So, to summarize purpose that might be descriptive study, you might be interested in the process or more in the outcomes. The conceptual model should like specify your core constructs, concepts and their interrelations.
And the research questions should name your dependent and independent variables and define hypotheses on their dependencies and relations. And in the sampling of course, it's very important to consider the available context and also ethical considerations. Thank you.
