So how to collect your data and what data to collect? So, when you start with your research and plan your data collection, of course, based on your purpose and on your conceptual model, you should become aware of what data to collect. Also, what quality of data do you want to collect?
Quantitative data, qualitative data. So, and what level of data is needed to answer your research questions? So, in a lot of cases, when starting with certain dependent variables, of course, you can find standardized methods to measure those.
So, if you would like to measure, for example, the motivation of your users for a certain topic, there are standardized psychological inventories to do so. And a lot of ideas and inspiration you get from the related work of your literature review, so you can find how researchers measured these constructs, concepts and dependent variables you work with. So, how do they measure and what kind of tests inventories did they apply or did they even develop their own measurements?
Very important nowadays, if you do studies that involve human subjects, is that you have to take into account ethical considerations. So you should ensure that you provide an informed consent in which you inform the subjects about what data is collected, where this data is processed, how it is stored and so on, and they should be able to give consent or reject this consent or also be able to be forgotten later. So, you need to check that your data collection is GDPR compliant.
And in that sense, also you need to ensure the ethical approval of your study when you collect personal data. So, what are classical ways of data collection that most of you might know. I will just give some considerations about those.
Of course, most well-known are questionnaires, and they come in different forms and shapes, the same with interviews, observations, tests, secondary data analysis. You also can work with focus groups if you work with different products or a product to shape the product. So, the data collection and the type of data you collect, of course also are related to the phase of your research.
And of course, you have a variety of means to collect that data nowadays, also with digital means. So online, face-to-face via telephone interviews, you have to consider what kind of data storage is necessary. For example, if you collect video data, you need huge amounts of storage available.
You need to consider how you anonymize your data and further process your data. And best you already can plan how you analyze your data before you collect the data. Because this has also an implication on how you structure your data collection.
And always keep in mind to relate the data collection to your research questions. So, you should only collect the data you really need to answer your research questions. So, what are typical considerations when you use a questionnaire?
As Cohen puts here, basically questionnaires are characterized to have a large sample or population. Advantages really are they can be cheap, reliable, they can be constructed very in a valid way. They are quick and easy to complete.
And there is a huge variety of questionnaires already available. So, when looking at the overall structure of the data collection process, you should look at first starting from the purpose of your data collection and check for ethical clearance. So, what data do you need to answer your research questions and purpose?
You structure this typically in the data management plan in which you specify what kind of data will be collected. So, you have clear definitions of the variables and measures you use from that. From that you define a sampling frame.
So how many subjects do you need to collect enough data to be able to draw valid conclusions? So, and connecting to that also you have to define the scales, topics and the clusters. So how many questions probably do you need to understand if a person has a certain trade or is anxious on in a certain situation, you probably need a variety of items or a list of items that consistently measure with the same construct.
So, you have then to define the items of your questionnaire and test already also to define a data analysis script. So, there is a variety of questions you have to take into account. In a lot of cases, the easy way is if you find validated questionnaires that you can consider for your data collection.
So, questionnaires come in a variety of forms and shapes. You can have open-ended questions; you can do sentence completion questionnaires; you can do multiple-choice questions that most of you may know; you can do ranking of items. You can do rating scales, of judging the importance of things; you can have matrix questionnaires.
But there are a few things you have to consider. In any case, if you use these, for example, and often discussed problem is the scale. So, let's say you use a typical Likert scale item, with one to five alternatives or five alternatives from one to five.
An important issue is how you construct the scale. So, what are the values you give to a scale? So, you should use realistic alternatives and the end points of your scale are important.
So, let's say if you use extremes like terrible, disgusting or super, these extremes tend to draw people away or people have a natural trend to go to the middle. So, you need to design your alternatives on the questionnaire well. And design it probably also in a way that you get equal intervals between the different values which has an influence on your results.
So, another well-known method are interviews which are typically more in a flexible design context. You can also combine them with questionnaires, of course, as another method, but interviews typically do something different. And this is nicely defined by Hochschild who notes that interviews can do what surveys cannot, which is to explore issues in depth, to see how and why people frame their ideas in the ways they do, how and why they make connections between ideas, values, events, opinions, behaviors, etc.
So, the interview is really an instrument to understand much more about the how and the why. And you can use it, of course, in combination, using a smaller sample of your big population to understand in depth your research questions. and there are different ways to do interviews.
So, you can have a very informal conversational interview, which is you come up with spontaneous questions, no specific sequence of topics predefined. It's an open conversation. The advantage of such a form of interview is that it's very flexible and you get a broad analysis of the field.
This is often used and can be very helpful if you're in a very early and exploratory stage. Then, of course, you also have like semi-structured interviews and all forms in between, but basically you have to think about: how much of my topics are predetermined and how much can you structure the interview in advance. So, for example, in the standardized open interview, which is also called semi-structured interview, you have the topics and questions determine before.
You know what you're going to ask. This reduces interviewer effects and interviewee effects. It gives you a better way to analyze the data.
And of course, this also makes the interviews you have with different people comparable. And the extreme form of that is really that everything is fixed and you want to standardize as much of the data collection process as possible. So, another aspect, when you do your data collection, and I gave you now two examples, one, for a typical quantitative method, which is the questionnaire, one for typical qualitative method, is to understand human factors in your process of data collection.
So, a little example I want to give you here, is understanding what kind of data collection you need to understand or capture the level of knowledge which is often measured in, like learning related tools or performance issues related to the human variation. So, you can, of course, use questionnaires to design quizzes, exams. You can do interviews to understand if a person has learned something, you can give persons assignments, let them write essays or do them multiple choice questionnaires.
You can give them practical exercises or really ask them to apply the competencies they have learned in a project, for example. So, you see the range of activities from answering simple quizzes to writing essays to really doing a complex project, this asks for, or, this requests to recall and perform different activities by the human. And I think this is a very important aspect when also measuring the performance or trying to measure human participation or human factor components in your study and in your research.
So, you really have to understand what you want to measure and use the appropriate method for that. And a very interesting way to do that also is, of course, to translate the measures into certain tasks. So, of course, you can observe subjects in a certain situation to learn about what they're doing and how they're working together, but you can, of course, also design the task to use specific or trigger specific behavior in situations.
So, to give a specific task on which subjects probably need to work together, and that, of course, gives you a much better insight into a specific behavior, or you can focus better on what you want to measure. And the same you can do in different contexts and settings. Beside those basic methods, there are a lot of different additional methods for data collection.
When you work with observations, you have to take into account coding schemes. So, you should check if the raters or the observers observe the similar behavior and classify the behavior in the same way. You can work with semantic and differential scales, which is a kind of Likert scale, but you have semantic extremes on your scales and you ask people to judge an object or a subject matter considering these scales.
You can also use secondary data analysis, so use existing data. There are huge repositories of research data out there and it might be necessary for you to do some meta-analysis or understand previous data in that context. And as I mentioned before, for a lot of dependent variables and a lot of measures, you ‘ll find standard inventories.
And this is a very important source to consider when you plan your data collection.
