STUDY PLANNING & DESIGN: A SERIES OF ACTION STEPS
SECTION 02 – CHOOSING THE METHOD
Upon completion of this section, you will be able to (based upon your objectives) write data collection and data analysis procedures for your research project.
ACTION 1: DETERMINE THE APPROPRIATE DATA TO COLLECT
WHAT IS DATA? WHAT IS INFORMATION?
Data can be distinguished from information, even though sometimes these words are used interchangeably.
To illustrate how each concept is different, you can look at word origins. For instance, the word data derives from the Latin word datum, which means “something given.” The idea that data is “given” mirrors how many researchers understand data, which is something that originates in perception, then is written down to make sure it is not forgotten, then is organized or corrected by careful researchers who have various concerns about their data’s quality. From this point of view, the elements of your visual field right now constitute data, it’s just that you haven’t gone through the trouble of storing it in writing and organizing it appropriately.
WHY DISTINGUISH DATA FROM INFORMATION?
The goal for this section is to write collection and analysis plans for your data. Distinguishing data from information sheds light on why you will need to come up with an analysis procedure, which is the same thing as a plan to take your data and derive (useful) information from it.
Information is an old word that originates from the 1300s. The root word inform meant roughly what it means now, which is communicating states of affairs, but originally it was meant mostly with respect to education or instruction. Embedded in this notion of information is usefulness, like the use of facts to enhance understanding. To illustrate how facts might be presented to enhance understanding, or data might be used to derive information, consider the case of someone who decides to track their spending. They might write in a spreadsheet all their expenditures for the month along with the specific date—that would be data. But if this person were to use this data by deriving, say, the average spending per week, or the total money spent during the month, the result would be information (and in this example, the information is more useful to your budgeting than the raw facts of each purchase).
COLLECTING DATA
Another way to put it is that data is collected, whereas information is derived. You can collect data through either gathering data that is already existing at the time the research is conceived, or through planning to collect data that is not yet in existence. Returning to the spending example, the expenditure data might already exist, since the person already wrote down these expenditures for a prior month, or used a budgeting program like Mint that automatically keeps track of spending. If that’s true, research done using this data set would be labeled a retrospective study. However, if this person is like most people, then they did not keep a rigorous track of their spending for any month. In that case, they would need to plan to record these facts about spending in the coming months, and research using that data would be called a prospective study.
In emergency dispatch, there are many sources of already-existing, or “retrospective” data, many of which are included in the table below. ProQA records, ePCR records— really any type of data with the word “record” in it that has already been collected before the study starts—would be considered retrospective data. Keep in mind, even though you don’t have to create these records yourself, that does not mean data collection in this case is necessarily simpler or easier than in prospective studies. Often, data that comes from these sources needs to be “cleaned,” which usually means either outliers are removed, or data entry errors are corrected.
RETROSPECTIVE VS. PROSPECTIVE RESEARCH
A retrospective study differs from a prospective one in the way investigators collect their data. For retrospective studies, the investigators collect data from past records and do not create any data through a future intervention. Prospective studies, meanwhile, employ various strategies to collect data on subjects over a future period.
COMMON TYPES OF DISPATCH DATA
| ProQA records | ePCR (EMS) records | AQUA records | CAD records |
| Hospital/ED data | Agency admin data (time cards, employee info, etc.) | Fire department data | Police/Law Enforcement data |
| ECNS data | Call audio | Survey results | Interview transcripts |
| Observation and field notes | |||
Meanwhile, types of data like interview transcripts, survey results, or observation and field notes, would generally be “prospective,” in that you would probably need to create the data yourself for this specific study. For something like an interview, this would involve finding the interviewees, obtaining their informed consent (more on this in the next section), setting up the recording equipment, conducting the actual interview, transcribing the recordings, etc.
CASE IN POINT - INTERVIEW TRANSCRIPTS
To determine the right data, you must first consider your objective. Consider the final version of the objective from the last section: To describe the reasons Center X emergency dispatchers quit their positions within an eight-month period.
With this objective, interview transcripts are an appropriate choice of data to collect. The word “describe” in the above objective explains why—the data you select should describe rather than measure, and interview transcripts are much more descriptive than most other types of dispatch data listed. As well, you might find the flexibility of an interview format most suited to your objective. If someone says something that you think requires probing or following-up, for instance, then an interview allows you to do so.
An alternative type of data to collect would be survey results. Indeed, this could be a valid way to describe what you’re interested in; however, a potential limitation is that a survey does not allow you to probe respondents if their answers require follow-up.
COMMON CHALLENGE: HOW DO I KNOW COLLECTING CERTAIN DATA WILL ACHIEVE MY OBJECTIVES?
A general answer to this question comes from the following distinction: the data you collect can be quantitative or qualitative. The difference really comes down to facts that report measurements (quantitative data), such as “the house is this number of feet long,” or “the temperature in the room is that many degrees,” and facts that report descriptive words or phrases (qualitative data), such as what people said in relation to a specific question (like “what is the meaning of life?”).
Some people say that qualitative data, in certain situations, is better for answering how or why questions. Certainly, if you are interested in, say, why a group of people decided to become dispatchers, then it seems like a verbal report of some kind is going to be more useful than measurements, such as their ages, heights, or salaries. However, there is a tradeoff, in that the findings of many qualitative studies, even though they might have “depth,” will not have the same broad applicability as well-designed quantitative studies. Another tradeoff is that the data from a qualitative study is generally harder to analyze.
ACTION TWO: PLAN WHERE TO OBTAIN THE DATA
Planning where you will obtain your data is important. Even if you know what kind of data you are looking for, that does not guarantee that you will have access to it.
Remembering the previous distinction made between retrospective (already existing) and prospective (not yet existing) data is helpful here. If your data already exists, you will need to figure out the specific location where you can access this data, which can be at your agency, or somewhere else.
However, if you are going to obtain prospective data, then you should consider at which locations this data creation will happen. For instance, if you are going to gather interview transcripts, where are you going to conduct the interviews? Or, if you are going to create observational notes related to dispatchers and their work, at which agencies are you going to focus your observations?
WHERE TO OBTAIN RETROSPECTIVE DATA
- Internally at my agency
- With or from partner organizations (such as a fire department)
- National Databases (such as NEMSIS or NFIRS)
CASE IN POINT - HOW TO CONDUCT AN INTERVIEW
If you are collecting data through interviews, then you are collecting prospective data. You need to plan where these interviews will be conducted, and how you will identify interviewees. After some thought, you plan to identify the interviewees internally through discussion with your manager. You also get the go-ahead to conduct face-to-face interviews in an office at your dispatch center that is not currently being used.
ACTION THREE: PLAN HOW TO ANALYZE THE DATA
As suggested earlier, data considered by itself is not necessarily instructive, useful, or illuminating. You will need to take the data to-be-collected and derive information. The first step toward accomplishing this feat is coming up with a data analysis plan. Below are some common types of analysis plans that you might want to use in your research:
- Structured review of audio calls
- Descriptive statistics (statistical analysis)
- Interpretation of interview or survey data
- Cause and effect (statistical analysis)
- Comparing groups (statistical analysis)
Going back to the distinction between quantitative and qualitative data, some of these plans are intended for data that is descriptive words and phrases, like an interpretation of interview or survey data. Meanwhile, the plans that are labeled “statistical analysis” are suited for data that consists of measurements.
You will want to consult with an expert or mentor to better understand which plan is going to be appropriate for your data and objectives. Similarly, the precise details of the plan should be figured out in consultation with an expert or mentor.
CASE IN POINT - CONSULTING A MENTOR
You consult a mentor about how to analyze interview transcripts for a study. After some discussion of possible approaches, you decide upon a specific qualitative analysis procedure, one that identifies themes (or patterns) in the data that are relevant to your research objective.
ACTION FOUR: PLAN HOW TO ANALYZE THE DATA
You can think of a data collection procedure as a kind of recipe. For instance, when someone writes a recipe for a certain dish, the intention is to be clear and detailed enough so that other people interested in the dish can replicate the food for themselves. Likewise, when you write a data collection procedure, the goal is to be clear and detailed enough so that interested researchers can replicate your procedure.
Initially, you should organize your procedure as a series of steps. Once you have that initial outline, you should present your work to a mentor or expert for an accountability check (see below). To help you cover the right details, these are the data collection questions that your procedure should answer:
DATA COLLECTION PROCEDURE QUESTIONS
| What data will be collected? |
| How will the data be collected? |
| When will it be collected? |
| Who will be responsible for collecting and recording the data? |
| Where will the collected data be stored? |
| How will you ensure the data is correct? |
ACCOUNTABILITY CHECK
Find a mentor or partner. Describe to them your research problem and your objectives, and present to them your initial outline of data collection steps. Have them check to see if all necessary steps are presented with enough detail.
ACTION FIVE: WRITE A DETAILED DATA ANALYSIS PROCEDURE
In action three, you should have already discussed a data analysis procedure with an expert or mentor. From this interaction, you should have enough information to draft a detailed analysis procedure. Like with the procedure for data collection, we suggest that your procedure undergo an accountability check to make sure that this aspect of your methodology is sound.