How to Make Your Research Method the Best it Can be

PILOT STUDIES

Whatever research method you have chosen to employ, you will find that your exact procedure and resources will develop as you take your research forward. No researcher can decide on the perfect procedure on the first draft of their research proposal. Instead it should evolve as more information becomes available. However, once you start collecting data, in most methods it is imperative that the procedure remains constant. To ensure that the optimum procedure has been selected it is essential that a pilot study is performed. Sommer and Sommer (2002, p9) describe this effectively.

“No matter how carefully you reviewed the background literature and designed the study, there is no substitute for a pilot study before the actual data are collected. A pilot study is a preliminary use of a procedure designed to identify problems and omissions before the actual study is conducted. Remember Murphy’s Law – if something can go wrong, it will. Even the best designed experiments will have flaws, and the most carefully developed interview sheet will have ambiguous questions or words that the respondent will misinterpret. There is no way to predict all potential problems in advance. Not even the most experienced survey researcher can compose a perfect questionnaire on the first try. Survey research firms test out their questions in a pilot study before beginning formal data collection. You should plan to do the same in your own research. Set aside a designated time for this after the procedures have been developed. Don’t skip the pilot study in a rush to collect data. A little effort put into it will pay big dividends in improving the precision, reliability, and validity of the data collected in the actual study.”

The pilot study doesn’t have to be a major task; just make sure you get the most from it.  So, for example, if you want to pilot the questionnaire you have designed, get a small sample of people to use it and question them afterwards on their understanding of the questions and answer options.  Alternatively you can ask them to say aloud what they are thinking as they complete the form (they may feel a little self-conscious doing this).  Using their comments you can refine or reword questions, leave out some or put new ones in.

Finally, make sure that you report on this in the Method section of your report.  It doesn’t have to be too detailed.  Something along the lines of:

A short pilot study with 5 participants resulted in the amendment of question 5, the removal of questions 3 and 10, and the inclusion of an additional question (see appendix A for details).

 

Sommer, R. & Sommer, B. (2002) A Practical Guide to Behavioural Research. Tools and Techniques. 5th Ed. Oxford: Oxford University Press

Image: digitalart / FreeDigitalPhotos.net

Choosing a Research Method

Once you have decided on your research question and started to read the existing literature on the topic, you must decide what approach you are going to employ to further your research. There are a variety of tools available to you to gather the data you need to answer your research question such as interviews, surveys, case studies, design and test, experiments etc. You will need to decide which tool is appropriate for the research questions and whether you have the skills to use it effectively.

No technique is perfect. For example experimentation suffers from artificiality, and observation may suffer from differences in opinion between observers. However, each tool will have some advantages too such as the economies of scale seen with online surveys. In some instances the most appropriate procedure cannot be used. In this situation a combination of other tools may be the best compromise. Ultimately though, the choice of tool should be decided by the question to be answered and the time, resources and skills available.

Table 1 below is adapted from Sommer and Sommer (2002, p6) and may help you choose the tool appropriate for your research question. For some questions more than one tool may be needed.

Table 1: choosing among research tools

PROBLEM	APPROACH	RESEARCH TOOL
To obtain reliable information under controlled conditions	Test people, equipment or procedures in a laboratory	Laboratory experimentSimulation
To find out how people behave in public	Watch them	Systematic observation
To find out how people behave in private	Ask them to keep diaries	Personal documents
To learn what people think	Ask them	Focus groupInterviewQuestionnaire/survey Attitude scale
To find patterns in written or visual material	Systematic tabulation (categorisation)	Content analysis (qualitative techniques)
To understand an unusual event or specific instantiation	Detailed and lengthy investigation	Case study
To find out what people have done in the past	Check public records	Archival research
To discover whether a function is achievable	Build it in the lab and test against set criteria	Design and test
To generate new guidelines or theories from existing knowledge	Read existing literature and synthesise conclusions or new meaning which may be applied to new situations	Discursive

Sommer, R. & Sommer, B. (2002) A Practical Guide to Behavioural Research. Tools and Techniques. 5th Ed. Oxford: Oxford University Press

Image: Stuart Miles / FreeDigitalPhotos.net