Assignment IV: Sampling; Measurement, Reliability, and Validity; Data Collection and Descriptive Statistics

Instructions:  This is a three part assignment on Sampling, Levels of Measurement, and Reliability and Validity.  Please read the entire lectures for each part and answer the questions listed below each of the three topics.  You will be given 100 points for this assignment; ten points per question.  Upload the completed assignment to me in a document form for grading. 

Sampling and Generalizability:

Imagine that you are assigned the task of measuring the general attitude of college students toward unrestricted searches of their lockers for drugs.  You are already enough of a research expert to know you will have to develop some kind of questionnaire and be sure it covers the important content areas and is easy to administer and score.  After all that preliminary work has been done, you are faced with the most important question:  Whom will you ask to complete the questionnaire all 4,500 students in all the colleges throughout the state?  You cannot do that because it would be too expensive.  Will you ask students at only those colleges where there is reportedly a drug problem?  You cannot do that either.  It is too likely that there also are drugs in the colleges that have not been identified as problem schools.  How about asking only seniors because they are supposed to know what is going on about town?  You cannot do that because freshmen, sophomores, and juniors use drugs as well.  What do you do?

These are decisions that cannot be taken lightly.  The success of any project depends on the way in which you select the people who will participate in your study…. whether you will be distributing a questionnaire or administering a treatment you think will improve memory in older people.  This assignment discusses various ways of selecting people to participate in research projects and the importance of the selection process to the research outcomes.  It is all about populations, samples, and sampling.

Population and Samples:
If everyone in the population cannot be tested, then the only other choice is to select a sample, or a subset of that population. Good sampling techniques include maximizing the degree to which this selected group will represent the population.
A population is a group of potential participants to whom you want to generalize the results of a study. A sample is a subset of that population. And generalizability is the name of the game; only when the results can be generalized from a sample to a population do the results of research have meaning beyond the limited setting in which they were originally obtained. When results are generalizable, they can be applied to different populations with the same characteristics ini different settings. When results are not generalizable (when the sample selected is not an accurate representation of the population, the results are applicable only to the people in the same sample who participated in the original research not to any others.
For example, if your want to find out about college students’ attitudes toward locker researches, one class of senior honors criminal justice students could be given the questionnaire. But how much are they like the general population of students who attend all the colleges in the state? Probably not much. Or 10% of the female freshmen and sophomore girls from all the colleges could be asked the same questions. This selection encompasses a far larger group than just the 30 or so students in the criminal justice class, but how representative are they? Once again, not very.
Our task is to devise a plan to ensure that the sample of students selected is representative of all students throughout the state. If this goal is reached, then the results can be generalized to the entire population with a high degree of confidence, even when using a small percentage of the 4,500 college students. In other words, if you select your sample correctly, the results can be generalized. How will you know if you are doing the job right?
One way to do a self-check is to ask yourself this question: Does the sample I selected from the population appear to have all the characteristics of the population, in the same proportion? Is the sample, in effect, a mini population?
To understand sampling, you first need to distinguish between two general sampling strategies: probability and non-probability. With Probability sampling, the likelihood of any one member of the population being selected is known. If there are 4,500 students in all the colleges, and if there are 1,000 seniors, then the odds of selecting one senior as part of the sample is 1,000: 4.500, or 0.22.
In non-probability sampling, the likelihood of selecting any one member from the population is not known. For example, if you do not know how many children are enrolled in the state’s colleges, then the likelihood of any one being selected cannot be computed.

Summary: 

Although some people might not agree with you on your selection of topics to study, what you choose is your business as long as you can provide a reasonable rationale to support what you are doing. Your selection of a sample, however, is another story entirely. There are many right ways, and then there is the wrong way. If you choose the wrong way (where you are arbitrary and follow not plan), you could very well sabotage your entire research effort because your results might have no generalizability and, therefore, no usefulness to the scientific community.

Your assignment: Please answer the following questions:
1. Why is sampling important to the success of research in the social and behavior sciences?
2. Name and discuss four probability sampling strategies.
3. Name and discuss two non-probability sampling strategies>
4. What is the difference between a probability and a non-probability sampling strategy? Provide an example of each. Also, what are the advantages and disadvantages of each type of sample?
5. With a population of 10,000 children (50% girls, 70% white and 30% non-white, and 57% single-parent family and 43% dual-parent family), what steps would you use to select a representative sample size of 150?
6. What are the implications of using a sample that is too big or a sample that is too small?

Lecture on Nominal, Ordinal, and Interval-Ratio Levels of Measurement

Often we design numbers to measure a concept, such as fear of crime or support for the police or capital punishment. Numbers are assigned t=in order to make the data amenable to statistical analysis. The numbers are used as a code. Statistically, the question is “Can we use mathematics to now analyze this code that we have established?” Does it make sense to treat the numbers as such and perform arithmetic operations on them. This code is called the level of measurement. It involves converting concepts to numerical data. There are four levels and each has different attributes. The levels of measurement are cumulative, however, like the steps on a ladder. You have to step on the first step to reach the second, and so on. Each succeeding level automatically possesses the attributes of the level preceding it, plus another distinct one.
The Nominal Level of Measurement:
The nominal level involves the process of classifying data into categories. When we classify respondents by race or sex, we are using nominal measurement. (i.e., 1 for male, 2 for female). The nominal level of measurement follows three basic rules:
1. The list of categories must be exhaustive and cover all the types of observations made.
2. The categories must be mutually exclusive. Each observation can only be classified in one way.
3. No ordering is present in the list of categories. The order is arbitrary, and no one
classification is superior to another.
At the nominal level, the numbers are actually substitutes for names and serve as a numerical label. It does not make sense to add them together or perform any other mathematical function on them. The only legitimate summarizing statistic is the largest category (or mode). It does not make sense to discuss the mean (average ) or median (midpoint) with nominal data. It cannot be summed and divided, nor can it be ranked in order from highest to lowest.

The ordinal level of measurement describes variables that can be ordered along some type of continuum.  Not only can these values be placed in categories, but they can be ordered as well  For this reason, the ordinal level of measurement often refers to variables as rankings of various outcomes, even if only two categories are involved, such as big and little.

For example, you already saw that Tall and Short are two possible outcomes when height is measured.  These are ordinal because they reflect ranking along the continuum of height.  Your rank in your high school graduating class was based (probably)  on grade point average.  You can be 1st or 300 or 150th of 300.  You will notice that you cannot tell anything about the absolute GPA score from that ranking but only the position relative to others.  You could be ranked 1st of 300 and have a GPA of 3.75 or be ranked 150th of 300 and have a GPA of 3.90.

Interval-ratio Level of Measurement:

This level of measurement describes variables that have equal intervals between then. They allow us  to determine the difference between points along the same type of continuum that we mentioned in the description of ordinal information.  For example, the difference between 30″ and 40″ is the same as the difference between 70″ and 80″.  There is a 10″ difference.  In other words, an inch is an inch. Put simply, using an interval scale, we can tell the difference between points along a continuum, but with ordinal scales we cannot.  Although an interval level scale is more precise and conveys more information than a nominal or ordinal level scale, you must be cautious about how you interpret the actual values along the scale.

Ratio describes variables that have equal intervals between them but also have an absolute zero. In the simplest terms, this means they are variables for which one possible value is zero or the actual absence of the variable or trait is possible.  This is indeed an interesting level of measurement.  It is by far the most precise.  It is the most interesting scale of the four discussed for other reasons:  First, the zero value is not an arbitrary one.  For example, you might think that because temperature in Celsius units has a zero point, it is ratio in nature.  True, it does have a zero point, but that zero is arbitrary.  A temperature of O degree C does not represent the absence of molecules bumping off one another creating heat (the non technical definition of temperature and my apologies to Lord Kelvin).  But the Kelvin scale of temperature does have a theoretical absolute zero (about-275 degree C), where there is no molecular activity, and here is a true zero or an absence of whatever is being measured (Molecular activity)

.
Your assignment:
1. Identify the level of measurement associated with each of the variables listed below:
a. Number of words correct as a spelling test score
b. The name of the neighborhood you live in
c. Age in years
d. Color expressed as wavelength
2. Name the four levels of measurements and provide an example of each

Reliability and Validity:  Why They are Very, Very Important: 

You have the sexiest looking car on the road, but if the tires are out of balance, you can forget good handling and a comfortable ride.  Where the rubber meets the road is crucial.

In the same way, you can have the most imaginative research question with a well defined, clearly articulated hypothesis, but if the tools you use to measure the behavior you want to study are faulty, you can forget your plans for success.  the reliability (or the consistency) and validity (or the does what it should qualities) of a measurement instrument are essential because the absence of these qualities could explain why you act incorrectly in accepting or rejecting your research hypothesis.

With that in mind, assessment tools must be reliable and valid; otherwise, the research hypothesis you reject may be correct but you will never know it.  

Reliability and validity are your first lines of defense against spurious and incorrect conclusions.  If the instrument fails, then everything else down the line fails as well.  

Something that is reliable will perform in the future as it has in the past. Reliability occurs when a test measures the same thing more than once and results in the same outcomes.

Some important ways to increase reliability include the following:

1.  Increase the number of items or observations.  The larger the sample from the  universe of behavior you are investigating, the more likely that the sample will be representative and reliable.

2.  Eliminate items that are unclear.  An unclear item is unreliable regardless of knowledge or ability level or individual traits, people respond to it differently at different times.

3.Standardize the conditions under which the test is taken.

4.  Moderate the degree of difficulty of the tests.

5.  Minimize the effects of external events.

6.  Standardize instructions.

7.  Maintain consistent scoring procedures.

How is reliability is measured?

A very useful and easy to understand statistical concept is called correlation and the measure of correlation, the correlation coefficient is used to measure reliability.  Another way to measure the reliability of a test is to give the test to a group of people at one point in time and then give the same test to the same group of people at a second point in time, say four months later.  You end up with two scores for each person.  When the scores tend to change similarly and in the same direction, the correlation tends to be positive and the reliability high.  If scores remain the same relative position on all tests, it is reliable.

How is Validity measured?

A good test must be reliable and valid.  Remember consistency, stability, and predictability for reliability? Well truthfulness, accuracy, authenticity, genuineness, and soundness for validity. Remember that the validity of an instrument is foten defined within the context of how the test is being used.  Here are the three aspects of validity:

1.  Validity refers to the results of a test, not to the test itself.  

2.  Validity is never a question of all or non.  The results of a test are not just valid or invalid.  This progression occurs in degrees from low validity to high validity.

3.  the validity of the results of a test must be interpreted within the context in which the test occurs.  If this were not the case, everything could be deemed to be valid just by changing its name.

The relationship between reliability and validity is straightforward and easy to understand:  A test can be reliable but not valid, but a test cannot be valid without first being reliable.  In other words, reliability is a necessary, but not sufficient condition of validity. 

Using The moral of this story is :  Use a test with established and acceptable levels of reliability and validity.  If you cannot find one, do one of two things.  Develop one for your thesis or dissertation, (which is a huge undertaking), and do no more than that, or change what you are measuring so you  are sure that what you ask can be answered in a fair and unbiased fashion.  There are no two ways about it; the measurement process is a critical part of putting together a research project and seeing it to fruition.  this part of the research project is especially important because a test without the appropriate levels of reliability or validity is of no use to you or anybody else. Using poorly designed measurement tools leads you down the path of never knowing whether you are on the right track or never really accurately measuring what you want.  Use your good sense and look around for instruments that have already been shown to have respectable levels of reliability and validity.  It will save you time, trouble, and endless headaches.

Your assignment:

1.  What is the relationship between reliability and validity?

2.  Describe two ways in which the reliability of a test can be established and explain the purpose of each.

Reference:  Salkind, N., (2012).  Exploring Research Methods, 7ed., Wadsworth. 

Quiz I: The Research Methods Process

Answer each of the questions below as accurately as you can.  You are not allowed to write just one sentence for each question.  Think critically and elaborate fully on each question. 

What are the six steps to conducting research?

Name and explain each step thoroughly. 

Final Exam: Class Discussion Assignment

Correlational Research is different from historical and descriptive research.  It provides some indication as to how two or more things are related to one another or, in effect, what they share or have in common, or how well a specific outcome might be predicted by one or more pieces of information.

One of the most important points about correlational research is that while it examines relationships between variables, it in no way implies that one causes changes in the other.  In other words, correlation and prediction examine associations but not causal relationships, wherein a change in one factor directly influences a change in another.

Give an example each of how correlational research differs from historical and descriptive research and how it is related to both historical and descriptive research.  You must write at least one complete paragraph when answering this question, and you must reply to at least two more students’ post.  one or two line answers will NOT be accepted and will not be given credit.