Week 9 & 10: Quantitative Descriptive Studies & T/Q Experiments

Week 9: Quantitative Descriptive Studies

Lauer & Asher:

Q1: The appropriate purposes for quantitative descriptive studies include "to generate variables, to operationally define them, and to develop an early understanding of their relationships" (L&A 82). It is an attempt to understand and explain any phenomena in research.These studies go beyond case studies/ethnographies "to isolate important variables developed by these studies, to define them further, and to quantify them at least roughly, if not with some accuracy, and to interrelate them" (L&A 82).
Q2: Subjects are selected after considering the number/type of variables the researcher wishes to study. A large # of subjects are necessary - at least 10X's as many subjects as variables.
Q3: Data is collected and analyzed by multiple methods, including any methods that give "researchers the data that they need to quantify and interrelate the variables they wish to study" (L&A 85). The choice of variables to observe and intercorrelate is important and is broken into independent/dependent. To relate the variables, a variety of statistical analyses can be used, including interval data, frequency counts, proportions and percentages, Chi square, Phi coefficient. To show interval with interval variables - correlational analysis. To relate interval variables to nominal variables - variance/F-test, t-test, point biserial correlation; To relate nominal to rank-ordered variables - Wilcoxon T, Mann-Whitney U. To relate rank-ordered variables to other rank-ordered variables to one group - Spearman's rho.
Q4: The kinds of generalizations possible include the ability to build theories about the composing process, the contexts of writing, the pedagogy of writing, but it can usually not be used with cause-and-effect relationships among variables (L&A 102).

Farber:

Q1: The purpose of this quantitative descriptive study is to examine the initial construct of nanoscale science and technology in written popular media and thereby explore the emergence of one new scientific concept and how it has endured in public discourse. Farber discusses his hope to study how science and technology are represented in public discourse and to build public recognition and awareness of science and technology.
Q2: Subjects selection came from studying writings in the popular press that specifically examined how science (nanoscience) emerged. Using these popular accounts of science, this study sought to understand how new representations in science and technology emerged and endured in popular media, and predicts that new representations would emerge similar to those described by the theories of change in scientific communities.
Q3: Data is collected and analyzed by searching key words that at first generated 885 articles, and then eliminating and categorizing them – so he studied the propositional content, the grammatical structure and the discourse analysis. Overall he looked for similar themes, rhemes (information presented after the verb), and topics. He then found that there were 39 representations of nanotechnology and nanoscience in 262 occurrences. Each representation occurred 6.89 times. Since this type of study is a composition study, there should be correlation between two or more raters or coders and a determination of internal-consistency reliability, but there is not.
Q4: Farber does make generalizations, as when he says that although predictions about nanotechnology are broad ranging, actual initial applications of nanotechnology have been most prominent in electronic computer-memory technology and in polymer coatings. But the methods of acquiring the findings seems flawed, as when he broadly states that the “process of presenting technical information for general audiences can be enabled by combining social and technical approaches” (161). The solid research that would allow such generalizations seems missing from the research.

Golen:

Q1: The purpose for this quantitative descriptive study is to determine the barriers most frequently encountered by affected listening effectiveness in business college students, as given in the Watson and Smeltzer (1984) study (expanding barriers from 14 to 25). Golen mentions there are a large # of potential barriers. Noting that no studies have determined factors of barriers to effective listening, he briefly examines several studies on listening behavior.
Q2: Using the backdrop of a major southwest state university, he examined 3 large business communication lecture sections of approx. 400 students, with 33 breakout sections of 35 students each. Golen used a random sample of 10 sections and only one of these students who attended the large lecture was included in the study. 279 questionnaires were collected and analyzed – each containing questions on 25 barriers to effective listening; the SAS was used to analyze the data. The barriers themselves were selected by identifying the common barriers listed in a review of listening literature.
Q3: Data collection: 279 students were in the study. The questionnaire contained a 5 point Likert-type scale, and an internal consistency measure – coefficient alpha – “indicated that the questionnaire was reliable (alpha equal to .79)” (28). The questionnaires were broken into 3 areas: the relative frequency of listening barriers, the results of a factor analysis, and the relationship between demographic variables and factors. Then six independent listening barrier factors were generated and determining whether there were any significant differences among the factors based on demographic information was determined. The study did note that “the results revealed no significant interaction effects. However, “there was a significant main effect for the students’ sex for two out of the six factors” (33). No further explanation was given.
Q4: Using the research from other studies, Golen generalized about their findings. I am not quite sure why, all of a sudden, these studies became so crucial to his own study. The end of his study was basically filled with the findings of others.

Week 10: Experiments
Lauer & Asher, "True Experiments":

The great advantage of true experiments is that it can suggest cause-and-effect relationships without threats to external validity (other than Mortality). They never, however, suggest certainty.
A. A true experiment seeks to demonstrate that "the researcher actively intervenes systematically to change or withhold or apply a treatment (or several) to determine what its effect is on criterion variables" (152). Uses randomization in which subjects are allocated to treatment and control groups. One treatment method applied to experimental groups and another (traditional method) to control group. All are evaluated using at least one common measure.
B. The control group affects subject selection in that they help establish randomization, or a situation in which groups can be “not unequal”: “If subjects are randomly allocated to two or more groups, the researcher can generalize that, over a large number of random allocations, all the groups of subjects will be expected to be identical on all variables” (155). The control group acts as a standard, typical condition, a foil in which the special treatment is absent.
C. Independent (differences prior to research) and dependent variables (introduced by researchers) impact data collection and analysis in that in true experiments, ALL variables, known and unknown, upon which humans can differ will be expected to be EQUAL in all treatment and control groups, both at the time of randomization and for the future, unless there is unequal intervention treatment.

True experiments must be done in natural environments. Thus researchers must have enough thick description of their treatment conditions to increase theoretical knowledge and allow for replication. The cause-and-effect relationship among new variables is more evident in this type of experiment, but a limitation on true experiment is its focus on limited variables and structured situations in order to determine possible cause/effect relationships.

Lauer & Asher, "Quasi Experiments":

A. A quasi-experiment seeks to demonstrate cause-and-effect inferences using subjects, treatments and criterion measurements. Can be classified into strong or weak categories, based on the equality or inequality of the groups as established by the pretest.
B. The control group affects subject selection in that there is no randomization of subjects to groups, so there is the possibility of internal validity threats (because already established groups stay in place).
C. Data collection and analysis of independent/dependent variables depends upon whether the quasi-experiment is classified as strong or weak. In strong quasi-experiments, the research can consider equal only those explicit variables measured by the pretests. (Not of great concern because of the relatively small # of variables that correlate with the criterion variables). Weak quasi-experiments rely upon initially unequal groups, so they differ on one more variables. Unequal groups acted upon by equal treatments will still be equally unequal on the criterion variables.


Carroll, Smith-Kerker, Ford & Mazuro-Rimetz:

This study is broken into two sections: Minimal Manual and Learn by Doing/by Book.
A. (Experiment 1). Minimal Manual: This experiment seeks to address self-instruction manuals for people learning computing devices. Specifically, the Minimal Manual is mentioned, which apparently affords a more efficient learning progress (it addresses the preference to start immediately on real tasks, the preference to skip reading, and errors as a consequence of active orientation in learning). The Minimal Manual seeks to capitalize on learning styles and strategies. (As an aside, this experiment bores me and it is possible that the authors may lack a personality – just one will do.) The manual is designed to be bare-bones (45 pages), without undue verbiage and is tested using Guided Exploration. They analyzed the training situation, including the errors made, and then re-worked the manual to be more open-ended, thus better resembling real work and maintaining learner motivation.
B. I could not find the subject selection, much less any mention of a randomization of subjects. Hence, this study is not a true experiment. No mention is made of using intact groups, so I doubt it is a quasi-experiment either.
C. As the study is poorly designed, it makes no difference whether they use independent or dependent variables and how these impact data collection and analysis. Criterion variables are unclear. There also should be correlation between two or more raters or coders and a determination of internal-consistency reliability, but there is no mention of such.

A. (Experiment 2). Learn while Doing (LBD)/Learn by Book (LBB): The focus is to contrast between a commercially developed, standard self-instruction manual (SS) and the Minimal Manual (MM). As there is no randomization of subjects, this is not a true experiment. As there is no use of intact groups, this study is not a quasi-experiment – though the authors may believe that it is because it does seek to establish cause-and-effect relationships.
B. There is no control group or randomization of subjects. Subjects (see p. 90) were sent from a temporary agency (thus, their subjects do not fall into either real or quasi) and consisted of 32 subjects.
C. It mentions two independent variables – the manual (either SS or MM), and instructions, either LWD or LBB. The LWD learners were given 5 hrs. to perform a series of tasks using the system, and the LDD were given 3 hrs. to use the manual in order to learn the system. They were separately given 2 hrs. to perform a series of tasks using the system. I have to wonder if allowing a variance of time between the LWD and LDD is consistent with the quasi-experiment. Again, no inter-related reliability, no clear criterion variables, no established reliability.


Notarantonio & Cohen:

A. This study on the effect of the Open and Dominant communication styles on sales effectiveness seeks to demonstrate the reliability and validity of the communicator style construct. According to the introduction, respondents to a 42-item questionnaire rated the Dominant/low Open and low Dominant/high Open salesperson more favorably than the high Dominant/high Open or Low Dominant/low Open salesperson. The study also considers various studies (Williams and Sprio; Norton, Bednar; Sheth, etc.) in terms of perceived communicator styles and their effect. This study was a quasi-experiment, though again the subjects were not really an already established group which must stay in place. It did use the same college and the same major, but a quasi-experiment implies a teacher, for example, who must study all classmates in a particular class because he or she can not send out the students they do not wish to study. In this study, these majors could be assembled into various groups and were not necessarily required to be tested together.
B. Subject selection consisted of 80 undergraduate business administration students at Bryant College. Demographics of sex, age range and college year were given. Subjects were randomly assigned to conditions, and to time and day (I wonder if they felt that was the same as the randomization of subjects as required in true experiments).
C. Data collection and analysis consisted of asking the aforementioned subjects to watch four videotapes (which were pretested by showing them to small groups of subjects – 10 each per group), and then giving respondents a communicatory style measure re: openness and dominance, in which they replied using a 7 point scale of agreement regarding the traits. The pretest consisted of a self-report of Nortan’s communicator style measure. The hypothesis (the more open a salesperson was, the more effective the individual would be in selling the product and the more positively she or he would be perceived by others) of the authors of the study (to empirically test whether or not different communicator styles affect the perceptions of the sales interactions) proved accurate. After viewing the tape, subjects were asked to complete the 42-item questionnaire on communicator style measures. 6 composite scores were identified, including a general attitude toward helping, product perceptions, customer/salesperson interactions, respondent purchasing behavior, product purchase probability, and salesperson perceptions. The analysis of this data consisted of a two-way ANOVAS run with Openness and Dominance as independent variables for the measure of openness, dominance and the six composite measures. All measures ranged from 1 to 7, with 7 strongly agree.

Kroll:

A. Here students in grades 5, 7, 9, 11, and college were taught a board game, after which they watched a demonstration film and then wrote directions for the game. The study provided a rich description for the growth of students’ informative writing skills at the aforementioned grade levels. This study included a review of three different procedures that have been employed in eliciting children’s explanations of games, with this study following the third method – asking students to explain a new game that they have just learned to play themselves.
B. Subject selection: students came from 4 schools in a small city in central Iowa and freshmen at Iowa State University (24 in grade 5, 26 in grade 7, 19 in grade 9, 27 in grade 11, 27 college freshmen). No mention is made of random selection, but the quotient in Lauer & Asher (10X’s as many subjects as variables) for a descriptive quantitative study seems to be met. A quasi-experiment has already established groups stay in place – and perhaps they consider this grouping from each grade level to have already been established, but no evidence is given of such. It is neither a randomized study, so it is not a true experiment, nor a true quasi-experiment. Students were “selected” based on their ability to follow the rules, to meet “normal” testing requirements and to speak English. (Also, students also had to watch the film a second time 3 days later and qualified for the study by earning 90% on the quiz with 16 being disqualified. College students were particularly mistreated because they were never allowed to really watch the game, were tested anyway, did poorly, and no one seemed bothered about it).
C. Data collection and analysis: the effect of grade level on game information scores was analyzed with a one-way ANOVA. The statistical significance of differences between mean information scores for adjacent grade levels was assessed with Scheffe post hoc tests. In addition to total scores for overall informativeness, scores for each of the ten individual game elements were given in order to assess the extent to which the scores reflected a development trend. A nonparametric trend analysis was employed and the resulting z statistics were used in order that the ten game elements from those exhibiting the strongest developmental trends to those with the weakest trends. A chi-square test was performed to assess the statistical significance of the association between grade level and students’ listing of the game pieces, between grade level and students’ mentioning the object of the game, and between grade level and students’ use of the three explanatory approaches. The strength of the relationship between the variables was assessed with Cramer’s statistics. Geez, and all of this for a game!

The results demonstrated a strong effect on the informativeness of students’ explanations. They also used a series of nonparametric trend analyses to determine the degree to which subjects’ explanations of each of the ten game elements improved across grade levels.