What is evidence-based research?

Well, when I hear a scholar or academic ask me where my research is, I have to take a deep breath and hope that my response is politically kind. You see, I am a practitioner first and an academic second. What that means to me is that I want to know what I need to do to help a child or their family make the world more accessible, more pro-socially available so that they have choices. My personal goal is to provide all individuals access to live a life that allows them to make the choices that benefit not only themselves but society as a whole. So, when a second grader is no longer hiding under a desk but is sitting at their desk engaged in their work, then I assume that the world is more accessible. The child can do the work and once they learn how to do the work, then they will be able to choose what they do with that material later. To me, this change from the child sitting under the desk to doing the work is “evidence” that the child has more pro-social choices. But, maybe you think that the child “likes” to sit under the desk. Well, that is a choice but that means the teacher will not know if the child can do the work or not. So, the child is not thinking about others, just themselves. To me, that type of thinking is restricted to being 3-7 years of age and will not promote pro-social interactions as a class, group, society. In other words, evidence of a change in behavior from not being able to participate in a group with others and what they are doing to being able to participate with others is “evidence” of whatever I was doing to change the child’s behavior worked. I have spent more than 50 years making those types of changes, quite a bit of evidence I would say. Plus I have taught others how to also make those types of changes. Such evidence is based on my background in clinical therapy where therapy is doing what is necessary to help the client make changes. This type of efficacy is based on solid theoretical knowledge as I can explain why I do what I do. But where is the research? At this point, I want to say, “Read the literature about language origins, acquisition, as well as philosophy of language!” In philosophy of language, logical inductive and deductive reasoning is used to debate, argue, support, predict outcomes; all a form of research. This type of research is very different from experimental work where there is at least one variable manipulated against a control group without the variable. You see, in my professional background and in most of education, it is almost impossible to do this type of research. Afterall, it is unethical to withhold or not provide a client or learner with what we believe works. So, then how do all of these programs, methods, etc. show evidence of effectiveness? Data! That is right, they collect data, lots of data. If I test a child in October and then offer a program and test again in April, there should be some growth. That data supposedly is “evidence” the child has grown. But was this just normal growth or a result of the program? We don’t really know. So the test items are made very specific and short term memorization and response is measured as a test outcome. But was the increase in test items greater than an academic year or just the fact that the child can memorize for the test? If not, who knows if another approach would not have provided the same or better results?

Let’s return to that second grader (based on a real case). If she doesn’t sit in her desk, she doesn’t do the work, and she isn’t tested. She will not be included in the classroom data. However, with some language-based strategies over about 20 minutes, she sat in her desk, and within minutes wrote an entire paragraph without spelling or grammatical mistakes. For me, I had “evidence” of what she needed and what I could provide for her to be pro-socially part of the classroom. Unfortunately, the teacher had already decided the second grader was not capable of what others in the classroom could do, so the child was referred to special education. She still could not do the reading, writing, or math at grade level in the way the classroom was teaching. And all the rewards in special education, time in special education that repeats what the classroom was doing will not meet the child’s learning needs and therefore will not provide evidence of what is effective for her.

I want to talk about one more concept with regards to “evidence” of efficacy…data vs research. Data is not research; it is data. It is collecting numbers of responses to stimuli. So, often these data are statistically analyzed to be able to determine how well a child is doing compared to other children. But children like the second grader are not included in that data and big picture purpose of the data is not always “evident.” You see, the research methods which use experimental group data assume that all learners are groupable and so the data is skewed before we ever start the statistical analysis because many children are omitted. Furthermore, the group data looks for “group evidence” of success. For example, 45% of the second graders are at grade level on a reading test, administered by the teacher. The school proudly announces that their new programs for reading are working, “evidence-based research” because the school has improved from 40% to 45%, a significant change in the data. But, what about the other 55% who did not pass at grade level? Expecting an average for a grade level to occur between 40-60% means we also expect 40-60% to fail. If you are the parent of one of the children who does not meet the average expectation, then what “evidence” do you have that education works for your child? Furthermore, what “evidence” do you have that any of these children will read for the purpose of learning, enjoyment, jobs, acquiring information, etc.?

Finally, I would be remiss if I did not also say that in all of my writings, I refer to specific cases. To me, this is “evidence” that learning is individually based, not a group average. And the research is based in epistemology of what I know about language. Is my work refereed? Absolutely. To publish with outside publishers, my work is reviewed by professionals, not just edited. So those books and all of the articles are refereed. “Evidence-based” research to me is grounded in the quality of questions asked not in the numbers.

Let’s talk about why professionals don’t want to use our clinical types of data in their decisions about approaches to education in the next blog…