Osborne_LTEC6516_Wk10_ReflectionBlog

This semester has been quite interesting, yet there is still more to learn about Computer-Mediated Discourse Analysis. Since this is a continuation of the LTEC 6512 qualitative research course, I can see why it does take more effort and time to conduct this type of research.  This is the second course that I’ve taken related to qualitative analysis and there are several take-aways and observations.

  • I can see where conducting qualitative research is beneficial when the study dictates going below the surface of an event or process. Qualitative research analysis is quite different than quantitative analysis.
  • Qualitative analysis affords a researcher the tools to consider information that goes beyond the measurement analysis processes of quantitative analysis.
  • I deal with quantitative analysis process every day at work. I mine different data sources to identify recurring defects in structural and electronic components and systems. The quantitative analysis provides the pointers to the error source. I can see where qualitative data analysis will be helpful in digging deeper into the root cause of these defects.
  • Understanding and using the correct instructional design framework is an important consideration when developing course content.
  • I thought it was quite helpful to see and understand the differences between Positivist and Non-Positivist research methods. Until it was developed in class I didn’t see the difference clearly in my readings.
  • The class content did touch on the various theoretical framework concepts. I think that it would be a great help for the students in Learning Technologies to have a survey course where all of these concepts are presented to provide an in-depth comparative analysis. This would be a great foundational skill development for every student.
  • My concern with both the LTEC 6512 and LTEC 6516 courses is that we didn’t get the hands-on experience that I hoped to develop. Developing and understanding the foundational theory and concepts of qualitative analysis is very important, but a practitioner must have and develop the coding and analysis capabilities.  I feel that I did get a lot of the front-end information, but I severely lacking in the coding and analysis skills.
  • A helpful part of this semesters studies is the concept of continually evaluating what and how I ask interview questions so as to minimize any biases that may directly impact or influence the outcomes of a study.
  • Since I work in a production environment, my focus is to analyze various processes and figure out how to identify and improve them. I have been looking at qualitative analysis processes and I’m trying to figure out how to create a standardized process formula or template that will reduce the amount of pushback and fear surrounding using qualitative analysis tools. After taking these two courses, I still see that there is a huge commitment associated with using this tool in research.
  • As I continue may Ph.D. studies I hope to develop better and consistent coding and analysis skills. I can see where the instructional designs where I work can benefit from this skill set. In a way I see this as learning an new language.
Advertisements

Osborne_LTEC6516_Wk6_Blog

Assignment:

Write a blog reflecting on what you read and what you saw in the video related to coding. How does the coding process help identify and vet new knowledge construction from qualitative research processes? How might technology distort “knowledge” communicated using social media, forums, synchronous tools, and others? What considerations should we make when using technology-mediated communication?

 

Response:

It was insightful to read through Aaron Cooley’s article entitled “Qualitative Research in Education: The Origins, Debates, and Politics of Creating Knowledge” (Cooley, 2013). Though I detect a hint of politically biased suppositions, ultimately Cooley identifies a key concern throughout the American education system, where “conversations as well as discourse at academic conferences often come to question the entire value of the education system” (Cooley, 2013, p. 248). There is much discussion and dissatisfaction concerning the overall effectiveness of the public-school systems and as a parent, I was dissatisfied with the outcomes of the school districts where I lived. What most parents have complained about but didn’t necessarily have a full conceptual grasp of is the “notion of the hidden curriculum” (Cooley, 2013, p. 251). As a result, many parents are choosing to put their children through homeschool and private school where they have more control of the education outcomes.

Cooley further discusses, the lack of acceptance of qualitative research within political and business entities. So, they view educational research and the associated qualitative research processes with skepticism. Hence the opinion of these groups holds that “qualitative work as noted by some critics allege, come from liberal and ideological college faculties and may be viewed as more suspect” (Cooley, 2013, p. 254). Cooley further notes that “proponents of qualitative methods should try to better educate critics in these policy circles to both the value and rigor of qualitative work” (Cooley, 2013, p. 254). In order to overcome these roadblocks, the challenge to the proponents of qualitative methods is two-fold:

  1. continue to develop qualitative studies that consistently provide substitutive results and findings
  2. develop processes that support analysis capabilities similar to those of quantitative method processes times

The old adage is ‘necessity is the mother of invention’. Current technologies have limitations to meet the demands of qualitative methods and supportive of coding analysis. With new technologies continuing to evolve, Artificial Intelligence has the potential to address these issues. Instead of making complaints about the circumstances, researchers need to develop their way through the issues.

Regarding the video discussion about coding, as a novice at this practice, it will take me an amount of time and practice to get the hang of the process. By performing multiple coding analysis activities, I will gain a better understanding of the process. Currently I’m working through a qualitative analysis project from the Spring semester. I am actively coding six interviews where I documented (field notes) the responses from six participants. The participants are active instructional designers that are designing and updating many training modules related to an aerospace production environment. The basis of the research project is to interview the participants to understand how through their instructional design activities they can know the effectiveness of the course outcomes. I hope to identify both positive outcomes and areas where they can improve their course designs. Even though this is a small scale qualitative analysis project, I hope to learn some of the essential skills in conducting non-positivist qualitative studies. This will help to build on the current body of knowledge and skill capabilities which will in turn help build trust and acceptance of the application of qualitative research in academia and business.

 

Reference

Cooley, A., (2013). Qualitative research in education: The origins, debates, and politics of creating knowledge. Education Studies, 49, 247-262.

Osborne_LTEC6516_Wk5_Blog

Assignment:

Take some time to REALLY think about what you believe you know about qualitative research after last week’s discussion and the video tonight. What were your misconceptions? What are challenges that come out of your past knowledge vs. the new? How will you take this forward into your research work?

 

Response:

After reviewing the qualitative research methods over the past couple weeks, I still have a lot to learn about all of the intricacies of this process. I had my first introduction to qualitative analysis this past semester. I have many years of experience with quantitative analysis in an industrial environment, where I collected process data, analyzed it, then created charts that depicted process defects and corrective action changes. I refer to this process as measurement and analysis (rack-and-stack). This type of processes are seen throughout many businesses and industry.

I reviewed the presentation posted for this week. I learned a lot from its content. There are some areas that I still need to gain further understanding. One of the key benefits is that it provides a roadmap into the qualitative analysis process. I now understand the difference between the Positivist and Non-Positivist qualitative process approaches. The qualitative research project that I am currently involved with will focus on the Non-Positivist process as I am focused on the subjective nature of how instructional designers approach and validate their course designs.

Since I have worked at the company for many years, I do have some preconceived opinions concerning the various course designs. Based on my understanding, I align with the ‘Reality’ sections Option A, where there is an objective reality, but I don’t see where anyone can gain a complete understanding of all that is involved in the process (ontological). However, each of the interviewees have indicated an agreement that there are gaps in the design, development, delivery and effectiveness of their courses (epistemology). Once I complete the coding along with two other professors, I’m anxious to see how my preconceived biases align with the results of the qualitative analysis.

I have developed a list of six interview questions that were given to six instructional designer participants. There were several additional questions related to their professional experience. I am currently in the process of coding each of their responses. I was not allowed to make audio and video recordings of the interview process, which placed limitations on the research study. I think that I captured much of the perceptions, feelings, opinions, and thought of the participants.

The design of the six interview questions was based on a course evaluation set of questions that are given to students at the conclusion of each training class. The course evaluation consisted of 23 Likert Scale type questions, in eight categories. The six questions that I created were a condensed version of those 23 evaluation questions. These six questions provide an overview synopsis of a course evaluation. The approach taken was what I refer to as a reverse engineering of the end of the course looking back to the design of the course. Overall, these six questions are a representation of the instructional designer’s efforts to design and develop each of their courses.

The topic of inquiry focuses on the approach that the instructional designer takes when designing the course. The six questions helped to answer the what, how, where, and who questions that are part of the topic of inquiry process. Being the ‘researcher as instrument’, I have considered two of the questions noted: what do I think I already know about the topic, and what is my subjective bias about the topic?  Since I have taken many courses at work for professional development and for required compliance training, I have formed some subjective biases. I don’t feel that these biases have influenced the interview questions created for this research study. The reason for my assessment is that the interview questions were developed directly from the evaluation instrument discussed earlier.

In order to establish the credibility of my research I will be incorporation three of the triangulation processes noted. These are as follows:

  • Member checks – once I completed the initial interview process, I transcribed my hand-written field notes into a Word document. I conducted a follow up meeting with each of the participants to review the transcribed document. I made updates and revisions based on their follow up responses.
  • Multiple coders – I am in process of coding each of the participants responses to the interview questions. I will obtain additional inputs from 1-2 other coders in order to confirm, expand, or eliminate my coding entries.
  • Field notes – I have transcribed all of my hand-written field notes into Word documents. In order to assist with the next phases of the qualitative analysis process, I will transcribe all of the Word document content into an Excel spreadsheet. This will enable better collaboration between coders and the final analysis.

My understanding of the coding analysis will be the use of a ‘constant-comparative analysis’ process, which will help to identify knowledge, skills, concerns, and opinions of the instructional designers.

After completing the qualitative analysis, I will need to write an article about the findings. In addition, I will need to create a presentation, which will include and executive summary of my findings, to be delivered to my professors as well as the instructional design participants and their managers. I hope the final qualitative analysis report will provide concrete suggestions for instructional design process improvements. The most important outcome is that I hope to better understand how to conduct effective qualitative research studies for my current Ph.D. and post-academic research studies.

One of the key benefits that is afforded to each Ph.D. student is the availability of tools that provide guidance, checks and balance in research endeavors. The concept where each new researcher builds on the shoulders of giants, is accomplished when the right tools are available that enhance the capabilities of the researcher. Otherwise the researcher is left to spend an inordinate amount of time developing the tools instead of focusing on the research.

With each Ph.D. course that I take, I’m looking for the tools that enable me to be an effective researcher. In both of the qualitative research courses that I have taken, I want to be sure I understand and have performed all of the steps necessary to gain the right experience to conduct qualitative research.  If possible, I hope that the content of these courses can serve as a foundation for future offerings of these courses. There needs to be a balance between the understanding of the foundational history and theory, but the hands-on experience is what’s needed for an effective outcome.

Osborne_LTEC6516_Wk4_Blog

Assignment:

How did your view of qualitative research change or not based on discussion tonight? What about reducing bias in research?

 

Response:

I still do have some apprehensions concerning engaging in qualitative research analysis. My concern centers around the correct method(s) in coding the responses to the surveys and questions of the individuals involved in the research study. I will be getting some experience as I will be using the research study that I started during the previous semester. I have been reading the articles assigned for week four and I have been able to relate to some of the challenges and tasks that I will need to be aware of in the coming days. I think the challenges related to my study are related to the make-up of the 6 individuals that I interviewed for the qualitative study. There are a number of things that I will need to recall as it relates to how each of the individuals answered the survey questions based on their individual experiences in conduction instructional design tasks.

I see that I have a lot to consider when I am conducting a qualitative research study. I will need to consider many variables. In the article entitled “Critical CinéEthnographic Methods: A new dimension for capturing the experience of learning” (Gratch & Warren), the discussion focuses on the use of audio and video to capture the interview content. One of the key benefits of the audio/video capture is that the analysis process will not be inhibited since the content can be reviewed over and over again. With the help of multiple evaluators, the qualitative analysis can become much deeper and richer as each of the evaluators employ each of their expertise in the analysis process. When each of the evaluators work together as a team to share their results and perspectives there will be expanded insights into the overall evaluation results.

The concept of minimizing bias in qualitative research is one that I see as taking time and experience. The process of minimizing personal bias in a qualitative research study appears to be a difficult task. If just one person is conducting the study, I can see where personal bias will be difficult to contain. However, when there are multiple evaluators these biases may be easily detected, unless everyone of the evaluators are equally biased.  I guess that the ultimate goal is to eliminate all personal bias but minimizing bias to the fullest extent possible may be the next best outcome. Ultimately with what bias exists, this must have minimal impact on the declared findings and results. If there are professional and experienced evaluators, they should be able and capable of identifying biased results.

 

Reference

Gratch, J., & Warren, S., (No Date). Critical CinéEthnographic methods: a new dimension for capturing the experience of learning. 1-26.

Osborne_LTEC6516_Wk3_Blog

Assignment:

Think about how you may apply qualitative data and CMDA in particular as a component of either your dissertation or another study and be prepared to discuss next week.

 

Response:

I have received a taste of applying qualitative data analysis in my previous course work. I constructed a set of questions based on the evaluations, given to learners, at the end of each of their courses. This approach is more or less a reverse engineering of those questions given to the learners. I want to understand how an instructional designer would develop their course(s) based on those questions given to the learners at the conclusion of the instruction delivery. I wasn’t sure that each of the instructional designers understood if they considered how their course content would impact each learner.

In light of the use of CMDA to analyze each of the instructional designer’s approach to course designs, I would like to understand if they took into consideration the connections between the course content and outcomes. This early in this course I have an inkling into the CMDA process. My understanding is that CMDA will analyze the collected qualitative data based on the verbal use of language and communication when responding to the survey questions. My first impression is that this will be a difficult task as I think that I’ll need a check sheet guide in order to grasp the concepts. If the process is very complex I hope that there are enough examples and samples of other qualitative analysis so that I have a guide to draw from which will improve my chances for a successful analysis. My concern is that unlike the straight forward approaches using quantitative analysis – the rack and stack methods, reading into the responses of the survey questions appears to be a daunting task. However, I’m eager to make a go of this process.

Osborne_LTEC6516_Wk2_Blog

Assignment:

Write a reflection on the synchronous class discussion regarding the readings from this week.

 

Response:

This weeks synchronous meeting was somewhat confusing to me in that I’m unsure of myself to be able to effectively identify the different elements of CMDA analysis. I’m not sure if this process is a global approach or a focused application. In some of the articles that I read the focus was on linguistics and the use of specific words. I think that I would become more comfortable after I have conducted several CMDA analysis assignments. I hope that this process is not require a unique set of skills based on one or more complex process methods.

In the qualitative process, that I am pulling forward from the Spring 2018 semester, I am conducting the coding of the responses of those individuals who participated in my study. My initial effort in the coding process did not go well in that I was repeating some of the comments made by the individual. I hope I’m not making this process more complicated than it needs to be. If the CMDA analysis is broad-based I hope I can understand some of the approaches so that I can at lease be able to use these methods going forward in my subsequent studies. The take-away from the online meeting is that there are many approaches to take when analyzing qualitative data. I hope we can establish and practice some of these methods so we can become proficient in these processes.

LTEC 6516_Osborne_Wk1_Blog

  • What is my general worldview in terms of what I think can be known, why that is the case, and how we can best understand the world?

Response:

This is a question that many of us dread as in a job interview when the individual is asked to answer the statemen “Tell me about yourself” – I digress. The question focuses on ‘what can be known’. In relation to qualitative research, this has been the focus when applied to “ontological and epistemological character of subjects and subjectivities” (Doucet & Nauthner, 2008, p. 399). In the context of qualitative research, the focus of the researcher is how they can fully understand and know what the individual(s) are communicating in their responses to the subject of inquiry. I think that the researcher must become knowledgeable about methods of creating the right set of questions to ask and the proper methods of coding the resultant responses. Based on previous courses, I can see where it will take a lot of hands-on practice as the textbooks can only provide guidance. At this early stage in conducting qualitative research, I find that it takes some effort to develop new ways of reviewing the responses of the participants to understand some of the deeper contexts and meanings of their responses.

 

  • What is research to me? What is its purpose? Do I prefer numbers or narratives or both?

Response:

Research is the process of finding and studying the peer reviewed research journals and studies of other scholars that focus on the critical and specific aspects of a particular subject or issue. These research resources provide a focused approach into the specifics of the issue at hand. The peer reviewed research provides a basis for my approach in conducting my own studies. As I contemplate the focus of my studies, I find that the previous research studies provide a guide and understanding of the approaches I can and should take in my research endeavors. In my job my main focus is on quantitative analysis, which I refer to as rack-and-stack of the measurement data (numbers). I have done this for years, in my role as a Quality Engineer. Those who are my work associates, we speak this same language daily. This switch in direction into qualitative research is like learning a new language. It requires a different approach to data collection and analysis. I hope to overcome the challenge to be able to properly and fluently code the qualitative research results so as to provide definitive and accurate results based on my research studies.

 

  • What is your main focus in terms of what you are planning for future research? Are you higher ed, K-12, corporate, or other? Are you looking to switch your focus to a new setting?

Response:

The focus of my research now and into the future is on the use of technologies in both instruction and on the job in industry. The broad category is corporate, but the focus is on industry and production environments. In my recent work experiences, over the past several decades, I have worked with individuals of mixed generations and mixed skill levels. In addition to these issue, the nature of the work being performed has become much more technical. My interest is to find ways to use technology to improve work productivity and product quality without increasing the cognitive load on the individual. Digital resources when properly designed can be a great resource to workers that will enable them to complete their tasks with out overburdening them with highly technical demands. Using computers to handle the technical details and providing the worker with the proper guidance will significantly decrease the cognitive load resulting in consistent quality of work outcomes.

 

Reference

Doucet, A., & Mauthner, N., (2008). What can be known and how? Narrated subjects and the Listening Guide. Qualitative Research (8)3, 399-409.