Discernment, or "It's a lot about Mom."

Articulate a research question

The first step was to articulate some sort of question. We'll call that step done.

Build the corpus

The next step was acquire the student reflections. This resulted in a set of mostly Microsoft Word documents. We converted them to plain text, and we anonymized them. Our analysis is not possible sans plain text, and anonymizing them (removing the students' identities) was an IRB (Institutional Review Board) requirement. The result can be viewed as a set of individual reflections or as a whole.

Read

From here the student, researcher, or scholar can begin to analyze (read) the materials in the traditional manner.

Count & tabulate features

At the same time, one can use text mining, natural language processing, and data science computing methods to count & tabulate features of the corpus (./adr, ./bib, ./ent, ./pos, ./urls, ./wrd). Once that is done, one can peruse the descriptive statistics generated from the results. This whole process alludes to the breadth and depth of the writings.

In this case, the corpus contains 1,400 files for a total of 2 million words. (The Bible is about .8 million words long.) At first glance, the corpus does not allude to the concept of discernment. On the other hand, notice the degree pronouns like "I", and "we" appear. Students may be writing about themselves.

unigrams

bigrams

pronouns

keywords

Perform reality check

At this point it is be a good idea to make sure we are not barking up the wrong tree, meaning, "What is this corpus about, and to what degree might we be able to address the research question?" Topic modeling (an unsupervised machine learning process used to enumerate latent themes) is a useful tecyhnique to use here. If the corups is modeled using eight topics, then we might say the corpus is about the following things and in the illustrated proportions:

    labels   weights                                           features
    people   0.86622  people life think things good know something t...
      life   0.71300  life others people community important however...
    school   0.46745   school college friends new time first high i’ve 
    career   0.18927  life career well-lived always lived time lives...
   believe   0.11733  believe faith vulnerability relationships stor...
   mission   0.10640  mission life statement career others well-live...
       god   0.10000  god leadership leader world true response refl...
 community   0.08545  community expectations hope encountered syndro...

Search

Since we are interested in the use of a specific word -- discernement, we can search the corpus for the word, and read items from the results in the traditional manner. Such a search returns only 345 reflections; the word discernment occurs in less than 25% of the entire corpus:

1. plain text results - readable by just about any software
2. HTML results - the most useful
3. comma-separated values (CSV) file - great for filtering and sorting

Articulate lexicon

After close reading of the materials, we can elaborate on our idea of discernment and thus articulate a lexicon -- a list of desireable words -- used to connote our idea of interest. In our case, the list includes: actions, calling, career, discernment, major, meaning, purpose, and values. Similarly, we want to enumerate words of little importance to our investigation, and those words are called stopwords. All of this is important because words are merely proxies for ideas. The lexicon and stopwords enable us to broaden as well as narrow the scope of discernment.

Read, again

We can refine our close reading to include only sentences of interest, thus, parse the corpus into sentences and filter the results by the stems of lexicon words. Each sentence in the result ought to include at least one of our lexicon words. Unfortunately, the result does not seem as meaningful as the set of sentences that only the stem of discernment. Some examples include:

• We must be patient with ourselves as we discern who we are, what we want from life, and what we are meant to do.
• I also think that my curiosity and my pursuance of truth through this curiosity will be important these next coming years as i attempt to discern what i think the truth about the world, people, and god, as well as my major and future job.
• I seek wisdom to discern my future.
• Although at times difficult to come to terms with, college is an experience with unimaginable changes both good and bad, which i am positive will shape me into the person i hope to become and give me the gift of discerning the things of most importance.
• I plan to use this idea of uniting with others as a compass throughout my life and especially make it a center in discerning my career path in a way that will embrace every human being rather than just certain individuals.

Concordance

Since words shall be known by the company they keep, concordancing for our lexicon words will result in a display of the lexicon words as well as the words immediately before and after the lexicon word. The result is not a set of sentences, but it does address the question, "With what other words does the word "discernment" (and its similar words) keep?" Some examples include:

		we discussed key questions that help to discern whether we are headed in the right
		day’s world, it is hard to find time to discern your path and find your joys in li
		ences and, as such, it is my mission to discern between the many forces in life an
		ny area is simply not enough – to truly discern what career path you will find you
		f this in mind come the time she was to discern what it was she wanted to do. she 
		e genuinely of interest to me, and then discern my major and career path based off

Count & tabulate bigrams

Bigrams are two-word phrases, and we might ask ourselves, "What two-word phrases exist where one of the words is a form of discernment (ie. "discern", "discerns", "discernment", etc.)?" The anwer can be garnered from a frequency table and visualized in a couple of ways:

word cloud

network

Count & tabulate collocations

Collocations are akin to combining concordance output and the concept of bigrams. Given a word, as well as a number of words on either side of the given word, what bigrams are statistically interesting. "Which combinations have high likelihood scores?" The resulting table can be visulized as network diagram:

collocations

List semantic distances

Another way to measure the frequency and relationships of words is to employ a technique called "word embedding", and it is akin to plotting words on the surface of a sphere. Then, given a word one can determine what words are nearby as well as the compute the distance between two words. Using our lexicon as input, we can create a table of semantic distances and plot the result. Thus, we can see that the concept of discernment was closely associated with the word "carrer" and "major":

semantic distances