We are dealing with the same data set as last session; as a reminder, the data are in _input/genitives.csv and you can find information about the variables/columns in _input/genitives.r. However, this session, we will deal with this in a multifactorial way. Specifically, we are asking, does the choice of a genitive construction (of vs. s) vary as a function of
a length-based short-before-long effect, but, this time, if we hypothesize a short-before-long effect, maybe we should not just be looking at the length of the possessor (POR_LENGTH), but how the length of the possessor compares to the length of the possessum (PUM_LENGTH); since such a comparison variable doesn’t exist yet in our data set, we need to create it first;
the degree/kind of animacy of the possessor (POR_ANIMACY: animate vs. collective vs. inanimate vs. locative vs. temporal);
whether the speakers are non-native speakers or native speakers of English (SPEAKER: nns vs. ns);
any pairwise interaction of these predictors;
the three-way interaction of these predictors?
rm(list=ls(all.names=TRUE))library(car); library(effects); library(magrittr); library(multcomp)source("_helpers/R2s.r"); source("_helpers/C.score.r"); source("_helpers/cohens.kappa.r")summary(d <-read.delim( # summarize d, the result of loadingfile="_input/genitives.csv", # this filestringsAsFactors=TRUE)) # change categorical variables into factors
CASE GENITIVE SPEAKER MODALITY POR_LENGTH
Min. : 2 of:2720 nns:2666 spoken :1685 Min. : 1.00
1st Qu.:1006 s : 880 ns : 934 written:1915 1st Qu.: 8.00
Median :2018 Median : 11.00
Mean :2012 Mean : 14.58
3rd Qu.:3017 3rd Qu.: 17.00
Max. :4040 Max. :204.00
PUM_LENGTH POR_ANIMACY POR_FINAL_SIB POR_DEF
Min. : 2.00 animate : 920 absent :2721 definite :2349
1st Qu.: 6.00 collective: 607 present: 879 indefinite:1251
Median : 9.00 inanimate :1671
Mean : 10.35 locative : 243
3rd Qu.: 13.00 temporal : 159
Max. :109.00
2 Deviance & baseline(s)
Let’s already compute the baselines for what will be the response variable, GENITIVE:
(baselines <-c("baseline 1"=max( # make baselines[1] the highestprop.table( # proportion in thetable(d$GENITIVE))), # frequency table of the response"baseline 2"=sum( # make baselines[2] the sum of theprop.table( # proportions in thetable(d$GENITIVE))^2))) # frequency table of the response squared
summary(m_01 <-glm( # make/summarize the gen. linear model m_01: GENITIVE ~1+# GENITIVE ~ an overall intercept (1) LEN_PORmPUM_LOG*POR_ANIMACY*SPEAKER, # & these predictors & their interactionfamily=binomial, # resp = binarydata=d, # those vars are in dna.action=na.exclude)) # skip cases with NA/missing data
pchisq( # compute the area under the chi-squared curveq =m_01$null.deviance-m_01$deviance, # for this chi-squared value: null - res. dev.df=m_01$df.null-m_01$df.residual, # at this df: null - residual dflower.tail=FALSE) # only using the right/upper tail/side
[1] 0
drop1(m_01, # drop each droppable predictor at a time from m_01 &test="Chisq") # test its significance w/ a LRT
Let’s fit the 2nd model with that 3-way interaction LEN_PORmPUM_LOG:POR_ANIMACY:SPEAKER deleted:
summary(m_02 <-update( # make m_02 an update of m_01, .~. # m_01, namely all of it (.~.), but then- LEN_PORmPUM_LOG:POR_ANIMACY:SPEAKER)) # remove this interaction
Let’s fit the 3rd model with that interaction LEN_PORmPUM_LOG:SPEAKER deleted:
summary(m_03 <-update( # make m_03 an update of m_02, .~. # m_02, namely all of it (.~.), but then- LEN_PORmPUM_LOG:SPEAKER)) # remove this interaction
pchisq( # compute the area under the chi-squared curveq=1596.2, # for this chi-squared valuedf=14, # at 14 dflower.tail=FALSE) # only using the right/upper tail/side
[1] 0
Let’s make ‘predictions’ and evaluate them:
d$PRED_PP_S <-predict( # make d$PRED_PP_S the result of predicting m_final, # from m_finaltype="response") # predicted probabilities of "s"# if you don't say type="response", predict gives you log oddsd$PRED_CAT <-factor( # make d$PRED_CAT the result of checkingifelse(d$PRED_PP_S>=0.5, # if the predicted prob. of "s" is >=0.5levels(d$GENITIVE)[2], # if yes, predict "s"levels(d$GENITIVE)[1])) # otherwise, predict "of"(c_m <-table( # confusion matrix: cross-tabulate"OBS"=d$GENITIVE, # observed genitives in the rows"PREDS"=d$PRED_CAT)) # predicted orders in the columns
PREDS
OBS of s
of 2454 266
s 310 570
c( # evaluate the confusion matrix"Prec. for s"=c_m[ "s", "s"] /sum(c_m[ , "s"]),"Acc./rec. for s"=c_m[ "s", "s"] /sum(c_m[ "s", ]),"Prec. for of"=c_m["of","of"] /sum(c_m[ ,"of"]),"Acc./rec. for of"=c_m["of","of"] /sum(c_m["of", ]),"Acc. (overall)"=mean(d$GENITIVE==d$PRED_CAT))
Prec. for s Acc./rec. for s Prec. for of Acc./rec. for of
0.6818182 0.6477273 0.8878437 0.9022059
Acc. (overall)
0.8400000
This looks like a pretty decent model, not bad at all (esp. Cohen’s kappa and the C-score).
5 Visual interpretation
5.1 The interaction LEN_PORmPUM_LOG:POR_ANIMACY
Let’s visualize the nature of the effect of LEN_PORmPUM_LOG:POR_ANIMACY based on the predictions:
(lean_d <-data.frame( # make lean_d a data frame of lean <-effect( # the effect"LEN_PORmPUM_LOG:POR_ANIMACY", # of LEN_PORmPUM_LOG:POR_ANIMACYxlevels=list(LEN_PORmPUM_LOG=-4:4), # w/ these values for LEN_PORmPUM_LOG m_final))) # in m_final
plot(lean, # plot the effect of LEN_PORmPUM_LOG:POR_ANIMACYtype="response", # but plot predicted probabilitiesylim=c(0, 1), # ylim: the whole range of possible probabilitiesgrid=TRUE) # & w/ a grid
This might be better with a one-panel kind of plot:
plot(lean, # plot the effect of LEN_PORmPUM_LOG:POR_ANIMACYtype="response", # but plot predicted probsylim=c(0, 1), grid=TRUE, # w/ these y-axis limits & w/ a gridmultiline=TRUE, # put all lines in one panelconfint=list(style="auto"), # keep confidence intervalslattice=list(key.args=list( # make thecolumns=5, cex.title=0))) # legend have 5 columns & no title
BTW: Do you recognize a couple of (related) general conceptual problems here?
5.2 Excursus on predictions and slopes
To better understand the results, we return to my very pedestrian approach from earlier: I generate a data frame nd of all predictions that help me understand the coefficients table in the summary output of the model. That means,
on the predictor side of the final model, nd will contain
each categorical predictor with all its levels;
each numeric predictor with values across its whole range while making sure those include 0 and 1;
on the response side of the final model, nd will contain
predictions on the probability scale (because those are the ones I usually (!) prefer to visualize);
predictions on the log odds scale (because those are the ones the coefficients table returns and because the slopes of curved lines can be difficult to compare).
Here, this would mean we’d do this (I am skipping the computation of the confidence intervals of the predicted probabilities):
nd <-expand.grid( # create a data frame nd that crosses# these values of each of these predictorsLEN_PORmPUM_LOG=-5:5,POR_ANIMACY =levels(d$POR_ANIMACY),SPEAKER =levels(d$SPEAKER) )[,3:1] # change the order of columns to something nicer# add predictions of the second level of the response: log odds % pred. probs.:nd$PREDS_LO_S <-predict(m_final, newdata=nd)nd$PRED_PP_S <-predict(m_final, newdata=nd, type="response")
For a model like ours, there are two main ways to look at this that are useful: (i) we would just look at the data frame as is but focus on the two numeric values of LEN_PORmPUM_LOG that best illustrate the slope …
…, or (ii) we group the data a bit differently, but again in a way that makes assessing the slopes of LEN_PORmPUM_LOG as obvious as possible:
nd.grouped <-with( # make nd.grouped the result of# taking only the 0:1 length difference subset nd[nd$LEN_PORmPUM_LOG %in%0:1,], tapply( # and apply PREDS_LO_S, # a grouping of the log-odds predictionslist(POR_ANIMACY, LEN_PORmPUM_LOG, SPEAKER), # by all predictors c)) # just listing the values (there's just 1 per combo)nd.grouped # what this looks like
Why do we get the same result twice? Because apply does this to both the native and the non-native speaker data, but since LEN_PORmPUM_LOG only interacts with POR_ANIMACY and that interaction does not interact with not also with SPEAKER, the slopes are the same for both levels of SPEAKER.
(There is a simpler way to do this – simpler in terms of coding, that is – but as usual I am trying to show you a way that makes the logic of the approach explicit.)
This plot now shows the (straight-line) log odds slopes in a more comparable way:
plot(lean, # plot the effect of LEN_PORmPUM_LOG:POR_ANIMACY# note: no type="response"ylim=c(-5, 5), grid=TRUE, # w/ these y-axis limits & w/ a gridmultiline=TRUE, # put all lines in one panelconfint=list(style="auto"), # keep confidence intervalslattice=list(key.args=list( # make thecolumns=5, cex.title=0))) # legend have 5 columns & no title
5.3 The interaction POR_ANIMACY:SPEAKER
Let’s visualize the nature of the effect of POR_ANIMACY:SPEAKER based on the predictions, as always, we do both perspectives:
(ansp_d <-data.frame( # make ansp_d a data frame of ansp <-effect( # the effect"POR_ANIMACY:SPEAKER", # of POR_ANIMACY:SPEAKER m_final))) # in m_final
plot(ansp, # plot the effect of POR_ANIMACY:SPEAKERtype="response", # but plot predicted probabilitiesylim=c(0, 1), # ylim: the whole range of possible probabilitiesgrid=TRUE) # & w/ a grid
plot(ansp, # plot the effect of POR_ANIMACY:SPEAKERtype="response", # but plot predicted probabilitiesylim=c(0, 1), # ylim: the whole range of possible probabilitiesx.var="SPEAKER", # w/ this predictor on the x-axisgrid=TRUE) # & w/ a grid
A multi-line plot might be better:
plot(ansp, # plot the effect of POR_ANIMACY:SPEAKERtype="response", # but plot predicted probabilitiesylim=c(0, 1), # ylim: the whole range of possible probabilitiesmultiline=TRUE, # all the lines in one panel, butconfint=list(style="auto"), # w/ confidence intervalsgrid=TRUE) # & w/ a grid
6 Excursus: predictions that are most wrong
To explore which predictions are the worst – wrong predictions made most confidently – let’s also compute the contributions to logloss; I’m doing this here in the very explicit two-step procedure just for didactic purposes:
d$PRED_PP_OBS <-ifelse( # d$PRED_PP_OBS is determined by ifelse d$GENITIVE=="s", # if the obs genitive is the 2nd level of the response d$PRED_PP_S, # take its predicted probability1-d$PRED_PP_S) # otherwise take 1 minus its predicted probabilityd$CONTRIBS2LL <--log(d$PRED_PP_OBS)head(d)
CASE GENITIVE SPEAKER MODALITY POR_LENGTH PUM_LENGTH POR_ANIMACY
1 2 of nns spoken 13 7 collective
2 3 of nns spoken 22 7 animate
3 4 of nns spoken 11 8 animate
4 5 of nns spoken 26 4 collective
5 6 s nns spoken 8 4 animate
6 7 s nns spoken 7 3 animate
POR_FINAL_SIB POR_DEF LEN_PORmPUM_LOG PRED_PP_S PRED_CAT PRED_PP_OBS
1 absent definite 0.8930848 0.10246571 of 0.8975343
2 absent definite 1.6520767 0.38528810 of 0.6147119
3 present definite 0.4594316 0.58241501 s 0.4175850
4 absent definite 2.7004397 0.01351966 of 0.9864803
5 absent definite 1.0000000 0.49254016 of 0.4925402
6 absent definite 1.2223924 0.45537135 of 0.4553713
CONTRIBS2LL
1 0.10810395
2 0.48660157
3 0.87326719
4 0.01361188
5 0.70817928
6 0.78664204
There are many ways one could look at this, the following are just two suggestions. First, we could look at the general distribution of the contributions to logloss to see what it looks like and what values might be considered ‘extreme’; an ecdf plot could be good for that:
plot(ecdf(d$CONTRIBS2LL)); grid()abline(h=0.9)
This shows a very extreme distribution: about 90% of the logloss values are ≤1 (but then our model was quite good).
Second, we could do a very simple monofactorial exploration of each predictor against logloss; one time, we do so for all logloss values (top row), the other we zoom into the ‘extreme’ logloss values:
It seems as if contributions to logloss values are frequently high for
length difference values close to 0, but also positive ones;
inanimate possessors;
to maybe a lesser extent for non-native speakers.
7 Excursus: prototypes
Something else that may well be worth your time involves taking a quick look at which combinations of predictor values (numeric as well as levels) yields the highest predicted probabilities for each level of the response variable. Why might that be interesting? Because of the connection it makes to prototypes. The prototype of a category can be defined “as an abstract entity – i.e., an abstract notion that may not actually exist in the real world –- that exhibits the features (or properties) f1-n that have the highest cue validity for a category, where cue validity, […], can in turn be defined as follows: A feature/property (e.g., ‘having a beak’) has a high cue validity for a category (e.g., ‘birds’) if
many members of the category have the feature (and yes, all birds have beaks);
most non-members of the category do not have the feature (most non-birds have no beaks – animals that are not birds but have beaks make up a small set of some turtles, octopuses, platypuses/echidnas, and pufferfish).” (Gries, Slocum, & Tobia 2024)
Armed with this definition, we might follow Gries (2003) and stipulate for our data here that
(observed or hypothetical) instances with the highest predicted probabilities of s-genitives should constitute prototypical s-genitives and
(observed or hypothetical) instances with the lowest predicted probabilities of s-genitives should constitute prototypical of-genitives
(according to our model at least). We can try and identify such prototypes in three places:
the model summary (i.e., the coefficients): m_final$coef;
the predictions we generated for new/hypothetical data in nd;
the predictions for the actual data (d$PRED_PP_S).
The inferences from these three places will converge but, because of the presence of interactions, identifying the highest predicted probabilities from the model’s coefficients can be tricky even in a relatively small final model like here (because of how you would need to add and subtract values). We will therefore immediately go to nd instead
The most obvious results from looking at all theoretically possible combinations as generated in nd are that
very low predicted probabilities, i.e. prototypical of-genitives, are strongly associated with possessors that are much longer than their possessums;
very high predicted probabilities, i.e. prototypical s-genitives, are strongly associated with possessors that are much shorter than their possessums.
We can see this clearly, for instance, if we sort nd by the predicted probabilities and then plot the predicted probabilities on the x-axis and whether the length difference is the positive extreme (5, red) or the negative extreme (-5, blue) on the y-axis: extremely high predicted probabilities of s-genitives on the right only arise with blue lines, i.e. when the length difference is smallest, and extremely high predicted probabilities of of-genitives on the left only arise with red lines, i.e. when the length difference is largest:
plot(type="h", col="#FF000040", axes=FALSE,xlab="Predicted probability of an s-genitive" , qwe$PRED_PP_S,ylab="Length difference is 5 (red) or -5 (blue)?", qwe$LEN_PORmPUM_LOG==5)lines(qwe$PRED_PP_S, qwe$LEN_PORmPUM_LOG==-5, type="h", col="#0000FF40")axis(1); axis(2, at=0:1, labels=c("FALSE", "TRUE"))
In addition, we can see that
very low predicted probabilities, i.e. prototypical of-genitives, are strongly associated with inanimate and locative possessors;
very high predicted probabilities, i.e. prototypical s-genitives, are strongly associated with animate (and maybe collective) possessors.
plot(type="h", col="#FF000040", axes=FALSE,xlab="Predicted probability of an s-genitive", qwe$PRED_PP_S,ylab="Animate (red) or inanimate (blue)?" , qwe$POR_ANIMACY=="animate")lines(qwe$PRED_PP_S, qwe$POR_ANIMACY=="inanimate", type="h", col="#0000FF40")axis(1); axis(2, at=0:1, labels=c("FALSE", "TRUE"))
And we know from POR_ANIMACY:SPEAKER above that animate, inanimate, and locative possessors don’t differ much in their predictions between the two levels of SPEAKER so we can leave it at that. (In fact, we will later see that SPEAKER only makes a significant difference for collectives.)
The above already made a lot of sense, but it was based on nd, i.e. all theoretically possible configurations of values. To be safe, I recommend you also look at d or, more specifically, d$PRED_PP_S because it is possible that many of the theoretically possible configurations in nd are actually never attested. That would be fine because we said above a prototype is “an abstract notion that may not actually exist in the real world” – we’re just looking at the combination(s) of features – but wouldn’t we want to know whether the abstract prototype we infer from our model actually exists? (Hint: the answer is “yes”.) So let’s check out d:
qwe <- d[order(d$PRED_PP_S),]qwe %>%head(15)
CASE GENITIVE SPEAKER MODALITY POR_LENGTH PUM_LENGTH POR_ANIMACY
661 741 of nns spoken 79 4 collective
3442 3853 of ns written 90 4 inanimate
3263 3646 of ns written 88 5 inanimate
2904 3243 of ns spoken 99 6 inanimate
1779 1993 of nns written 94 4 inanimate
1074 1192 of nns spoken 116 5 inanimate
804 900 of nns spoken 88 4 inanimate
3149 3524 of ns written 40 3 inanimate
2366 2633 of nns written 62 3 inanimate
3218 3597 of ns written 89 4 collective
245 278 of nns spoken 58 4 collective
261 296 of nns spoken 58 4 collective
654 734 of nns spoken 76 4 inanimate
239 271 of nns spoken 52 3 inanimate
1449 1615 of nns written 68 4 inanimate
POR_FINAL_SIB POR_DEF LEN_PORmPUM_LOG PRED_PP_S PRED_CAT PRED_PP_OBS
661 absent definite 4.303781 0.002085659 of 0.9979143
3442 absent indefinite 4.491853 0.002310730 of 0.9976893
3263 absent indefinite 4.137504 0.002722820 of 0.9972772
2904 absent definite 4.044394 0.002842763 of 0.9971572
1779 present indefinite 4.554589 0.003088579 of 0.9969114
1074 absent indefinite 4.536053 0.003115190 of 0.9968848
804 absent definite 4.459432 0.003227640 of 0.9967724
3149 absent definite 3.736966 0.003277476 of 0.9967225
2366 absent definite 4.369234 0.003365210 of 0.9966348
3218 absent definite 4.475733 0.003470205 of 0.9965298
245 absent definite 3.857981 0.003513241 of 0.9964868
261 absent definite 3.857981 0.003513241 of 0.9964868
654 absent indefinite 4.247928 0.003559484 of 0.9964405
239 absent indefinite 4.115477 0.003784387 of 0.9962156
1449 absent indefinite 4.087463 0.003833740 of 0.9961663
CONTRIBS2LL
661 0.002087837
3442 0.002313404
3263 0.002726533
2904 0.002846812
1779 0.003093358
1074 0.003120053
804 0.003232860
3149 0.003282859
2366 0.003370885
3218 0.003476241
245 0.003519426
261 0.003519426
654 0.003565834
239 0.003791566
1449 0.003841108
qwe %>%tail(15)
CASE GENITIVE SPEAKER MODALITY POR_LENGTH PUM_LENGTH POR_ANIMACY
24 27 s nns spoken 4 26 animate
872 972 s nns spoken 3 25 collective
898 1003 s nns spoken 3 25 collective
671 753 s nns spoken 4 34 collective
1386 1545 s nns written 1 7 animate
1406 1565 s nns written 1 7 animate
887 990 s nns spoken 3 26 collective
2968 3317 s ns spoken 4 25 collective
3238 3619 s ns written 4 27 collective
1147 1273 s nns spoken 6 59 animate
2466 2756 s nns written 11 109 animate
2981 3330 s ns spoken 4 28 collective
2979 3328 s ns spoken 4 31 collective
529 595 s nns spoken 3 36 collective
689 771 s nns spoken 4 64 collective
POR_FINAL_SIB POR_DEF LEN_PORmPUM_LOG PRED_PP_S PRED_CAT PRED_PP_OBS
24 absent definite -2.700440 0.9206995 s 0.9206995
872 absent definite -3.058894 0.9216670 s 0.9216670
898 absent definite -3.058894 0.9216670 s 0.9216670
671 absent definite -3.087463 0.9240524 s 0.9240524
1386 absent definite -2.807355 0.9257793 s 0.9257793
1406 present definite -2.807355 0.9257793 s 0.9257793
887 absent definite -3.115477 0.9263265 s 0.9263265
2968 absent definite -2.643856 0.9364706 s 0.9364706
3238 absent definite -2.754888 0.9437919 s 0.9437919
1147 absent definite -3.297681 0.9454444 s 0.9454444
2466 absent definite -3.308753 0.9458261 s 0.9458261
2981 absent definite -2.807355 0.9469688 s 0.9469688
2979 absent definite -2.954196 0.9549815 s 0.9549815
529 absent definite -3.584963 0.9561546 s 0.9561546
689 absent definite -4.000000 0.9725900 s 0.9725900
CONTRIBS2LL
24 0.08262154
872 0.08157127
898 0.08157127
671 0.07898655
1386 0.07711937
1406 0.07711937
887 0.07652856
2968 0.06563719
3238 0.05784957
1147 0.05610022
2466 0.05569653
2981 0.05448917
2979 0.04606335
529 0.04483566
689 0.02779269
It seems that our prototypes as inferred from nd are also attested:
the most strongly predicted of-genitives have long possessors, short possessums and are mostly inanimate (but also some collectives). Also, while we didn’t include the predictor POR_DEF in the model, it is striking to see that there is about a 50-50 balance of definite and indefinite for these of-genitives;
the most strongly predicted s-genitives have short possessors, long possessums and are mostly animate and collectives. Also, while we didn’t include the predictor POR_DEF in the model, it is striking to see that there is not a single case of indefinite for these s-genitives.
8 Write-up
To determine whether the choice of a genitive construction (of vsn s) varies as a function of
the length difference between logged possessor length and logged possessum length;
the degree/kind of animacy of the possessor (POR_ANIMACY: animate vs. collective vs. inanimate vs. locative vs. temporal);
whether the speakers are non-native speakers or native speakers of English (SPEAKER: nns vs. ns);
any pairwise interaction of these predictors;
the three-way interaction of these predictors,
A generalized linear model selection process was undertaken (using backwards stepwise model selection of all predictors and controls based on significance testing). The final model resulting from the elimination of ns predictors contained the effects of LEN_PORmPUM_LOG:POR_ANIMACY and SPEAKER:POR_ANIMACY and was highly significant (LR=1596.2, df=14, p<0.0001) with a good amount of explanatory/predictive power (McFadden’s R2=0.397, Nagelkerke’s R2=0.534, C=0.899). The summary table of the model is shown here.
the interaction LEN_PORmPUM_LOG:POR_ANIMACY has the effect that
the length difference effect is strongest for collective possessors, followed by locatives, then animates, then inanimates, then temporals, but all of them are significantly different from 0;
the only differences between slopes for different animacy levels that are significant (acc. to post-hoc tests adjusted for multiple testing with Holm’s procedure) are the following:
collective vs. animate;
collective vs. inanimate;
collective vs. temporal;
the interaction SPEAKER:POR_ANIMACY only has the effect that native speakers are significantly more likely to use s genitives with collectives than learners are – all other speaker differences within each animacy level are ns. [Show plots]
If we operationalize prototypes with extreme values of predictions, prototypival of-genitives have long possessors, short possessums and are mostly inanimate (with some collectives) whereas prototypical s-genitives have short possessors, long possessums and are mostly animate and collectives; also, while we didn’t include the predictor POR_DEF in the model, prototypical s-genitives seem to ‘dislike’ indefinite possessors.
Finally, the model did particularly badly with length difference values close to 0 or positive and/or inanimate possessors, but also, to a lesser extent, with non-native speakers.
---title: "Ling 105: session 05: bin. logistic regr. 2 (key)"author: - name: "[Stefan Th. Gries](https://www.stgries.info)" affiliation: - UC Santa Barbara - JLU Giessen orcid: 0000-0002-6497-3958date: "2024-04-29 12:34:56"date-format: "DD MMM YYYY HH-mm-ss"editor: sourceformat: html: page-layout: full code-fold: false code-link: true code-copy: true code-tools: true code-line-numbers: true code-overflow: scroll number-sections: true smooth-scroll: true toc: true toc-depth: 4 number-depth: 4 toc-location: left monofont: lucida console tbl-cap-location: top fig-cap-location: bottom fig-width: 5 fig-height: 5 fig-format: png fig-dpi: 300 fig-align: center embed-resources: trueexecute: cache: false echo: true eval: true warning: false---# IntroductionWe are dealing with the same data set as last session; as a reminder, the data are in [_input/genitives.csv](_input/genitives.csv) and you can find information about the variables/columns in [_input/genitives.r](_input/genitives.r). However, this session, we will deal with this in a multifactorial way. Specifically, we are asking, does the choice of a genitive construction (*of* vs. *s*) vary as a function of* a length-based short-before-long effect, but, this time, if we hypothesize a short-before-long effect, maybe we should not just be looking at the length of the possessor (`POR_LENGTH`), but how the length of the possessor compares to the length of the possessum (`PUM_LENGTH`); since such a comparison variable doesn't exist yet in our data set, we need to create it first;* the degree/kind of animacy of the possessor (`POR_ANIMACY`: *animate* vs. *collective* vs. *inanimate* vs. *locative* vs. *temporal*);* whether the speakers are non-native speakers or native speakers of English (`SPEAKER`: *nns* vs. *ns*);* any pairwise interaction of these predictors;* the three-way interaction of these predictors?```{r prepworkspace}rm(list=ls(all.names=TRUE))library(car); library(effects); library(magrittr); library(multcomp)source("_helpers/R2s.r"); source("_helpers/C.score.r"); source("_helpers/cohens.kappa.r")summary(d <-read.delim( # summarize d, the result of loadingfile="_input/genitives.csv", # this filestringsAsFactors=TRUE)) # change categorical variables into factors```# Deviance & baseline(s)Let's already compute the baselines for what will be the response variable, `GENITIVE`:```{r}(baselines <-c("baseline 1"=max( # make baselines[1] the highestprop.table( # proportion in thetable(d$GENITIVE))), # frequency table of the response"baseline 2"=sum( # make baselines[2] the sum of theprop.table( # proportions in thetable(d$GENITIVE))^2))) # frequency table of the response squared```Let's compute the deviance of the null model:```{r}m_00 <-glm(GENITIVE ~1, family=binomial, data=d, na.action=na.exclude)deviance(m_00)sum(residuals(m_00)^2)```# Exploration & preparationSome exploration of the relevant variables:```{r}# the predictor(s)/response on its/their ownhist(d$LEN_PORmPUM_LOG <-log2(d$POR_LENGTH)-log2(d$PUM_LENGTH), main="")table(d$POR_ANIMACY)table(d$SPEAKER)# the predictor(s) w/ the responsespineplot(d$GENITIVE ~ d$LEN_PORmPUM_LOG)table(d$POR_ANIMACY, d$GENITIVE)table(d$SPEAKER, d$GENITIVE)ftable(d$POR_ANIMACY, d$SPEAKER, d$GENITIVE)```# Modeling & numerical interpretationLet's fit our initial regression model:```{r}summary(m_01 <-glm( # make/summarize the gen. linear model m_01: GENITIVE ~1+# GENITIVE ~ an overall intercept (1) LEN_PORmPUM_LOG*POR_ANIMACY*SPEAKER, # & these predictors & their interactionfamily=binomial, # resp = binarydata=d, # those vars are in dna.action=na.exclude)) # skip cases with NA/missing datapchisq( # compute the area under the chi-squared curveq =m_01$null.deviance-m_01$deviance, # for this chi-squared value: null - res. dev.df=m_01$df.null-m_01$df.residual, # at this df: null - residual dflower.tail=FALSE) # only using the right/upper tail/sidedrop1(m_01, # drop each droppable predictor at a time from m_01 &test="Chisq") # test its significance w/ a LRT```Let's fit the 2nd model with that 3-way interaction `LEN_PORmPUM_LOG:POR_ANIMACY:SPEAKER` deleted:```{r}summary(m_02 <-update( # make m_02 an update of m_01, .~. # m_01, namely all of it (.~.), but then- LEN_PORmPUM_LOG:POR_ANIMACY:SPEAKER)) # remove this interactionanova(m_01, m_02, # compare m_01 to m_02test="Chisq") # using a LRTdrop1(m_02, # drop each droppable predictor at a time from m_02 &test="Chisq") # test its significance w/ a LRT```Let's fit the 3rd model with that interaction `LEN_PORmPUM_LOG:SPEAKER` deleted:```{r}summary(m_03 <-update( # make m_03 an update of m_02, .~. # m_02, namely all of it (.~.), but then- LEN_PORmPUM_LOG:SPEAKER)) # remove this interactionanova(m_02, m_03, # compare m_02 to m_03test="Chisq") # using a LRTdrop1(m_03, # drop each droppable predictor at a time from m_03 &test="Chisq") # test its significance w/ a LRTConfint(m_final <- m_03); rm(m_03); invisible(gc())```Let's compute the overall significance test for this model:```{r}anova(m_00, m_final, # compare m_00 to m_finaltest="Chisq") # using a LRTpchisq( # compute the area under the chi-squared curveq=1596.2, # for this chi-squared valuedf=14, # at 14 dflower.tail=FALSE) # only using the right/upper tail/side```Let's make 'predictions' and evaluate them:```{r}d$PRED_PP_S <-predict( # make d$PRED_PP_S the result of predicting m_final, # from m_finaltype="response") # predicted probabilities of "s"# if you don't say type="response", predict gives you log oddsd$PRED_CAT <-factor( # make d$PRED_CAT the result of checkingifelse(d$PRED_PP_S>=0.5, # if the predicted prob. of "s" is >=0.5levels(d$GENITIVE)[2], # if yes, predict "s"levels(d$GENITIVE)[1])) # otherwise, predict "of"(c_m <-table( # confusion matrix: cross-tabulate"OBS"=d$GENITIVE, # observed genitives in the rows"PREDS"=d$PRED_CAT)) # predicted orders in the columnsc( # evaluate the confusion matrix"Prec. for s"=c_m[ "s", "s"] /sum(c_m[ , "s"]),"Acc./rec. for s"=c_m[ "s", "s"] /sum(c_m[ "s", ]),"Prec. for of"=c_m["of","of"] /sum(c_m[ ,"of"]),"Acc./rec. for of"=c_m["of","of"] /sum(c_m["of", ]),"Acc. (overall)"=mean(d$GENITIVE==d$PRED_CAT))c(R2s(m_01),"Cohen's kappa"=cohens.kappa(c_m)[[1]],"C"=C.score(d$GENITIVE, d$PRED_PP_S))```This looks like a pretty decent model, not bad at all (esp. Cohen's *kappa* and the *C*-score).# Visual interpretation## The interaction `LEN_PORmPUM_LOG:POR_ANIMACY`Let's visualize the nature of the effect of `LEN_PORmPUM_LOG:POR_ANIMACY` based on the predictions:```{r}(lean_d <-data.frame( # make lean_d a data frame of lean <-effect( # the effect"LEN_PORmPUM_LOG:POR_ANIMACY", # of LEN_PORmPUM_LOG:POR_ANIMACYxlevels=list(LEN_PORmPUM_LOG=-4:4), # w/ these values for LEN_PORmPUM_LOG m_final))) # in m_finalplot(lean, # plot the effect of LEN_PORmPUM_LOG:POR_ANIMACYtype="response", # but plot predicted probabilitiesylim=c(0, 1), # ylim: the whole range of possible probabilitiesgrid=TRUE) # & w/ a grid```This might be better with a one-panel kind of plot:```{r}plot(lean, # plot the effect of LEN_PORmPUM_LOG:POR_ANIMACYtype="response", # but plot predicted probsylim=c(0, 1), grid=TRUE, # w/ these y-axis limits & w/ a gridmultiline=TRUE, # put all lines in one panelconfint=list(style="auto"), # keep confidence intervalslattice=list(key.args=list( # make thecolumns=5, cex.title=0))) # legend have 5 columns & no title```BTW: Do you recognize a couple of (related) general conceptual problems here?## Excursus on predictions and slopes```{r}#| echo: false# 1. we cannot see whether lines/slopes are significantly different from each other because the lines are curved: there's no easy way to compare the confidence intervals;# 2. it's therefore also not really straightforward to, for instance, rank-order the slopes from the plot;# 3. it's therefore also not obvious how we see whether the slopes for, say, inanimates and temporals are significantly different from 0.```To better understand the results, we return to my very pedestrian approach from earlier: I generate a data frame `nd` of all predictions that help me understand the coefficients table in the summary output of the model. That means,* on the predictor side of the final model, `nd` will contain + each categorical predictor with all its levels; + each numeric predictor with values across its whole range while making sure those include 0 and 1;* on the response side of the final model, `nd` will contain + predictions on the probability scale (because those are the ones I usually (!) prefer to visualize); + predictions on the log odds scale (because those are the ones the coefficients table returns and because the slopes of curved lines can be difficult to compare).Here, this would mean we'd do this (I am skipping the computation of the confidence intervals of the predicted probabilities):```{r nd}nd <-expand.grid( # create a data frame nd that crosses# these values of each of these predictorsLEN_PORmPUM_LOG=-5:5,POR_ANIMACY =levels(d$POR_ANIMACY),SPEAKER =levels(d$SPEAKER) )[,3:1] # change the order of columns to something nicer# add predictions of the second level of the response: log odds % pred. probs.:nd$PREDS_LO_S <-predict(m_final, newdata=nd)nd$PRED_PP_S <-predict(m_final, newdata=nd, type="response")```For a model like ours, there are two main ways to look at this that are useful: (i) we would just look at the data frame as is but focus on the two numeric values of `LEN_PORmPUM_LOG` that best illustrate the slope ...```{r}nd[nd$LEN_PORmPUM_LOG %in%0:1,]```..., or (ii) we group the data a bit differently, but again in a way that makes assessing the slopes of `LEN_PORmPUM_LOG` as obvious as possible:```{r}nd.grouped <-with( # make nd.grouped the result of# taking only the 0:1 length difference subset nd[nd$LEN_PORmPUM_LOG %in%0:1,], tapply( # and apply PREDS_LO_S, # a grouping of the log-odds predictionslist(POR_ANIMACY, LEN_PORmPUM_LOG, SPEAKER), # by all predictors c)) # just listing the values (there's just 1 per combo)nd.grouped # what this looks like```This way, we can use `apply` to immediately see what the slopes of `LEN_PORmPUM_LOG` are for each level of `POR_ANIMACY`:```{r}apply( # apply to nd.grouped, 1, # nd.grouped row-by-row diff) # the function diff```Why do we get the same result twice? Because `apply` does this to both the native and the non-native speaker data, but since `LEN_PORmPUM_LOG` only interacts with `POR_ANIMACY` and that interaction does not interact with not also with `SPEAKER`, the slopes are the same for both levels of `SPEAKER`.(There is a simpler way to do this -- simpler in terms of coding, that is -- but as usual I am trying to show you a way that makes the logic of the approach explicit.)```{r}#| echo: false#| eval: falseemmeans::emtrends(m_final, ~ POR_ANIMACY, var="LEN_PORmPUM_LOG") # also discussed at end```This plot now shows the (straight-line) log odds slopes in a more comparable way:```{r}plot(lean, # plot the effect of LEN_PORmPUM_LOG:POR_ANIMACY# note: no type="response"ylim=c(-5, 5), grid=TRUE, # w/ these y-axis limits & w/ a gridmultiline=TRUE, # put all lines in one panelconfint=list(style="auto"), # keep confidence intervalslattice=list(key.args=list( # make thecolumns=5, cex.title=0))) # legend have 5 columns & no title```## The interaction `POR_ANIMACY:SPEAKER`Let's visualize the nature of the effect of `POR_ANIMACY:SPEAKER` based on the predictions, as always, we do both perspectives:```{r}(ansp_d <-data.frame( # make ansp_d a data frame of ansp <-effect( # the effect"POR_ANIMACY:SPEAKER", # of POR_ANIMACY:SPEAKER m_final))) # in m_finalplot(ansp, # plot the effect of POR_ANIMACY:SPEAKERtype="response", # but plot predicted probabilitiesylim=c(0, 1), # ylim: the whole range of possible probabilitiesgrid=TRUE) # & w/ a gridplot(ansp, # plot the effect of POR_ANIMACY:SPEAKERtype="response", # but plot predicted probabilitiesylim=c(0, 1), # ylim: the whole range of possible probabilitiesx.var="SPEAKER", # w/ this predictor on the x-axisgrid=TRUE) # & w/ a grid```A multi-line plot might be better:```{r}plot(ansp, # plot the effect of POR_ANIMACY:SPEAKERtype="response", # but plot predicted probabilitiesylim=c(0, 1), # ylim: the whole range of possible probabilitiesmultiline=TRUE, # all the lines in one panel, butconfint=list(style="auto"), # w/ confidence intervalsgrid=TRUE) # & w/ a grid```# Excursus: predictions that are most wrongTo explore which predictions are the worst -- wrong predictions made most confidently -- let's also compute the contributions to logloss; I'm doing this here in the very explicit two-step procedure just for didactic purposes:```{r}d$PRED_PP_OBS <-ifelse( # d$PRED_PP_OBS is determined by ifelse d$GENITIVE=="s", # if the obs genitive is the 2nd level of the response d$PRED_PP_S, # take its predicted probability1-d$PRED_PP_S) # otherwise take 1 minus its predicted probabilityd$CONTRIBS2LL <--log(d$PRED_PP_OBS)head(d)```There are many ways one could look at this, the following are just two suggestions. First, we could look at the general distribution of the contributions to logloss to see what it looks like and what values might be considered 'extreme'; an ecdf plot could be good for that:```{r}plot(ecdf(d$CONTRIBS2LL)); grid()abline(h=0.9)```This shows a very extreme distribution: about 90% of the logloss values are ≤1 (but then our model was quite good).Second, we could do a very simple monofactorial exploration of each predictor against logloss; one time, we do so for all logloss values (top row), the other we zoom into the 'extreme' logloss values:```{r}#| fig-width: 10par(mfrow=c(2,3))with(d, {plot(CONTRIBS2LL ~ LEN_PORmPUM_LOG, xlim=c(-5,5))plot(CONTRIBS2LL ~ POR_ANIMACY)plot(CONTRIBS2LL ~ SPEAKER)})with(d[d$CONTRIBS2LL>1,], {plot(CONTRIBS2LL ~ LEN_PORmPUM_LOG, xlim=c(-5,5))plot(CONTRIBS2LL ~ POR_ANIMACY)plot(CONTRIBS2LL ~ SPEAKER)})par(mfrow=c(1,1))```It seems as if contributions to logloss values are frequently high for* length difference values close to 0, but also positive ones;* inanimate possessors;* to maybe a lesser extent for non-native speakers.# Excursus: prototypesSomething else that may well be worth your time involves taking a quick look at which combinations of predictor values (numeric as well as levels) yields the highest predicted probabilities for each level of the response variable. Why might that be interesting? Because of the connection it makes to prototypes. The prototype of a category can be defined "as an abstract entity -- i.e., an abstract notion that may not actually exist in the real world –- that exhibits the features (or properties) *f*~1-*n*~ that have the highest cue validity for a category, where cue validity, [...], can in turn be defined as follows: A feature/property (e.g., 'having a beak') has a high cue validity for a category (e.g., 'birds') if* many members of the category have the feature (and yes, all birds have beaks);* most non-members of the category do not have the feature (most non-birds have no beaks -- animals that are not birds but have beaks make up a small set of some turtles, octopuses, platypuses/echidnas, and pufferfish)." ([Gries, Slocum, & Tobia 2024](https://www.sciencedirect.com/science/article/pii/S2666799123000394?via%3Dihub))Armed with this definition, we might follow [Gries (2003)](https://www.stgries.info/research/2003_STG_PrototypCxs_ARCL.pdf) and stipulate for our data here that* (observed or hypothetical) instances with the highest predicted probabilities of *s*-genitives should constitute prototypical *s*-genitives and* (observed or hypothetical) instances with the lowest predicted probabilities of *s*-genitives should constitute prototypical *of*-genitives(according to our model at least). We can try and identify such prototypes in three places:* the model summary (i.e., the coefficients): `m_final$coef`;* the predictions we generated for new/hypothetical data in `nd`;* the predictions for the actual data (`d$PRED_PP_S`).The inferences from these three places will converge but, because of the presence of interactions, identifying the highest predicted probabilities from the model's coefficients can be tricky even in a relatively small final model like here (because of how you would need to add and subtract values). We will therefore immediately go to `nd` instead```{r}qwe <- nd[order(nd$PRED_PP_S),]qwe %>%head(15)qwe %>%tail(15)```The most obvious results from looking at all theoretically possible combinations as generated in `nd` are that* very low predicted probabilities, i.e. prototypical *of*-genitives, are strongly associated with possessors that are much longer than their possessums;* very high predicted probabilities, i.e. prototypical *s*-genitives, are strongly associated with possessors that are much shorter than their possessums.We can see this clearly, for instance, if we sort `nd` by the predicted probabilities and then plot the predicted probabilities on the *x*-axis and whether the length difference is the positive extreme (5, red) or the negative extreme (-5, blue) on the *y*-axis: extremely high predicted probabilities of *s*-genitives on the right only arise with blue lines, i.e. when the length difference is smallest, and extremely high predicted probabilities of *of*-genitives on the left only arise with red lines, i.e. when the length difference is largest:```{r}plot(type="h", col="#FF000040", axes=FALSE,xlab="Predicted probability of an s-genitive" , qwe$PRED_PP_S,ylab="Length difference is 5 (red) or -5 (blue)?", qwe$LEN_PORmPUM_LOG==5)lines(qwe$PRED_PP_S, qwe$LEN_PORmPUM_LOG==-5, type="h", col="#0000FF40")axis(1); axis(2, at=0:1, labels=c("FALSE", "TRUE"))```In addition, we can see that* very low predicted probabilities, i.e. prototypical *of*-genitives, are strongly associated with inanimate and locative possessors;* very high predicted probabilities, i.e. prototypical *s*-genitives, are strongly associated with animate (and maybe collective) possessors.```{r}plot(type="h", col="#FF000040", axes=FALSE,xlab="Predicted probability of an s-genitive", qwe$PRED_PP_S,ylab="Animate (red) or inanimate (blue)?" , qwe$POR_ANIMACY=="animate")lines(qwe$PRED_PP_S, qwe$POR_ANIMACY=="inanimate", type="h", col="#0000FF40")axis(1); axis(2, at=0:1, labels=c("FALSE", "TRUE"))```And we know from `POR_ANIMACY:SPEAKER` above that animate, inanimate, and locative possessors don't differ much in their predictions between the two levels of `SPEAKER` so we can leave it at that. (In fact, we will later see that `SPEAKER` only makes a significant difference for collectives.)The above already made a lot of sense, but it was based on `nd`, i.e. all theoretically possible configurations of values. To be safe, I recommend you also look at `d` or, more specifically, `d$PRED_PP_S` because it is possible that many of the theoretically possible configurations in `nd` are actually never attested. That would be fine because we said above a prototype is "an abstract notion that may not actually exist in the real world" -- we're just looking at the combination(s) of features -- but wouldn't we want to know whether the abstract prototype we infer from our model actually exists? (Hint: the answer is "yes".) So let's check out `d`:```{r}qwe <- d[order(d$PRED_PP_S),]qwe %>%head(15)qwe %>%tail(15)```It seems that our prototypes as inferred from `nd` are also attested:* the most strongly predicted *of*-genitives have long possessors, short possessums and are mostly inanimate (but also some collectives). Also, while we didn't include the predictor `POR_DEF` in the model, it is striking to see that there is about a 50-50 balance of *definite* and *indefinite* for these *of*-genitives;* the most strongly predicted *s*-genitives have short possessors, long possessums and are mostly animate and collectives. Also, while we didn't include the predictor `POR_DEF` in the model, it is striking to see that there is not a single case of *indefinite* for these *s*-genitives.# Write-upTo determine whether the choice of a genitive construction (*of* vsn *s*) varies as a function of* the length difference between logged possessor length and logged possessum length;* the degree/kind of animacy of the possessor (`POR_ANIMACY`: *animate* vs. *collective* vs. *inanimate* vs. *locative* vs. *temporal*);* whether the speakers are non-native speakers or native speakers of English (`SPEAKER`: *nns* vs. *ns*);* any pairwise interaction of these predictors;* the three-way interaction of these predictors,A generalized linear model selection process was undertaken (using backwards stepwise model selection of all predictors and controls based on significance testing). The final model resulting from the elimination of ns predictors contained the effects of `LEN_PORmPUM_LOG:POR_ANIMACY` and `SPEAKER:POR_ANIMACY` and was highly significant (*LR*=1596.2, *df*=14, *p*<0.0001) with a good amount of explanatory/predictive power (McFadden's *R*^2^=0.397, Nagelkerke's *R*^2^=0.534, *C*=0.899). The summary table of the model is shown here.| | Estimate | 95%-CI | *se* | *z* | *p*~2-tailed~ ||:---------------------------------------------------------------------|:---------|:---------------|:------|:-------|:--------------|| (Intercept (LEN_PORmPUM_LOG=0, POR_ANIMACY=*animate*, SPEAKER=*nns*) | 0.64 | [0.47, 0.82] | 0.09 | 7.21 | <0.001 || LEN_PORmPUM_LOG~0→1~ | -0.67 | [-0.81, -0.54] | 0.07 | -9.58 | <0.001 || POR_ANIMACY~*animate*→*collective*~ | -1.76 | [-2.08, -1.46] | 0.16 | -11.20 | <0.001 || POR_ANIMACY~*animate*→*inanimate*~ | -4.30 | [-4.74, -3.90] | 0.21 | -20.04 | <0.001 || POR_ANIMACY~*animate*→*locative*~ | -2.30 | [-2.84, -1.81] | 0.26 | -8.79 | <0.001 || POR_ANIMACY~*animate*→*temporal*~ | -1.32 | [-1.77, -0.88] | 0.23 | -5.82 | <0.001 || SPEAKER~*nns*→*ns*~ | -0.03 | [-0.34, 0.29] | 0.16 | -0.16 | 0.8708 || POR_ANIMACY~*animate*→*collective*~ + LEN_PORmPUM_LOG~0→1~ | -0.50 | [-0.76, -0.26] | 0.13 | -3.91 | 0.001 || POR_ANIMACY~*animate*→*inanimate*~ + LEN_PORmPUM_LOG~0→1~ | 0.21 | [-0.10, 0.52] | 0.16 | 1.30 | 0.1949 || POR_ANIMACY~*animate*→*locative*~ + LEN_PORmPUM_LOG~0→1~ | -0.34 | [-0.80, 0.08] | 0.22 | -1.53 | 0.1249 || POR_ANIMACY~*animate*→*temporal*~ + LEN_PORmPUM_LOG~0→1~ | 0.31 | [-0.01, 0.62] | 0.16 | 1.95 | 0.051 || POR_ANIMACY~*animate*→*collective*~:SPEAKER~*nns*→*ns*~ | 0.74 | [0.18, 1.30] | 0.28 | 2.59 | 0.0095 || POR_ANIMACY~*animate*→*inanimate*~:SPEAKER~*nns*→*ns*~ | -0.29 | [-1.33, 0.60] | 0.48 | -0.61 | 0.544 || POR_ANIMACY~*animate*→*locative*~:SPEAKER~*nns*→*ns*~ | -0.53 | [-1.43, 0.31] | 0.44 | -1.20 | 0.2283 || POR_ANIMACY~*animate*→*temporal*~:SPEAKER~*nns*→*ns*~ | 0.09 | [-0.71, 0.88] | 0.40 | 0.22 | 0.8238 |The results indicate that* the interaction `LEN_PORmPUM_LOG:POR_ANIMACY` has the effect that + the length difference effect is strongest for collective possessors, followed by locatives, then animates, then inanimates, then temporals, but all of them are significantly different from 0; + the only differences between slopes for different animacy levels that are significant (acc. to post-hoc tests adjusted for multiple testing with Holm's procedure) are the following: - *collective* vs. *animate*; - *collective* vs. *inanimate*; - *collective* vs. *temporal*;* the interaction `SPEAKER:POR_ANIMACY` only has the effect that native speakers are significantly more likely to use *s* genitives with collectives than learners are -- all other speaker differences within each animacy level are ns. [Show plots]If we operationalize prototypes with extreme values of predictions, prototypival *of*-genitives have long possessors, short possessums and are mostly inanimate (with some collectives) whereas prototypical *s*-genitives have short possessors, long possessums and are mostly animate and collectives; also, while we didn't include the predictor `POR_DEF` in the model, prototypical *s*-genitives seem to 'dislike' *indefinite* possessors.Finally, the model did particularly badly with length difference values close to 0 or positive and/or inanimate possessors, but also, to a lesser extent, with non-native speakers.```{r}#| echo: false#| eval: false# I wrote: "the length difference effect is strongest for collective possessors, followed by locatives, then animates, then inanimates, then temporals, but all of them are significantly different from 0" -- how do I know that?!``````{r}#| echo: false#| eval: false# the length difference effect is strongest for collectives, then locatives, then animates, then inanimates, then temporals,# but all of them are significantly different from 0;summary(multcomp::glht(m_final, matrix(c(0,1,0,0,0,0,0,0,0,0,0,0,0,0,0), nrow=1)))summary(multcomp::glht(m_final, matrix(c(0,1,0,0,0,0,0,1,0,0,0,0,0,0,0), nrow=1)))summary(multcomp::glht(m_final, matrix(c(0,1,0,0,0,0,0,0,1,0,0,0,0,0,0), nrow=1)))summary(multcomp::glht(m_final, matrix(c(0,1,0,0,0,0,0,0,0,1,0,0,0,0,0), nrow=1)))summary(multcomp::glht(m_final, matrix(c(0,1,0,0,0,0,0,0,0,0,1,0,0,0,0), nrow=1)))# the only differences between slopes for different animacy levels that are significant# (acc. to post-hoc tests adjusted for multiple testing with Holm's procedure) are the following:(slopes <- emmeans::emtrends( # compute the slopesobject=m_final, # in this model, namely~ POR_ANIMACY, # grouped by this categorical predictorvar="LEN_PORmPUM_LOG")) # of this numeric predictorpairs(slopes, # compute pairwise comparisons of slopesadjust="holm") # adjust for multiple comparisons``````{r}#| echo: false#| eval: false# the interaction `SPEAKER:POR_ANIMACY`: native speakers are significantly more likely to use *s* genitives with collectives than learners --# all other speaker differences within each animacy level are ns(means1 <- emmeans::emmeans( # compute the meansobject=m_final, # for the model m_final~ SPEAKER | POR_ANIMACY)) # SPEAKER grouped by POR_ANIMACYpairs(means1, # compute pairwise comparisons of meansadjust="holm") # adjust for multiple comparisons(means2 <- emmeans::emmeans( # compute the meansobject=m_final, # for the model m_final~ POR_ANIMACY | SPEAKER)) # POR_ANIMACY grouped by SPEAKERpairs(means2, # compute pairwise comparisons of meansadjust="holm") # adjust for multiple comparisons``````{r sessionInfo}sessionInfo()```
