daviddoji/ISLP_labs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v2.1 of Ch13

Browse Source
This commit is contained in:
Jonathan Taylor
2023-08-20 19:37:25 -07:00
parent fc0c9152cb
commit c82e9d5067
2 changed files with 232 additions and 242 deletions
Download Patch File Download Diff File Expand all files Collapse all files

82
Ch13-multiple-lab.Rmd
View File

@@ -1,22 +1,9 @@
---
jupyter:
jupytext:
cell_metadata_filter: -all
formats: ipynb,Rmd
main_language: python
text_representation:
extension: .Rmd
format_name: rmarkdown
format_version: '1.2'
jupytext_version: 1.14.7
---
# Chapter 13 # Chapter 13
# Lab: Multiple Testing # Lab: Multiple Testing
We include our usual imports seen in earlier labs. We include our usual imports seen in earlier labs.
@@ -28,7 +15,7 @@ import statsmodels.api as sm
from ISLP import load_data from ISLP import load_data
``` ```
We also collect the new imports We also collect the new imports
needed for this lab. needed for this lab.
@@ -60,7 +47,7 @@ true_mean = np.array([0.5]*50 + [0]*50)
X += true_mean[None,:] X += true_mean[None,:]
``` ```
To begin, we use `ttest_1samp()` from the To begin, we use `ttest_1samp()` from the
`scipy.stats` module to test $H_{0}: \mu_1=0$, the null `scipy.stats` module to test $H_{0}: \mu_1=0$, the null
hypothesis that the first variable has mean zero. hypothesis that the first variable has mean zero.
@@ -70,7 +57,7 @@ result = ttest_1samp(X[:,0], 0)
result.pvalue result.pvalue
``` ```
The $p$-value comes out to 0.931, which is not low enough to The $p$-value comes out to 0.931, which is not low enough to
reject the null hypothesis at level $\alpha=0.05$. In this case, reject the null hypothesis at level $\alpha=0.05$. In this case,
$\mu_1=0.5$, so the null hypothesis is false. Therefore, we have made $\mu_1=0.5$, so the null hypothesis is false. Therefore, we have made
@@ -121,7 +108,7 @@ amounts to quite a weak signal, and it resulted in a high number of
Type II errors. Let’s instead simulate data with a stronger signal, Type II errors. Let’s instead simulate data with a stronger signal,
so that the ratio of the mean to the standard deviation for the false so that the ratio of the mean to the standard deviation for the false
null hypotheses equals $1$. We make only 10 Type II errors. null hypotheses equals $1$. We make only 10 Type II errors.
```{python} ```{python}
true_mean = np.array([1]*50 + [0]*50) true_mean = np.array([1]*50 + [0]*50)
@@ -167,7 +154,7 @@ ax.legend()
ax.axhline(0.05, c='k', ls='--'); ax.axhline(0.05, c='k', ls='--');
``` ```
As discussed previously, even for moderate values of $m$ such as $50$, As discussed previously, even for moderate values of $m$ such as $50$,
the FWER exceeds $0.05$ unless $\alpha$ is set to a very low value, the FWER exceeds $0.05$ unless $\alpha$ is set to a very low value,
such as $0.001$. Of course, the problem with setting $\alpha$ to such such as $0.001$. Of course, the problem with setting $\alpha$ to such
@@ -189,7 +176,7 @@ for i in range(5):
fund_mini_pvals fund_mini_pvals
``` ```
The $p$-values are low for Managers One and Three, and high for the The $p$-values are low for Managers One and Three, and high for the
other three managers. However, we cannot simply reject $H_{0,1}$ and other three managers. However, we cannot simply reject $H_{0,1}$ and
$H_{0,3}$, since this would fail to account for the multiple testing $H_{0,3}$, since this would fail to account for the multiple testing
@@ -219,8 +206,8 @@ reject, bonf = mult_test(fund_mini_pvals, method = "bonferroni")[:2]
reject reject
``` ```
The $p$-values `bonf` are simply the `fund_mini_pvalues` multiplied by 5 and truncated to be less than The $p$-values `bonf` are simply the `fund_mini_pvalues` multiplied by 5 and truncated to be less than
or equal to 1. or equal to 1.
@@ -228,7 +215,7 @@ or equal to 1.
bonf, np.minimum(fund_mini_pvals * 5, 1) bonf, np.minimum(fund_mini_pvals * 5, 1)
``` ```
Therefore, using Bonferroni’s method, we are able to reject the null hypothesis only for Manager Therefore, using Bonferroni’s method, we are able to reject the null hypothesis only for Manager
One while controlling FWER at $0.05$. One while controlling FWER at $0.05$.
@@ -240,17 +227,18 @@ hypotheses for Managers One and Three at a FWER of $0.05$.
mult_test(fund_mini_pvals, method = "holm", alpha=0.05)[:2] mult_test(fund_mini_pvals, method = "holm", alpha=0.05)[:2]
``` ```
As discussed previously, Manager One seems to perform particularly As discussed previously, Manager One seems to perform particularly
well, whereas Manager Two has poor performance. well, whereas Manager Two has poor performance.
```{python} ```{python}
fund_mini.mean() fund_mini.mean()
``` ```
Is there evidence of a meaningful difference in performance between Is there evidence of a meaningful difference in performance between
these two managers? We can check this by performing a paired $t$-test using the `ttest_rel()` function these two managers? We can check this by performing a paired $t$-test using the `ttest_rel()` function
from `scipy.stats`: from `scipy.stats`:
@@ -260,7 +248,7 @@ ttest_rel(fund_mini['Manager1'],
fund_mini['Manager2']).pvalue fund_mini['Manager2']).pvalue
``` ```
The test results in a $p$-value of 0.038, The test results in a $p$-value of 0.038,
suggesting a statistically significant difference. suggesting a statistically significant difference.
@@ -285,8 +273,8 @@ tukey = pairwise_tukeyhsd(returns, managers)
print(tukey.summary()) print(tukey.summary())
``` ```
The `pairwise_tukeyhsd()` function provides confidence intervals The `pairwise_tukeyhsd()` function provides confidence intervals
for the difference between each pair of managers (`lower` and for the difference between each pair of managers (`lower` and
`upper`), as well as a $p$-value. All of these quantities have `upper`), as well as a $p$-value. All of these quantities have
@@ -316,7 +304,7 @@ for i, manager in enumerate(Fund.columns):
fund_pvalues[i] = ttest_1samp(Fund[manager], 0).pvalue fund_pvalues[i] = ttest_1samp(Fund[manager], 0).pvalue
``` ```
There are far too many managers to consider trying to control the FWER. There are far too many managers to consider trying to control the FWER.
Instead, we focus on controlling the FDR: that is, the expected fraction of rejected null hypotheses that are actually false positives. Instead, we focus on controlling the FDR: that is, the expected fraction of rejected null hypotheses that are actually false positives.
The `multipletests()` function (abbreviated `mult_test()`) can be used to carry out the Benjamini--Hochberg procedure. The `multipletests()` function (abbreviated `mult_test()`) can be used to carry out the Benjamini--Hochberg procedure.
@@ -326,7 +314,7 @@ fund_qvalues = mult_test(fund_pvalues, method = "fdr_bh")[1]
fund_qvalues[:10] fund_qvalues[:10]
``` ```
The *q-values* output by the The *q-values* output by the
Benjamini--Hochberg procedure can be interpreted as the smallest FDR Benjamini--Hochberg procedure can be interpreted as the smallest FDR
threshold at which we would reject a particular null hypothesis. For threshold at which we would reject a particular null hypothesis. For
@@ -353,8 +341,8 @@ null hypotheses!
(fund_pvalues <= 0.1 / 2000).sum() (fund_pvalues <= 0.1 / 2000).sum()
``` ```
Figure 13.6 displays the ordered Figure 13.6 displays the ordered
$p$-values, $p_{(1)} \leq p_{(2)} \leq \cdots \leq p_{(2000)}$, for $p$-values, $p_{(1)} \leq p_{(2)} \leq \cdots \leq p_{(2000)}$, for
the `Fund` dataset, as well as the threshold for rejection by the the `Fund` dataset, as well as the threshold for rejection by the
@@ -383,7 +371,7 @@ else:
sorted_set_ = [] sorted_set_ = []
``` ```
We now reproduce the middle panel of Figure 13.6. We now reproduce the middle panel of Figure 13.6.
```{python} ```{python}
@@ -398,7 +386,7 @@ ax.scatter(sorted_set_+1, sorted_[sorted_set_], c='r', s=20)
ax.axline((0, 0), (1,q/m), c='k', ls='--', linewidth=3); ax.axline((0, 0), (1,q/m), c='k', ls='--', linewidth=3);
``` ```
## A Re-Sampling Approach ## A Re-Sampling Approach
Here, we implement the re-sampling approach to hypothesis testing Here, we implement the re-sampling approach to hypothesis testing
@@ -414,8 +402,8 @@ D['Y'] = pd.concat([Khan['ytrain'], Khan['ytest']])
D['Y'].value_counts() D['Y'].value_counts()
``` ```
There are four classes of cancer. For each gene, we compare the mean There are four classes of cancer. For each gene, we compare the mean
expression in the second class (rhabdomyosarcoma) to the mean expression in the second class (rhabdomyosarcoma) to the mean
expression in the fourth class (Burkitt’s lymphoma). Performing a expression in the fourth class (Burkitt’s lymphoma). Performing a
@@ -435,8 +423,8 @@ observedT, pvalue = ttest_ind(D2[gene_11],
observedT, pvalue observedT, pvalue
``` ```
However, this $p$-value relies on the assumption that under the null However, this $p$-value relies on the assumption that under the null
hypothesis of no difference between the two groups, the test statistic hypothesis of no difference between the two groups, the test statistic
follows a $t$-distribution with $29+25-2=52$ degrees of freedom. follows a $t$-distribution with $29+25-2=52$ degrees of freedom.
@@ -464,8 +452,8 @@ for b in range(B):
(np.abs(Tnull) > np.abs(observedT)).mean() (np.abs(Tnull) > np.abs(observedT)).mean()
``` ```
This fraction, 0.0398, This fraction, 0.0398,
is our re-sampling-based $p$-value. is our re-sampling-based $p$-value.
It is almost identical to the $p$-value of 0.0412 obtained using the theoretical null distribution. It is almost identical to the $p$-value of 0.0412 obtained using the theoretical null distribution.
@@ -521,7 +509,7 @@ for j in range(m):
Tnull_vals[j,b] = ttest_.statistic Tnull_vals[j,b] = ttest_.statistic
``` ```
Next, we compute the number of rejected null hypotheses $R$, the Next, we compute the number of rejected null hypotheses $R$, the
estimated number of false positives $\widehat{V}$, and the estimated estimated number of false positives $\widehat{V}$, and the estimated
FDR, for a range of threshold values $c$ in FDR, for a range of threshold values $c$ in
@@ -539,7 +527,7 @@ for j in range(m):
FDRs[j] = V / R FDRs[j] = V / R
``` ```
Now, for any given FDR, we can find the genes that will be Now, for any given FDR, we can find the genes that will be
rejected. For example, with FDR controlled at 0.1, we reject 15 of the rejected. For example, with FDR controlled at 0.1, we reject 15 of the
100 null hypotheses. On average, we would expect about one or two of 100 null hypotheses. On average, we would expect about one or two of
@@ -555,7 +543,7 @@ the genes whose estimated FDR is less than 0.1.
sorted(idx[np.abs(T_vals) >= cutoffs[FDRs < 0.1].min()]) sorted(idx[np.abs(T_vals) >= cutoffs[FDRs < 0.1].min()])
``` ```
At an FDR threshold of 0.2, more genes are selected, at the cost of having a higher expected At an FDR threshold of 0.2, more genes are selected, at the cost of having a higher expected
proportion of false discoveries. proportion of false discoveries.
@@ -563,7 +551,7 @@ proportion of false discoveries.
sorted(idx[np.abs(T_vals) >= cutoffs[FDRs < 0.2].min()]) sorted(idx[np.abs(T_vals) >= cutoffs[FDRs < 0.2].min()])
``` ```
The next line generates Figure 13.11, which is similar The next line generates Figure 13.11, which is similar
to Figure 13.9, to Figure 13.9,
except that it is based on only a subset of the genes. except that it is based on only a subset of the genes.
@@ -575,5 +563,5 @@ ax.set_xlabel("Number of Rejections")
ax.set_ylabel("False Discovery Rate"); ax.set_ylabel("False Discovery Rate");
``` ```

392
Ch13-multiple-lab.ipynb
View File

@@ -2,7 +2,7 @@
"cells": [ "cells": [
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "75b2d75c", "id": "687e9b80",
"metadata": {}, "metadata": {},
"source": [ "source": [
"\n", "\n",
@@ -14,7 +14,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "34e410a6", "id": "9f594a41",
"metadata": {}, "metadata": {},
"source": [ "source": [
"We include our usual imports seen in earlier labs." "We include our usual imports seen in earlier labs."
@@ -23,13 +23,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 1, "execution_count": 1,
"id": "1f928b2d", "id": "7cc4fbeb",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:48.489971Z", "iopub.execute_input": "2023-08-21T02:29:16.417394Z",
"iopub.status.busy": "2023-08-07T00:19:48.489727Z", "iopub.status.busy": "2023-08-21T02:29:16.417287Z",
"iopub.status.idle": "2023-08-07T00:19:50.216508Z", "iopub.status.idle": "2023-08-21T02:29:17.613483Z",
"shell.execute_reply": "2023-08-07T00:19:50.215573Z" "shell.execute_reply": "2023-08-21T02:29:17.613156Z"
} }
}, },
"outputs": [], "outputs": [],
@@ -43,7 +43,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "12319e0a", "id": "08ba7bed",
"metadata": {}, "metadata": {},
"source": [ "source": [
"We also collect the new imports\n", "We also collect the new imports\n",
@@ -53,13 +53,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 2, "execution_count": 2,
"id": "eb4b32aa", "id": "595efc18",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.219989Z", "iopub.execute_input": "2023-08-21T02:29:17.615551Z",
"iopub.status.busy": "2023-08-07T00:19:50.219429Z", "iopub.status.busy": "2023-08-21T02:29:17.615375Z",
"iopub.status.idle": "2023-08-07T00:19:50.223239Z", "iopub.status.idle": "2023-08-21T02:29:17.617379Z",
"shell.execute_reply": "2023-08-07T00:19:50.222392Z" "shell.execute_reply": "2023-08-21T02:29:17.617087Z"
}, },
"lines_to_next_cell": 2 "lines_to_next_cell": 2
}, },
@@ -78,7 +78,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "a2747e58", "id": "69e5023e",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## Review of Hypothesis Tests\n", "## Review of Hypothesis Tests\n",
@@ -92,13 +92,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 3, "execution_count": 3,
"id": "e12ac0cd", "id": "985d1d6e",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.225454Z", "iopub.execute_input": "2023-08-21T02:29:17.618995Z",
"iopub.status.busy": "2023-08-07T00:19:50.225335Z", "iopub.status.busy": "2023-08-21T02:29:17.618887Z",
"iopub.status.idle": "2023-08-07T00:19:50.228651Z", "iopub.status.idle": "2023-08-21T02:29:17.620921Z",
"shell.execute_reply": "2023-08-07T00:19:50.228301Z" "shell.execute_reply": "2023-08-21T02:29:17.620629Z"
} }
}, },
"outputs": [], "outputs": [],
@@ -111,7 +111,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "70d37233", "id": "9ae561c4",
"metadata": {}, "metadata": {},
"source": [ "source": [
"To begin, we use `ttest_1samp()` from the\n", "To begin, we use `ttest_1samp()` from the\n",
@@ -122,13 +122,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 4,
"id": "04d0f49e", "id": "753d612a",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.230854Z", "iopub.execute_input": "2023-08-21T02:29:17.622537Z",
"iopub.status.busy": "2023-08-07T00:19:50.230727Z", "iopub.status.busy": "2023-08-21T02:29:17.622429Z",
"iopub.status.idle": "2023-08-07T00:19:50.236745Z", "iopub.status.idle": "2023-08-21T02:29:17.626063Z",
"shell.execute_reply": "2023-08-07T00:19:50.236388Z" "shell.execute_reply": "2023-08-21T02:29:17.625801Z"
} }
}, },
"outputs": [ "outputs": [
@@ -150,7 +150,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "cf83426f", "id": "5d9dc17f",
"metadata": {}, "metadata": {},
"source": [ "source": [
"The $p$-value comes out to 0.931, which is not low enough to\n", "The $p$-value comes out to 0.931, which is not low enough to\n",
@@ -169,13 +169,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": 5,
"id": "d1f0c695", "id": "facd6569",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.238822Z", "iopub.execute_input": "2023-08-21T02:29:17.627714Z",
"iopub.status.busy": "2023-08-07T00:19:50.238685Z", "iopub.status.busy": "2023-08-21T02:29:17.627617Z",
"iopub.status.idle": "2023-08-07T00:19:50.263876Z", "iopub.status.idle": "2023-08-21T02:29:17.651726Z",
"shell.execute_reply": "2023-08-07T00:19:50.263508Z" "shell.execute_reply": "2023-08-21T02:29:17.651448Z"
}, },
"lines_to_next_cell": 0 "lines_to_next_cell": 0
}, },
@@ -195,7 +195,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "3d8e0d96", "id": "4094daa7",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Since this is a simulated data set, we can create a $2 \\times 2$ table\n", "Since this is a simulated data set, we can create a $2 \\times 2$ table\n",
@@ -205,13 +205,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 6, "execution_count": 6,
"id": "7a9594a0", "id": "e89ef3eb",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.266708Z", "iopub.execute_input": "2023-08-21T02:29:17.653344Z",
"iopub.status.busy": "2023-08-07T00:19:50.266387Z", "iopub.status.busy": "2023-08-21T02:29:17.653256Z",
"iopub.status.idle": "2023-08-07T00:19:50.280831Z", "iopub.status.idle": "2023-08-21T02:29:17.662644Z",
"shell.execute_reply": "2023-08-07T00:19:50.280194Z" "shell.execute_reply": "2023-08-21T02:29:17.662346Z"
}, },
"lines_to_next_cell": 0 "lines_to_next_cell": 0
}, },
@@ -282,7 +282,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "9610c817", "id": "a10ba423",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Therefore, at level $\\alpha=0.05$, we reject 15 of the 50 false\n", "Therefore, at level $\\alpha=0.05$, we reject 15 of the 50 false\n",
@@ -299,19 +299,20 @@
"amounts to quite a weak signal, and it resulted in a high number of\n", "amounts to quite a weak signal, and it resulted in a high number of\n",
"Type II errors. Let’s instead simulate data with a stronger signal,\n", "Type II errors. Let’s instead simulate data with a stronger signal,\n",
"so that the ratio of the mean to the standard deviation for the false\n", "so that the ratio of the mean to the standard deviation for the false\n",
"null hypotheses equals $1$. We make only 10 Type II errors.\n" "null hypotheses equals $1$. We make only 10 Type II errors.\n",
" "
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 7, "execution_count": 7,
"id": "25f7fc5d", "id": "ae184aaf",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.283336Z", "iopub.execute_input": "2023-08-21T02:29:17.664327Z",
"iopub.status.busy": "2023-08-07T00:19:50.283198Z", "iopub.status.busy": "2023-08-21T02:29:17.664213Z",
"iopub.status.idle": "2023-08-07T00:19:50.317664Z", "iopub.status.idle": "2023-08-21T02:29:17.690928Z",
"shell.execute_reply": "2023-08-07T00:19:50.317356Z" "shell.execute_reply": "2023-08-21T02:29:17.690657Z"
}, },
"lines_to_next_cell": 0 "lines_to_next_cell": 0
}, },
@@ -394,7 +395,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "bb70c597", "id": "7ca15d3f",
"metadata": {}, "metadata": {},
"source": [ "source": [
" " " "
@@ -402,7 +403,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "f6953d33", "id": "9e9f5977",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## Family-Wise Error Rate\n", "## Family-Wise Error Rate\n",
@@ -417,13 +418,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 8, "execution_count": 8,
"id": "369b5bd3", "id": "0295fe68",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.319399Z", "iopub.execute_input": "2023-08-21T02:29:17.692568Z",
"iopub.status.busy": "2023-08-07T00:19:50.319268Z", "iopub.status.busy": "2023-08-21T02:29:17.692459Z",
"iopub.status.idle": "2023-08-07T00:19:50.674219Z", "iopub.status.idle": "2023-08-21T02:29:17.899403Z",
"shell.execute_reply": "2023-08-07T00:19:50.673887Z" "shell.execute_reply": "2023-08-21T02:29:17.899081Z"
} }
}, },
"outputs": [ "outputs": [
@@ -454,7 +455,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "3a81479e", "id": "fecaca9e",
"metadata": {}, "metadata": {},
"source": [ "source": [
"As discussed previously, even for moderate values of $m$ such as $50$,\n", "As discussed previously, even for moderate values of $m$ such as $50$,\n",
@@ -473,13 +474,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 9, "execution_count": 9,
"id": "9ce7a19f", "id": "406e59a8",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.676029Z", "iopub.execute_input": "2023-08-21T02:29:17.901146Z",
"iopub.status.busy": "2023-08-07T00:19:50.675909Z", "iopub.status.busy": "2023-08-21T02:29:17.901041Z",
"iopub.status.idle": "2023-08-07T00:19:50.715451Z", "iopub.status.idle": "2023-08-21T02:29:17.939312Z",
"shell.execute_reply": "2023-08-07T00:19:50.715145Z" "shell.execute_reply": "2023-08-21T02:29:17.939019Z"
} }
}, },
"outputs": [ "outputs": [
@@ -505,7 +506,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "7561e3a3", "id": "87bab88b",
"metadata": {}, "metadata": {},
"source": [ "source": [
"The $p$-values are low for Managers One and Three, and high for the\n", "The $p$-values are low for Managers One and Three, and high for the\n",
@@ -530,7 +531,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "5b608e46", "id": "d0c7a2a0",
"metadata": {}, "metadata": {},
"source": [ "source": [
"The `mult_test()` function takes $p$-values and a `method` argument, as well as an optional\n", "The `mult_test()` function takes $p$-values and a `method` argument, as well as an optional\n",
@@ -541,13 +542,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 10, "execution_count": 10,
"id": "de6cffed", "id": "d4f6a247",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.717476Z", "iopub.execute_input": "2023-08-21T02:29:17.941032Z",
"iopub.status.busy": "2023-08-07T00:19:50.717258Z", "iopub.status.busy": "2023-08-21T02:29:17.940919Z",
"iopub.status.idle": "2023-08-07T00:19:50.719841Z", "iopub.status.idle": "2023-08-21T02:29:17.943369Z",
"shell.execute_reply": "2023-08-07T00:19:50.719577Z" "shell.execute_reply": "2023-08-21T02:29:17.943081Z"
}, },
"lines_to_next_cell": 2 "lines_to_next_cell": 2
}, },
@@ -570,7 +571,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "5135c6b9", "id": "4d5bc7e7",
"metadata": {}, "metadata": {},
"source": [ "source": [
"The $p$-values `bonf` are simply the `fund_mini_pvalues` multiplied by 5 and truncated to be less than\n", "The $p$-values `bonf` are simply the `fund_mini_pvalues` multiplied by 5 and truncated to be less than\n",
@@ -580,13 +581,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 11, "execution_count": 11,
"id": "0de71500", "id": "01a29d71",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.721450Z", "iopub.execute_input": "2023-08-21T02:29:17.944859Z",
"iopub.status.busy": "2023-08-07T00:19:50.721342Z", "iopub.status.busy": "2023-08-21T02:29:17.944760Z",
"iopub.status.idle": "2023-08-07T00:19:50.723962Z", "iopub.status.idle": "2023-08-21T02:29:17.946888Z",
"shell.execute_reply": "2023-08-07T00:19:50.723691Z" "shell.execute_reply": "2023-08-21T02:29:17.946639Z"
} }
}, },
"outputs": [ "outputs": [
@@ -608,7 +609,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "1f0bc112", "id": "e68c9051",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Therefore, using Bonferroni’s method, we are able to reject the null hypothesis only for Manager\n", "Therefore, using Bonferroni’s method, we are able to reject the null hypothesis only for Manager\n",
@@ -622,13 +623,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 12, "execution_count": 12,
"id": "f7e87bdb", "id": "95454eb4",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.725450Z", "iopub.execute_input": "2023-08-21T02:29:17.948474Z",
"iopub.status.busy": "2023-08-07T00:19:50.725349Z", "iopub.status.busy": "2023-08-21T02:29:17.948372Z",
"iopub.status.idle": "2023-08-07T00:19:50.777055Z", "iopub.status.idle": "2023-08-21T02:29:17.990740Z",
"shell.execute_reply": "2023-08-07T00:19:50.776612Z" "shell.execute_reply": "2023-08-21T02:29:17.990464Z"
}, },
"lines_to_next_cell": 2 "lines_to_next_cell": 2
}, },
@@ -651,23 +652,24 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "f762fecd", "id": "587b5b48",
"metadata": {}, "metadata": {},
"source": [ "source": [
"As discussed previously, Manager One seems to perform particularly\n", "As discussed previously, Manager One seems to perform particularly\n",
"well, whereas Manager Two has poor performance." "well, whereas Manager Two has poor performance.\n",
" "
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 13, "execution_count": 13,
"id": "e88be376", "id": "1f1ac764",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.779763Z", "iopub.execute_input": "2023-08-21T02:29:17.992261Z",
"iopub.status.busy": "2023-08-07T00:19:50.779554Z", "iopub.status.busy": "2023-08-21T02:29:17.992149Z",
"iopub.status.idle": "2023-08-07T00:19:50.783798Z", "iopub.status.idle": "2023-08-21T02:29:17.995141Z",
"shell.execute_reply": "2023-08-07T00:19:50.783418Z" "shell.execute_reply": "2023-08-21T02:29:17.994894Z"
}, },
"lines_to_next_cell": 2 "lines_to_next_cell": 2
}, },
@@ -694,7 +696,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "88dbf0a6", "id": "e3c4b716",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Is there evidence of a meaningful difference in performance between\n", "Is there evidence of a meaningful difference in performance between\n",
@@ -705,13 +707,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 14, "execution_count": 14,
"id": "41149af6", "id": "298d975d",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.786752Z", "iopub.execute_input": "2023-08-21T02:29:17.996686Z",
"iopub.status.busy": "2023-08-07T00:19:50.786580Z", "iopub.status.busy": "2023-08-21T02:29:17.996590Z",
"iopub.status.idle": "2023-08-07T00:19:50.791095Z", "iopub.status.idle": "2023-08-21T02:29:17.999332Z",
"shell.execute_reply": "2023-08-07T00:19:50.790607Z" "shell.execute_reply": "2023-08-21T02:29:17.999076Z"
} }
}, },
"outputs": [ "outputs": [
@@ -733,7 +735,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "1aca6122", "id": "3908d7d2",
"metadata": {}, "metadata": {},
"source": [ "source": [
"The test results in a $p$-value of 0.038,\n", "The test results in a $p$-value of 0.038,\n",
@@ -757,13 +759,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 15, "execution_count": 15,
"id": "61aabda7", "id": "be117713",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:50.793577Z", "iopub.execute_input": "2023-08-21T02:29:18.000853Z",
"iopub.status.busy": "2023-08-07T00:19:50.793394Z", "iopub.status.busy": "2023-08-21T02:29:18.000747Z",
"iopub.status.idle": "2023-08-07T00:19:51.318999Z", "iopub.status.idle": "2023-08-21T02:29:18.487357Z",
"shell.execute_reply": "2023-08-07T00:19:51.318071Z" "shell.execute_reply": "2023-08-21T02:29:18.487078Z"
}, },
"lines_to_next_cell": 2 "lines_to_next_cell": 2
}, },
@@ -799,7 +801,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "e0084fc5", "id": "0fdf963f",
"metadata": {}, "metadata": {},
"source": [ "source": [
"The `pairwise_tukeyhsd()` function provides confidence intervals\n", "The `pairwise_tukeyhsd()` function provides confidence intervals\n",
@@ -817,13 +819,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 16, "execution_count": 16,
"id": "cbcad4de", "id": "537c4ea8",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:51.321200Z", "iopub.execute_input": "2023-08-21T02:29:18.489069Z",
"iopub.status.busy": "2023-08-07T00:19:51.321046Z", "iopub.status.busy": "2023-08-21T02:29:18.488949Z",
"iopub.status.idle": "2023-08-07T00:19:51.411142Z", "iopub.status.idle": "2023-08-21T02:29:18.570869Z",
"shell.execute_reply": "2023-08-07T00:19:51.410682Z" "shell.execute_reply": "2023-08-21T02:29:18.570427Z"
} }
}, },
"outputs": [ "outputs": [
@@ -845,7 +847,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "6278d13c", "id": "8590f246",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## False Discovery Rate\n", "## False Discovery Rate\n",
@@ -858,13 +860,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 17, "execution_count": 17,
"id": "b5842190", "id": "2c88ec87",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:51.413331Z", "iopub.execute_input": "2023-08-21T02:29:18.572454Z",
"iopub.status.busy": "2023-08-07T00:19:51.413176Z", "iopub.status.busy": "2023-08-21T02:29:18.572341Z",
"iopub.status.idle": "2023-08-07T00:19:51.848427Z", "iopub.status.idle": "2023-08-21T02:29:19.005707Z",
"shell.execute_reply": "2023-08-07T00:19:51.847956Z" "shell.execute_reply": "2023-08-21T02:29:19.005387Z"
} }
}, },
"outputs": [], "outputs": [],
@@ -876,7 +878,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "80fc2fcc", "id": "80e77fab",
"metadata": {}, "metadata": {},
"source": [ "source": [
"There are far too many managers to consider trying to control the FWER.\n", "There are far too many managers to consider trying to control the FWER.\n",
@@ -887,13 +889,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 18, "execution_count": 18,
"id": "7c9d8bed", "id": "b6d56819",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:51.850663Z", "iopub.execute_input": "2023-08-21T02:29:19.007847Z",
"iopub.status.busy": "2023-08-07T00:19:51.850523Z", "iopub.status.busy": "2023-08-21T02:29:19.007564Z",
"iopub.status.idle": "2023-08-07T00:19:51.854777Z", "iopub.status.idle": "2023-08-21T02:29:19.010742Z",
"shell.execute_reply": "2023-08-07T00:19:51.854196Z" "shell.execute_reply": "2023-08-21T02:29:19.010371Z"
} }
}, },
"outputs": [ "outputs": [
@@ -916,7 +918,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "4f73096d", "id": "b4662444",
"metadata": {}, "metadata": {},
"source": [ "source": [
"The *q-values* output by the\n", "The *q-values* output by the\n",
@@ -932,13 +934,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 19, "execution_count": 19,
"id": "bfa39f7c", "id": "b00da3a1",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:51.856795Z", "iopub.execute_input": "2023-08-21T02:29:19.012400Z",
"iopub.status.busy": "2023-08-07T00:19:51.856678Z", "iopub.status.busy": "2023-08-21T02:29:19.012298Z",
"iopub.status.idle": "2023-08-07T00:19:51.859719Z", "iopub.status.idle": "2023-08-21T02:29:19.015314Z",
"shell.execute_reply": "2023-08-07T00:19:51.859327Z" "shell.execute_reply": "2023-08-21T02:29:19.014978Z"
}, },
"lines_to_next_cell": 0 "lines_to_next_cell": 0
}, },
@@ -960,7 +962,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "ccb44c8d", "id": "fdccb808",
"metadata": {}, "metadata": {},
"source": [ "source": [
"We find that 146 of the 2,000 fund managers have a $q$-value below\n", "We find that 146 of the 2,000 fund managers have a $q$-value below\n",
@@ -976,13 +978,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 20, "execution_count": 20,
"id": "70b69b47", "id": "1c230117",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:51.861924Z", "iopub.execute_input": "2023-08-21T02:29:19.016857Z",
"iopub.status.busy": "2023-08-07T00:19:51.861522Z", "iopub.status.busy": "2023-08-21T02:29:19.016769Z",
"iopub.status.idle": "2023-08-07T00:19:51.864394Z", "iopub.status.idle": "2023-08-21T02:29:19.019332Z",
"shell.execute_reply": "2023-08-07T00:19:51.863987Z" "shell.execute_reply": "2023-08-21T02:29:19.019032Z"
}, },
"lines_to_next_cell": 2 "lines_to_next_cell": 2
}, },
@@ -1004,7 +1006,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "c8a969f4", "id": "6112239d",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Figure 13.6 displays the ordered\n", "Figure 13.6 displays the ordered\n",
@@ -1026,13 +1028,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 21, "execution_count": 21,
"id": "4c0ddea1", "id": "62289650",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:51.865978Z", "iopub.execute_input": "2023-08-21T02:29:19.021112Z",
"iopub.status.busy": "2023-08-07T00:19:51.865869Z", "iopub.status.busy": "2023-08-21T02:29:19.020904Z",
"iopub.status.idle": "2023-08-07T00:19:51.868792Z", "iopub.status.idle": "2023-08-21T02:29:19.023622Z",
"shell.execute_reply": "2023-08-07T00:19:51.868357Z" "shell.execute_reply": "2023-08-21T02:29:19.023338Z"
} }
}, },
"outputs": [], "outputs": [],
@@ -1051,7 +1053,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "ddeb3900", "id": "c36b13b7",
"metadata": {}, "metadata": {},
"source": [ "source": [
"We now reproduce the middle panel of Figure 13.6." "We now reproduce the middle panel of Figure 13.6."
@@ -1060,13 +1062,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 22, "execution_count": 22,
"id": "0314eac9", "id": "18b3c0ed",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:51.871473Z", "iopub.execute_input": "2023-08-21T02:29:19.025191Z",
"iopub.status.busy": "2023-08-07T00:19:51.871214Z", "iopub.status.busy": "2023-08-21T02:29:19.025074Z",
"iopub.status.idle": "2023-08-07T00:19:52.126671Z", "iopub.status.idle": "2023-08-21T02:29:19.262207Z",
"shell.execute_reply": "2023-08-07T00:19:52.126261Z" "shell.execute_reply": "2023-08-21T02:29:19.261823Z"
}, },
"lines_to_next_cell": 2 "lines_to_next_cell": 2
}, },
@@ -1096,7 +1098,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "83416f4a", "id": "d87198e4",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## A Re-Sampling Approach\n", "## A Re-Sampling Approach\n",
@@ -1110,13 +1112,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 23, "execution_count": 23,
"id": "b59b8137", "id": "eb79e606",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:52.129167Z", "iopub.execute_input": "2023-08-21T02:29:19.264174Z",
"iopub.status.busy": "2023-08-07T00:19:52.128842Z", "iopub.status.busy": "2023-08-21T02:29:19.264030Z",
"iopub.status.idle": "2023-08-07T00:19:52.208320Z", "iopub.status.idle": "2023-08-21T02:29:19.339232Z",
"shell.execute_reply": "2023-08-07T00:19:52.207936Z" "shell.execute_reply": "2023-08-21T02:29:19.338912Z"
}, },
"lines_to_next_cell": 2 "lines_to_next_cell": 2
}, },
@@ -1145,7 +1147,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "5534c8d4", "id": "659ee2b8",
"metadata": {}, "metadata": {},
"source": [ "source": [
"There are four classes of cancer. For each gene, we compare the mean\n", "There are four classes of cancer. For each gene, we compare the mean\n",
@@ -1161,13 +1163,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 24, "execution_count": 24,
"id": "96fb2f61", "id": "1afbcf47",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:52.210235Z", "iopub.execute_input": "2023-08-21T02:29:19.341009Z",
"iopub.status.busy": "2023-08-07T00:19:52.210101Z", "iopub.status.busy": "2023-08-21T02:29:19.340889Z",
"iopub.status.idle": "2023-08-07T00:19:52.215004Z", "iopub.status.idle": "2023-08-21T02:29:19.344670Z",
"shell.execute_reply": "2023-08-07T00:19:52.214604Z" "shell.execute_reply": "2023-08-21T02:29:19.344391Z"
}, },
"lines_to_next_cell": 2 "lines_to_next_cell": 2
}, },
@@ -1195,7 +1197,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "3131124e", "id": "61f24919",
"metadata": {}, "metadata": {},
"source": [ "source": [
"However, this $p$-value relies on the assumption that under the null\n", "However, this $p$-value relies on the assumption that under the null\n",
@@ -1214,13 +1216,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 25, "execution_count": 25,
"id": "fdc229fa", "id": "f73f4c6d",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:52.217206Z", "iopub.execute_input": "2023-08-21T02:29:19.346368Z",
"iopub.status.busy": "2023-08-07T00:19:52.217085Z", "iopub.status.busy": "2023-08-21T02:29:19.346227Z",
"iopub.status.idle": "2023-08-07T00:19:54.776066Z", "iopub.status.idle": "2023-08-21T02:29:21.776569Z",
"shell.execute_reply": "2023-08-07T00:19:54.775642Z" "shell.execute_reply": "2023-08-21T02:29:21.776267Z"
}, },
"lines_to_next_cell": 2 "lines_to_next_cell": 2
}, },
@@ -1253,7 +1255,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "c7fc4557", "id": "a97f74af",
"metadata": {}, "metadata": {},
"source": [ "source": [
"This fraction, 0.0398,\n", "This fraction, 0.0398,\n",
@@ -1265,13 +1267,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 26, "execution_count": 26,
"id": "e3894695", "id": "062daf19",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:54.778563Z", "iopub.execute_input": "2023-08-21T02:29:21.778366Z",
"iopub.status.busy": "2023-08-07T00:19:54.778388Z", "iopub.status.busy": "2023-08-21T02:29:21.778242Z",
"iopub.status.idle": "2023-08-07T00:19:55.017161Z", "iopub.status.idle": "2023-08-21T02:29:21.990476Z",
"shell.execute_reply": "2023-08-07T00:19:55.016821Z" "shell.execute_reply": "2023-08-21T02:29:21.989965Z"
}, },
"lines_to_next_cell": 0 "lines_to_next_cell": 0
}, },
@@ -1307,7 +1309,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "3bd21158", "id": "e81b939b",
"metadata": {}, "metadata": {},
"source": [ "source": [
"The re-sampling-based null distribution is almost identical to the theoretical null distribution, which is displayed in red.\n", "The re-sampling-based null distribution is almost identical to the theoretical null distribution, which is displayed in red.\n",
@@ -1325,13 +1327,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 27, "execution_count": 27,
"id": "3b7392cb", "id": "6d14fcad",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:19:55.019036Z", "iopub.execute_input": "2023-08-21T02:29:21.992665Z",
"iopub.status.busy": "2023-08-07T00:19:55.018920Z", "iopub.status.busy": "2023-08-21T02:29:21.992515Z",
"iopub.status.idle": "2023-08-07T00:39:19.291005Z", "iopub.status.idle": "2023-08-21T02:34:05.930300Z",
"shell.execute_reply": "2023-08-07T00:39:19.287314Z" "shell.execute_reply": "2023-08-21T02:34:05.929181Z"
} }
}, },
"outputs": [], "outputs": [],
@@ -1358,7 +1360,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "1b92df1b", "id": "06286699",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Next, we compute the number of rejected null hypotheses $R$, the\n", "Next, we compute the number of rejected null hypotheses $R$, the\n",
@@ -1371,13 +1373,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 28, "execution_count": 28,
"id": "cac15616", "id": "8f0ec909",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:39:19.314420Z", "iopub.execute_input": "2023-08-21T02:34:05.935513Z",
"iopub.status.busy": "2023-08-07T00:39:19.313452Z", "iopub.status.busy": "2023-08-21T02:34:05.935323Z",
"iopub.status.idle": "2023-08-07T00:39:19.544251Z", "iopub.status.idle": "2023-08-21T02:34:06.118079Z",
"shell.execute_reply": "2023-08-07T00:39:19.543932Z" "shell.execute_reply": "2023-08-21T02:34:06.117633Z"
} }
}, },
"outputs": [], "outputs": [],
@@ -1394,7 +1396,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "f6779ea0", "id": "e26b64c6",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Now, for any given FDR, we can find the genes that will be\n", "Now, for any given FDR, we can find the genes that will be\n",
@@ -1412,13 +1414,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 29, "execution_count": 29,
"id": "9661eb10", "id": "f11339e5",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:39:19.546693Z", "iopub.execute_input": "2023-08-21T02:34:06.120138Z",
"iopub.status.busy": "2023-08-07T00:39:19.546543Z", "iopub.status.busy": "2023-08-21T02:34:06.119994Z",
"iopub.status.idle": "2023-08-07T00:39:19.549970Z", "iopub.status.idle": "2023-08-21T02:34:06.123846Z",
"shell.execute_reply": "2023-08-07T00:39:19.549697Z" "shell.execute_reply": "2023-08-21T02:34:06.123478Z"
} }
}, },
"outputs": [ "outputs": [
@@ -1457,7 +1459,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "001e3fc1", "id": "e145621b",
"metadata": {}, "metadata": {},
"source": [ "source": [
"At an FDR threshold of 0.2, more genes are selected, at the cost of having a higher expected\n", "At an FDR threshold of 0.2, more genes are selected, at the cost of having a higher expected\n",
@@ -1467,13 +1469,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 30, "execution_count": 30,
"id": "18ad4900", "id": "d2600773",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:39:19.552090Z", "iopub.execute_input": "2023-08-21T02:34:06.126460Z",
"iopub.status.busy": "2023-08-07T00:39:19.552004Z", "iopub.status.busy": "2023-08-21T02:34:06.126346Z",
"iopub.status.idle": "2023-08-07T00:39:19.554743Z", "iopub.status.idle": "2023-08-21T02:34:06.129561Z",
"shell.execute_reply": "2023-08-07T00:39:19.554473Z" "shell.execute_reply": "2023-08-21T02:34:06.129124Z"
} }
}, },
"outputs": [ "outputs": [
@@ -1524,7 +1526,7 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "8767f70c", "id": "32e600ff",
"metadata": {}, "metadata": {},
"source": [ "source": [
"The next line generates Figure 13.11, which is similar\n", "The next line generates Figure 13.11, which is similar\n",
@@ -1535,13 +1537,13 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 31, "execution_count": 31,
"id": "28c276b6", "id": "924b7705",
"metadata": { "metadata": {
"execution": { "execution": {
"iopub.execute_input": "2023-08-07T00:39:19.556715Z", "iopub.execute_input": "2023-08-21T02:34:06.131323Z",
"iopub.status.busy": "2023-08-07T00:39:19.556515Z", "iopub.status.busy": "2023-08-21T02:34:06.131207Z",
"iopub.status.idle": "2023-08-07T00:39:19.650514Z", "iopub.status.idle": "2023-08-21T02:34:06.216626Z",
"shell.execute_reply": "2023-08-07T00:39:19.650181Z" "shell.execute_reply": "2023-08-21T02:34:06.216270Z"
}, },
"lines_to_next_cell": 0 "lines_to_next_cell": 0
}, },
@@ -1566,18 +1568,18 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "e4b5d621", "id": "b9f54695",
"metadata": {}, "metadata": {},
"source": [ "source": [
"\n" " \n"
] ]
} }
], ],
"metadata": { "metadata": {
"jupytext": { "jupytext": {
"cell_metadata_filter": "-all", "cell_metadata_filter": "-all",
"formats": "ipynb,Rmd", "main_language": "python",
"main_language": "python" "notebook_metadata_filter": "-all"
}, },
"language_info": { "language_info": {
"codemirror_mode": { "codemirror_mode": {