From 0a035acc0571f37567b0b62b93119322fb1c887e Mon Sep 17 00:00:00 2001 From: "behinger (s-ccs 001)" Date: Thu, 28 Sep 2023 12:51:54 +0000 Subject: [PATCH] merge --- cheatsheets/julia.qmd | 2 +- .../1_mon/firststeps/firststeps_handout.qmd | 16 +- material/3_wed/vis/handout.ipynb | 199 +++++++++++++++--- material/3_wed/vis/handout.qmd | 161 ++++++++++++-- material/3_wed/vis/tasks.qmd | 161 ++++++++++++++ 5 files changed, 485 insertions(+), 54 deletions(-) diff --git a/cheatsheets/julia.qmd b/cheatsheets/julia.qmd index 31dfcc3..2e2b938 100644 --- a/cheatsheets/julia.qmd +++ b/cheatsheets/julia.qmd @@ -11,7 +11,7 @@ | What is ...? | `typeof(obj)` | `type(obj)` | `class(obj)` | | Is it really a ...? | `isa(obj, SomeType)` | `isinstance(obj, SomeType)` | `is(obj, SomeType)` | -## debugging +## debugging {#debugging} | | | |------------------------------------|------------------------------------| diff --git a/material/1_mon/firststeps/firststeps_handout.qmd b/material/1_mon/firststeps/firststeps_handout.qmd index 7241231..cf65da3 100644 --- a/material/1_mon/firststeps/firststeps_handout.qmd +++ b/material/1_mon/firststeps/firststeps_handout.qmd @@ -413,5 +413,19 @@ Macros allow to programmers to edit the actual code **before** it is run. We wil a = "123" @show a ``` + + ## Debugging -XXX \ No newline at end of file + +### Debug messages +```julia +@debug "my debug message" +ENV["JULIA_DEBUG"] = Main +ENV["JULIA_DEBUG"] = MyPackage + +``` + +### Debugger proper: +[Cheatsheet for debugging](../../../cheatsheets/julia.qmd#debugging) + +In most cases, `@run myFunction(1,2,3)` is sufficient. \ No newline at end of file diff --git a/material/3_wed/vis/handout.ipynb b/material/3_wed/vis/handout.ipynb index 236d912..c9df3e4 100644 --- a/material/3_wed/vis/handout.ipynb +++ b/material/3_wed/vis/handout.ipynb @@ -6,10 +6,13 @@ "source": [ "---\n", "\n", + "\n", + "\n", + "\n", "jupyter: julia-1.9\n", "---" ], - "id": "d0dd3b54" + "id": "18cb9530" }, { "cell_type": "markdown", @@ -26,6 +29,32 @@ "\n", "I will use `GLMakie` or `CairoMakie`. To switch use `CairoMakie.activate!()`\n", "\n", + "## Standard plotting commands\n" + ], + "id": "d83f21c4" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "f = Figure()\n", + "x = rand(100)\n", + "y = rand(100)\n", + "\n", + "scatter(f[1,1],x,y)\n", + "lines(f[1,2],x,y)\n", + "hist(f[2,1],x)\n", + "density!(f[2,1],x) # inplace -> add to current plot\n", + "stem(f[2,2],x)" + ], + "id": "811907e9", + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "## Layouts for scientific figures\n", "\n", "# Pluto.jl for easy interactivity\n", @@ -46,7 +75,7 @@ "\n", "### Loading data" ], - "id": "0ca52ae6" + "id": "69dda8de" }, { "cell_type": "code", @@ -59,7 +88,7 @@ "penguins = dropmissing(DataFrame(PalmerPenguins.load()))\n", "first(penguins, 6)" ], - "id": "56b4e637", + "id": "334f10aa", "execution_count": null, "outputs": [] }, @@ -82,42 +111,150 @@ "### AoG basics\n", "\n", "\n", - "`data * mapping * visual`\n", - "\n", - "```julia\n", + "`data * mapping * visual`\n" + ], + "id": "0bfbe711" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ " vis_pen = data(penguins) * mapping(:bill_length_mm, :bill_depth_mm) * visual(Scatter) \n", - " draw(vis_pen)\n", - "```\n", - "\n", - "### Adding color\n", - "\n", - "```julia\n", + " draw(vis_pen)" + ], + "id": "cc740038", + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Adding color\n" + ], + "id": "3fd4693f" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ "vis_pencolor = data(penguins) * mapping(:bill_length_mm, :bill_depth_mm, color = :species) * visual(Scatter)\n", - "draw(vis_pencolor)\n", - "\n", - "```\n", - "But that is a bit redundant, you can shortcut this, by reusing existing mappings / inputs:\n", - "```julia\n", + "draw(vis_pencolor)" + ], + "id": "b8e715a3", + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "But that is a bit redundant, you can shortcut this, by reusing existing mappings / inputs:" + ], + "id": "ca61723b" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ "vis_pencolor2 = vis_pen * mapping(color=:species)\n", - "draw(vis_pencolor2)\n", - "\n", - "```\n", - "\n", + "draw(vis_pencolor2)" + ], + "id": "84f60a09", + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "### Why `Algebra`OfGraphics?\n", "\n", "Follows some algebraic rules of multiplying out sums\n", "\n", - "`data * mapping * visual(Scatter+Lines)`\n", - "\n", - "```julia\n", - "\n", - " vis_pen = data(penguins) * mapping(:bill_length_mm, :bill_depth_mm) * visual(Scatter+Lines) \n", - "```\n", - "\n", - "### Faceting\n", - "jl plt = penguin_bill * layers * mapping(color = :species, col = :sex)" + "`data * mapping * (visual(Scatter)+visual(Lines))`\n" ], - "id": "3ead8444" + "id": "cad8ff41" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "data(penguins) * mapping(:bill_length_mm, :bill_depth_mm) * (visual(Scatter)+visual(Lines)) |> draw" + ], + "id": "84fc3a4e", + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Faceting" + ], + "id": "1b7f38f7" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "data(penguins) * mapping(:bill_length_mm, :bill_depth_mm) * mapping(color = :species, col = :sex) |> draw" + ], + "id": "90ba7bee", + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "data(penguins) * mapping(:bill_length_mm, :bill_depth_mm) * mapping(color = :species, col = :sex,row=:body_mass_g => x-> x>3500) |> draw" + ], + "id": "84d76ff4", + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Linear & Non-linear summaries" + ], + "id": "ed061ded" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "data(penguins) * mapping(:bill_length_mm, :bill_depth_mm, color=:species) * (linear() + visual(Scatter)) |> draw" + ], + "id": "bfaf33f5", + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "data(penguins) * mapping(:bill_length_mm, :bill_depth_mm, color=:species) * (smooth() + visual(Scatter)) |> draw" + ], + "id": "00aaa0bd", + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Interactivity\n", + "\n", + "With Makie.jl, two ways of interactivity:\n", + "\n", + "**Observables** - very general way, a little bit more verbose\n", + "\n", + "**Pluto.jl Sliders** - very simple, need to redraw plot everytime^[it is technically possible t combine Pluto with Observables, but it is a bit buggy] \n" + ], + "id": "fb64ae39" } ], "metadata": { diff --git a/material/3_wed/vis/handout.qmd b/material/3_wed/vis/handout.qmd index 65d9007..5f19126 100644 --- a/material/3_wed/vis/handout.qmd +++ b/material/3_wed/vis/handout.qmd @@ -1,7 +1,4 @@ ---- -jupyter: julia-1.9 ---- # Makie.jl ## Backends @@ -13,9 +10,135 @@ Four backends: I will use `GLMakie` or `CairoMakie`. To switch use `CairoMakie.activate!()` +## Standard plotting + +```julia +f = Figure() +x = rand(100) +y = rand(100) + +scatter(f[1,1],x,y) +lines(f[1,2],x,y) +hist(f[2,1],x) +density!(f[2,1],x) # inplace -> add to current plot +stem(f[2,2],x) +``` + ## Layouts for scientific figures -# Pluto.jl for easy interactivity +Makie has the best layouting tool I have ever used. [full tutorial here](https://docs.makie.org/stable/tutorials/layout-tutorial/) + +```julia +f = Figure() + +# we plan to generate two subfigures (with subplots each) - better to generate two "separate" layouts +ga = f[1, 1] = GridLayout() +gb = f[2, 1] = GridLayout() + +axtop = Axis(ga[1,1]) +axmain = Axis(ga[2, 1], xlabel = "before", ylabel = "after") +axright = Axis(ga[2, 2]) + + +labels = ["treatment", "placebo", "control"] +d = randn(3, 100, 2) .+ [1, 3, 5] + +for (label, col) in zip(labels, eachslice(d, dims = 1)) + scatter!(axmain, col, label = label) + density!(axtop, col[:, 1]) + density!(axright, col[:, 2], direction = :y) +end + +linkyaxes!(axmain, axright) +linkxaxes!(axmain, axtop) +hidedecorations!(axtop, grid = false) +hidedecorations!(axright, grid = false) + +#--- add a legend +leg = Legend(ga[1, 2], axmain) + +# absolute size for now :shrug: +leg.width =100 +leg.height =100 + +leg.tellwidth = true +leg.tellheight = true + +#---- +# second plot +ax,h = heatmap(gb[1,1],rand(100,10),colorrange = [0,1]) +ax2,h2 = heatmap(gb[1,2],rand(100,10),colorrange = [0,1]) +cb = Colorbar(gb[1,3],h) +cb.alignmode = Mixed(right=0) + +#---- +# Labels +Label(ga[1, 1, TopLeft()], "A1", font = :bold, padding = (0, 0, 5, 0)) +Label(ga[2, 1, TopLeft()], "A2", font = :bold, padding = (0, 0, 5, 0)) +Label(ga[2, 2, TopLeft()], "A3", font = :bold, padding = (0, 0, 5, 0)) + +Label(gb[1, 1, TopLeft()], "B", font = :bold, padding = (0, 0, 5, 0)) + +#--- +# top plot needs more space +rowsize!(f.layout,2,Relative(0.3)) + +#--- +f +``` +# Interactivity + +With Makie.jl, two ways of interactivity: + +**Observables** - very general way, a little bit more verbose + +**Pluto.jl Sliders** - very simple, need to redraw plot everytime^[it is technically possible to combine Pluto with Observables, but it is a bit buggy] + + +## Pluto.jl + +### Installation / Start +```julia +]add Pluto +Pluto.run() +``` +::: {.callout-tip} +If you need remote access, run it via `Pluto.run(host="0.0.0.0")` +::: + +### Sliders + +A slider is defined like this: +```julia +@bind yourVarName PlutoUI.Slider(from:to) # from:step:to is optional, step by def 1 +``` + +if you move the slider, `yourVarName` + all cells that depend on that variable are automatically recalculated. Quick & dirty way to generate an interactive plot + +## Bonus: Makie Interactivity + +There is another way to get to interactivity. Using `Observables.jl` + +To provide a simple example of the logic: +```julia +using GLMakie + +x = rand(10_000) +obs_ix = Observable(1) # index to plot until +scatter(@lift(x[1:obs_ix])) # non-interactive example # <1> + +f = Figure() +obs_sl = GLMakie.Slider(f[2,1],range=1:length(x)) +y = @lift(x[1:$(obs_sl.value)]) +ax,s = scatter(f[1,1],y) +xlims!(ax,0,length(x)) + +``` +1. `@lift` does the heavy lifting (hrhr) here. It adds a listener to `obs_ix`, whenever that value is changed, the value of the output of `@lift` is changed as well +## Task 2: Interactivity +[Click here for the next task](tasks.qmd#2) + + # Grammar of Graphics @@ -24,15 +147,15 @@ The grammar of graphics is a convenient way to build common explorative plots. For example: -#### For ggplot enthusiasts: +## For ggplot enthusiasts You could use [TidierPlots.jl - a ggplot clone](https://github.com/TidierOrg/TidierPlots.jl) -Check out the [../../../../cheatsheets/ggplotAOG.qmd]: +Check out the [AoG/GGplot cheatsheet](../../../../cheatsheets/ggplotAOG.qmd): ## AlgebraOfGraphics.jl ### Loading data -```{julia} +```julia using GLMakie # backend using AlgebraOfGraphics using PalmerPenguins, DataFrames # example dataset @@ -58,20 +181,20 @@ Tidy data make your visualization life much easier as you will see! `data * mapping * visual` -```{julia} +```julia vis_pen = data(penguins) * mapping(:bill_length_mm, :bill_depth_mm) * visual(Scatter) draw(vis_pen) ``` ### Adding color -```{julia} +```julia vis_pencolor = data(penguins) * mapping(:bill_length_mm, :bill_depth_mm, color = :species) * visual(Scatter) draw(vis_pencolor) ``` But that is a bit redundant, you can shortcut this, by reusing existing mappings / inputs: -```{julia} +```julia vis_pencolor2 = vis_pen * mapping(color=:species) draw(vis_pencolor2) @@ -83,34 +206,30 @@ Follows some algebraic rules of multiplying out sums `data * mapping * (visual(Scatter)+visual(Lines))` -```{julia} +```julia data(penguins) * mapping(:bill_length_mm, :bill_depth_mm) * (visual(Scatter)+visual(Lines)) |> draw ``` ### Faceting -```{julia} +```julia data(penguins) * mapping(:bill_length_mm, :bill_depth_mm) * mapping(color = :species, col = :sex) |> draw ``` -```{julia} +```julia data(penguins) * mapping(:bill_length_mm, :bill_depth_mm) * mapping(color = :species, col = :sex,row=:body_mass_g => x-> x>3500) |> draw ``` ### Linear & Non-linear summaries -```{julia} +```julia data(penguins) * mapping(:bill_length_mm, :bill_depth_mm, color=:species) * (linear() + visual(Scatter)) |> draw ``` -```{julia} +```julia data(penguins) * mapping(:bill_length_mm, :bill_depth_mm, color=:species) * (smooth() + visual(Scatter)) |> draw ``` -# Interactivity +## Task 3 -With Makie.jl, two ways of interactivity: - -**Observables** - very general way, a little bit more verbose - -**Pluto.jl Sliders** - very simple, need to redraw plot everytime^[it is technically possible t combine Pluto with Observables, but it is a bit buggy] \ No newline at end of file +[Click here for the next task](tasks.qmd#3) \ No newline at end of file diff --git a/material/3_wed/vis/tasks.qmd b/material/3_wed/vis/tasks.qmd index e69de29..87b1c46 100644 --- a/material/3_wed/vis/tasks.qmd +++ b/material/3_wed/vis/tasks.qmd @@ -0,0 +1,161 @@ +# Setting up Pluto.jl + +Pluto is nice as you can prototype pretty fast. + +::: callout-important +Pluto.jl has its own dependency management included! + +If you want to add packages that are not registered, you have to activate your own environment. For example + +```julia +using Pkg +Pkg.activate(mktempdir()) +Pkg.add("/path/to/your/package/CoolPackage") +Pkg.add(url="https://github.com/username/MyPackage.jl") + +using CoolPackage,MyPackage +``` + +::: + +To run pluto in the first place use: + +``` julia +]add Pluto +Pluto.run() +``` + + +# Task 1: Visualize some statistic properties {#1} + +## 1. Data + +### Generate 500 normally distributed samples + +::: callout-tip +You might want to make your results reproducible by fixing some seeds for the random generators. The two most common random generators used in julia are `Random.MersenneTwister` and `StableRNGs.StableRNG` - For this execrise I would recommend the latter (even though MersenneTwister is much more common to be used), thus run: + +``` julia +using StableRNGs +randn(StableRNG(1),100) +``` + +to get 100 random numbers. +::: + +Scale the random numbers to fullfill `std(x) ≈ 10` + +### functionize it +Next wrap that code in a function `simulate` which takes two arguments, a random seed and the number of samples + +## 2. cumulative mean +Calculate the cumulative mean of a single simulation. save it to a variable + +Note that there is no `cummean` function, but clever element-wise division in combination with `cumsum` should lead you there - or you just use a loop :shrug: + +::: {.callout-tip collapse="true"} +## click to show solution + + `cumsum(x) ./ 1:length(x)` +::: + +## 3. Plotting! +Now for your first plot. Use a `scatter` plot to visualize the cummulative mean output, if you do not generate a `Figure()` + `ax = f[1,1] = Axis(f)` manually, you can get it back by the scatter call. `f,ax,s = scatter()`. This is helpful as we later want to extend the `Axis` and `Figure` with other plot elements + +Use `hlines!` to add a horizontal line at your "true" value + +## 4. Subplot +### simulate repeatedly +Let's simulate 1000x datasets, each with a different seed, and take the mean over all simulated values + +::: {.callout-tip collapse="true"} +## click to show tip +An easy way to call a function many times is to broadcast it on an array e.g. `1:1000` - you could also use `map` to do it, but I don't think it is as clear :) +::: + + + +::: {.callout-tip collapse="true"} +## click to show solution + + `simulate.(1:1000,nmax)` +::: + +### Mean it +calculate the mean of each simulation + +::: {.callout-tip collapse="true"} +## click to show solution + + ```julia +using Statistics +mean.(simulate.(1:1000,nmax)) + +# or +sum.(...) ./ nmax + ``` +::: + +### Add it as a subplot +We want to add a histogram of the 1000 means to the plot. + +1. Add a new Axis to `f[1,2]` +2. use it to plot the histogram of the means via `hist!` - don't forget to change the `direction=:x` to flip the histogram +3. link the axes using `linkaxes` + +## 5. Prettify it +There are some simple tricks to make a plot look nicer: + +- remove the "box" using `hidespines!(ax,:r,:t) +- resize the right sub-plot to be smaller `colsize!` and `Relative(X)` +- hide the x-grid (type `ax.`+ `TAB` to find all possible attributes) +- hide the `xlabels` + `xticks` + `bottomspine` from the right subplot +- add two Labels `(A)` and `(B)` to the plot +- Bonus: use `color` to color the cummulative sum samples according to how many samples went into that sum. `colormap=:Reds` looks good to me! + +::: {.callout-tip collapse="true"} +## Bonus: Click for more fancy labels + +You can create a slightly fancier label by adding a circle around it :) +```julia + Label(f[1,2,TopLeft()],"B",padding=[0,0,5,0]) + Label(f[1,2,TopLeft()],"⭕",padding=[0,0,8,0],fontsize=30) +``` +::: + +# Task 2: Interactivity! {#2} + + +Using the `Pluto.jl` reactive backend, changing a value in some cell will automatically update all other cells - including plots. + +We can use Sliders instead of fixing the parameters of the simulation + +A slider is defined like this: +```julia +@bind yourVarName PlutoUI.Slider(from:to) # from:step:to is optional, step by def 1 +``` + +## Adding interactivity via sliders + +1. Define a slider that controls the number of samples from 1:500 +2. Define a second slider that adds a constant offset to all values of the simulation simulation +3. make sure to fix the x/y-limits to get a nice looking plot :-) + +:::{.callout-tip collapse="true"} +## Bonus: Advanced slider management + +After understanding the slightly awkward syntax, the following gives a nice collection of Sliders, Checkboxes, Widgets etc. with at the same time being drag-and-dropable and in a sidebar. Neat! + +```julia +using PlutoExtras + +PlutoExtras.BondTable([ + PlutoExtras.@BondsList "Sliders" let + "name A" = @bind(varA,PlutoUI.Slider(1:500)) + "name B" = @bind(varB, PlutoUI.Slider(-5:5)) + end + ]) +``` +::: + +# Task 3: AlgebraOfGraphics \ No newline at end of file