---

---




# Environments
### What is a environment
- A list of "installed" packages^[libraries, dlls, .so etc.] at certain versions
- sometimes includes the operation system

### **Reproducible** vs. **Replicable**
- **Reproducible**: Someone else can get the same results given your code + data
- **Replicable**: Someone else can repeat the whole study on new data

### Why do I need it?
- Version control
    - managing multiple projects
    - testing out things
    - your Toolbox needs other packages than e.g. your test-environemnt
- Reproducibility
    - other researchers want to know what packages to install
- Collaboration
    - you want to work on the same data / codebase




## Environments in Julia
Every folder with an `Project.toml` file has it's own environment (see below)

The "base" environment is active by default:

``` julia
(@v1.9) pkg>
```

Keep this as empty+tidy as possible! 

Tip: (you could also start julia by `julia --project="."`)


### Typical commands

##### `activate`

Use `activate .` or `activate ./path/to` creates a new `Project.toml` in the selected folder (`.` means current folder), or activates it, if it already exists.

##### `status` (`st`)

Shows the currently installed packages

##### `add`

Multiple ways to add packages to the `Project.toml`:

-   `add UnicodePlots`
-   `add https://github.com/JuliaPlots/UnicodePlots.jl`
-   specify branch: `add UnicodePlots#unicodeplots-docs`
-   specify version `add UnicodePlots@3.3`
-   `add ./path/to/localPackage`

::: callout-note
Folders have to be git-repositories, see below. Probably better use `develop`
:::

##### `remove` (`rm`)

remove package from `Project.toml` (not from `~/.julia`, use `gc` - garbage collect for this)

##### `develop`

-   `dev --local UnicodePlots`
-   `dev ./Path/To/LocalPackage/`

::: callout-note
You can't select a branch with `dev` and need to do it manually
:::

::: callout-note
You are asking for the difference of `dev ./Path/Package` and `add ./Path/Package`? Good question! `dev` will always track the actual content of the folder - whereas `add` will make a "snapshot" of the last commit in that folder (has to be an git for `add`!). And you have to use `]up` to actually update to new changes
:::



##### `pin` / `free`

You can pin versions of packages, so that they are not updated. Unpin with `free` - also undo `develop` by using `free`

##### `instantiate` / `resolve`

`instantiate` setup all dependencies in the given `Project.toml`+`Manifest.toml`

`resolve` update the `Manifest.toml` to respect the local setup


# Project vs. Package

| | Project |  Package |
| --- | --- | ---- |
|  installable/reuse?   |   :white_check_mark:  |   :negative_squared_cross_mark:    |
| should be reproducible | :white_check_mark: | :negative_squared_cross_mark:   |
| produces something? | :white_check_mark:   | :negative_squared_cross_mark:  |
| compatabilities declared? | :negative_squared_cross_mark:   | :white_check_mark: |
| formal requirements in julia? | :negative_squared_cross_mark:  | :white_check_mark: |

## `Project.toml` & `Manifest.toml`
#### 📄Project.toml
The "big picture": keeps track of user-added dependencies (+ compatabilities + header)

```
[deps]
PythonCall = "6099a3de-0909-46bc-b1f4-468b9a2dfc0d"
RCall = "6f49c342-dc21-5d91-9882-a32aef131414"
```


#### 📄Manifest.toml
The "details": keeps track of all versions of all dependencies, and dependencies of dependencies
```
julia_version = "1.9.2"

[[deps.AbstractPlutoDingetjes]]
deps = ["Pkg"]
git-tree-sha1 = "8eaf9f1b4921132a4cff3f36a1d9ba923b14a481"
uuid = "6e696c72-6542-2067-7265-42206c756150"
version = "1.1.4"

[[deps.ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"
version = "1.1.1"

[[deps.BibInternal]]
git-tree-sha1 = "3a760b38ba8da19e64d29244f06104823ff26f25"
uuid = "2027ae74-3657-4b95-ae00-e2f7d55c3e64"
version = "0.3.4"

[...]
```


## Projects in Julia

Formally, projects don't have specific requirements. You should activate an environment (`Project.toml`+`Manifest.toml`) in the main folder though. I recommend the following minimal structure:

-   `./src/` - all functions should go there
-   `./scripts/` - all actual scripts should go here,
-   `./README.md` - Write what this is about, who you are etc.
-   `./Project.toml` - Your explicit dependencies
-   `./Manifest.toml` - Your implicit dependencies + versions <-- this makes it reproducible!

::: callout-tip
One recommendation is to use `DrWatson.initialize_project([path])` to start a new project - it will generate a nice folder structure + provide some other helpful `DrWatson.jl` features.



:::


:::{.callout-tip collapse="true"}
## Click to expand the full datastructure
```
│projectdir          <- Project's main folder. It is initialized as a Git
│                       repository with a reasonable .gitignore file.
│
├── _research        <- WIP scripts, code, notes, comments,
│   |                   to-dos and anything in an alpha state.
│   └── tmp          <- Temporary data folder.
│
├── data             <- **Immutable and add-only!**
│   ├── sims         <- Data resulting directly from simulations.
│   ├── exp_pro      <- Data from processing experiments.
│   └── exp_raw      <- Raw experimental data.
│
├── plots            <- Self-explanatory.
├── notebooks        <- Jupyter, Weave or any other mixed media notebooks.
│
├── papers           <- Scientific papers resulting from the project.
│
├── scripts          <- Various scripts, e.g. simulations, plotting, analysis,
│   │                   The scripts use the `src` folder for their base code.
│   └── intro.jl     <- Simple file that uses DrWatson and uses its greeting.
│
├── src              <- Source code for use in this project. Contains functions,
│                       structures and modules that are used throughout
│                       the project and in multiple scripts.
│
├── README.md        <- Optional top-level README for anyone using this project.
├── .gitignore       <- by default ignores _research, data, plots, videos,
│                       notebooks and latex-compilation related files.
│
├── Manifest.toml    <- Contains full list of exact package versions used currently.
└── Project.toml     <- Main project file, allows activation and installation.
                        Includes DrWatson by default.
```
:::


## Packages in Julia
Several thousand packages exist in Julia already. Take a thorough look before starting something new!



### Minimal requirements for `]add` to work

Minimal structure

One git-repository containing:

-   `./src/MyStatsPackage.jl`
    -   (`module MyStatsPackage`)
-   `./Project.toml`
    -   `name = "MyStatsPackage"`
    -   `uuid ="b4cd1eb8-1e24-11e8-3319-93036a3eb9f3"`
    -   (`[compat]` entries)
    -   (`version= "0.1.0"`)


### Additional requirements to register

Julia supports many registries (you can host your own!), which are just fancy GITs that index what version is available at what git-url for each registered package.

The default registry is [JuliaRegistries/General](https://github.com/JuliaRegistries/General).

[To register at the general registiry, you need additionally:](https://juliaregistries.github.io/RegistryCI.jl/stable/guidelines/):

- `[compat]` entries for all dependencies
- a `version=`
- a supported license
- Some restrictions on the name (e.g. nothing with `Julia`, only ASCII, etc.)


## Let's generate our first package!

```julia
] generate MyStatsPackage
```
### Adding dependencies
```julia
]activate ./path/to/MyStatsPackage
]add ProgressMeter
]compat # <1>
```
1. let's directly add a compat entry for ProgressMeter

### Semantic Versioning
Following `semver` - three parts:

`v2.7.5`

means:

- **Major** 2
- **Minor** 7
- **Bugfix** 5

- Bump **Major** if you propose backward-breaking changes
- Bump **Minor** if you only introduce new features
-Bump **Bugfix** if you, well, fix bugs

**Special case:**

`v0.37.1`

Means package is in development and not stable.
Bump **Major** if you release it
Bump **Minor** for breaking changes
Bump **Bugfix** if you fix bugs or release new features

### Compat entries
compat entries define with what versions your package is compatible with

```julia
[compat]
AllMinorReleases1 = "1" # <1> 
AllMinorReleases2 = "1.5" #<2>
AllMinorReleases3 = "1.5.3" #<3>
ExactPackage = "=1.5.6" #<4>
MultiVersionexample = "0.5,1.2,2"
DevelopPackage = "0.2.3" # <5>
```
1. [1.0.0-2)
2. [1.5.0-2)
3. [1.5.3-2)
4. [1.5.6]
5. [0.2.3 - 0.3)

As you can see, develop version (`version < 1`) are treated a bit special in Julia, and different to `semver`. [Read more here](https://pkgdocs.julialang.org/v1/compatibility/#compat-pre-1.0)

::: callout-warning
keep the compat list in alphabetical order - github-actions might behave very strange else.
:::

### Internals of a package

The file `./src/MyStatsPackage.jl` should contain:
```julia
module MyStatsPackage
using ProgressMeter # <1>
include("src/stats_functions.jl")

export sum
export mean, tstat

end
```
1. We use GLMakie as a simple example as you need it on Wednesday again anyway - it does take a while to install though!

Now we are ready to use the package from a different environment
```julia
]dev ./path/to/MyStatsPackage
```
