daviddoji/CalculusWithJuliaNotes.jl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

lots of cleanup

Browse Source
This commit is contained in:
jverzani
2026-08-11 17:17:08 -04:00
parent ae461659e0
commit 253295ff6e
91 changed files with 18284 additions and 7872 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
quarto/alternatives/Project.toml
View File

@@ -1,10 +1,12 @@
[deps]
BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
CairoMakie = "13f3f980-e62b-5c42-98c6-ff1f3baf88f0"
CalculusWithJulia = "a2e0e22d-7d4c-5312-9169-8b992201a882"
ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210"
GLMakie = "e9467ef8-e4e7-5192-8a1a-b1aee30e663a"
GeometryBasics = "5c1252a2-5f33-56bf-86c9-59e7332b4326"
IJulia = "7073ff75-c697-5162-941a-fcdaad2a7d2a"
Implicit3DPlotting = "d997a800-832a-4a4c-b340-7dddf3c1ad50"
Integrals = "de52edbc-65ea-441a-8357-d3a637375a31"
LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
@@ -26,6 +28,7 @@ Roots = "f2b01f46-fcfa-551c-844a-d8ac1e96c665"
SplitApplyCombine = "03a91e81-4c3e-53e1-a0a4-9c0c8f19dd66"
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
SymPy = "24249f21-da20-56a4-8eb1-6a02cf4ae2e6"
SymbolicIntegration = "315ce56f-eed0-411d-ab8a-2fbdf9327b51"
SymbolicLimits = "19f23fe9-fdab-4a78-91af-e7b7767979c3"
SymbolicNumericIntegration = "78aadeae-fbc0-11eb-17b6-c7ec0477ba9e"
Symbolics = "0c5d862f-8b57-4792-8d23-62f2024744c7"

116
quarto/alternatives/SciML.qmd
View File

@@ -1,7 +1,6 @@
# The SciML suite of packages
The `Julia` ecosystem advances rapidly. For much of it, the driving force is the [SciML](https://github.com/SciML) organization (Scientific Machine Learning).
@@ -17,6 +16,27 @@ These packages are in a process of rapid development and change to them is expec
:::
This section uses these packages from `SciML`:
```{julia}
using NonlinearSolve
using ModelingToolkit
using Optimization
using OptimizationOptimJL
using Integrals
```
In addition, we use these packages:
```{julia}
using ForwardDiff
using QuadGK
using Plots
using StaticArrays
using BenchmarkTools
```
## Symbolic math (`Symbolics`)
@@ -26,7 +46,7 @@ The `Symbolics`, `SymbolicUtils`, and `ModelingToolkit` packages are provided by
## Solving equations
Solving one or more equations (simultaneously) is different in the linear case (where solutions are readily found – though performance can distinguish approaches) and the nonlinear case – where for most situations, numeric approaches are required.
Solving one or more equations (simultaneously) is different in the linear case (where solutions are readily found---though performance can distinguish approaches) and the nonlinear case---where for most situations, numeric approaches are required.
### `LinearSolve`
@@ -69,7 +89,9 @@ However, it is more performant and not much more work to allow for a vector of s
f(u, p) = @. (u^5 - u - 1)
```
The function definition expects a container for the "`x`" variables and allows the passing of a container to hold parameters. We could have used the dotted operations for the power and each subtraction to allow vectorization of these basic math operations, as `u` is a container of values. The `@.` macro makes adding the "dots" quite easy, as illustrated above. It converts "every function call or operator in expr into a `dot call`."
The function definition expects a container for the "`x`" variables and allows the passing of a container to hold parameters.
We could have used the dotted operations for the power and each subtraction to allow vectorization of these basic math operations, as `u` is a container of values. The `@.` macro makes adding the "dots" quite easy, as illustrated above. It converts "every function call or operator in expr into a `dot call`."
A problem is set up with this function and an initial guess. The `@SVector` specification for the guess is for performance purposes and is provided by the `StaticArrays` package.
@@ -90,7 +112,7 @@ soln = solve(prob, NewtonRaphson())
Again, the derivative of `f` is computed automatically.
The basic interface for retrieving the numeric solution from the solution object is to use indexing:
The basic interface for retrieving the numeric solution from the solution object is to use indexing notation with an empty index:
```{julia}
@@ -101,7 +123,7 @@ soln[]
:::{.callout-note}
## Note
This interface is more performant than `Roots`, though it isn't an apples to oranges comparison, as different stopping criteria are used by the two. In order to compare, we help out the call to `NonlinearProblem` to indicate the problem is non-mutating by adding a "`false`", as follows:
This interface is more performant than `Roots`, though it isn't like comparing apples to oranges, as different stopping criteria are used by the two. In order to compare, we help out the call to `NonlinearProblem` to indicate the problem is non-mutating by adding a "`false`", as follows:
```{julia}
@@ -198,7 +220,7 @@ The gradient can be computed different ways within `Julia`, but here we use the
```{julia}
∇peaks(x, p=nothing) = NonlinearSolve.ForwardDiff.gradient(peaks, x)
∇peaks(x, p=nothing) = ForwardDiff.gradient(peaks, x)
u0 = @SVector[1.0, 1.0]
prob = NonlinearProblem(∇peaks, u0)
u = solve(prob, NewtonRaphson())
@@ -234,29 +256,29 @@ The extra step is to specify a "`NonlinearSystem`." It is a system, as in practi
```{julia}
ns = NonlinearSystem([eq], [x], [α], name=:ns);
ns = complete(ns)
@mtkcompile sys = NonlinearSystem([eq], [x], [α])
```
The `name` argument is special. The name of the object (`ns`) is assigned through `=`, but the system must also know this same name. However, the name on the left is not known when the name on the right is needed, so it is up to the user to keep them synchronized. The `@named` macro handles this behind the scenes by simply rewriting the syntax of the assignment:
```{julia}
@named ns = NonlinearSystem([eq], [x], [α]);
ns = complete(ns)
```
The name of the object (`sys`) is assigned through `=`, but the system must also know this same name. However, the name on the left is not known when the name on the right is needed, the`@mtkcompile` macro does this bookkeeping (an alternate is to use `@named`) and builds and compiles the system.
With the system defined, we can pass this to `NonlinearProblem`, as was done with a function. The parameter is specified here, and in this case is `α => 1.0`. The initial guess is `[1.0]`:
A system has unknowns, parameters, and observables:
```{julia}
prob = NonlinearProblem(mtkcompile(ns), [1.0], Dict(α => 1.0))
[:u=>unknowns(sys), :p=>parameters(sys), :o=>observables(sys)]
```
The problem is solved as before:
The initial condition and parameters are given as a dictionary mapping the variables to values, as with:
```{julia}
op = Dict(x => 1.5, α => 1.0)
```
Then the problem may be setup and solved as before:
```{julia}
prob = NonlinearProblem(sys, op)
solve(prob, NewtonRaphson())
```
@@ -295,11 +317,11 @@ The minus sign is needed here as optimization routines find *minimums*, not maxi
Next, we define an optimization function with information on how its derivatives will be taken. The following uses `ForwardDiff`, which is a good choice in the typical calculus setting, where there are a small number of inputs (just $1$ here.)
Next, we define an optimization function with information on how its derivatives will be taken. T
```{julia}
F = OptimizationFunction(A, Optimization.AutoForwardDiff())
F = OptimizationFunction(A)
x0 = [4.0]
prob = OptimizationProblem(F, x0)
```
@@ -313,11 +335,11 @@ soln = solve(prob, NelderMead())
:::{.callout-note}
## Note
We use the method `Newton` and not `NewtonRaphson`, as above. Both methods are similar, but they come from different packages – the latter for solving non-linear equation(s), the former for solving optimization problems.
We use the method `NelderMead` above, and not the more performant `Newton` and not `NewtonRaphson`, as above. Both methods are similar, but they come from different packages---the latter for solving non-linear equation(s), the former for solving optimization problems.
:::
The solution is an object containing the identified answer and more. To get the value, use index notation:
The solution is an object containing the identified answer and more. To get the value, again use index notation with an empty index:
```{julia}
@@ -334,9 +356,8 @@ height(xstar), A(xstar)
The `minimum` property also holds the identified minimum:
```{julia}
soln.minimum # compare with A(soln[], nothing)
soln.objective # compare with A(soln[], nothing)
```
The package is a wrapper around other packages. The output of the underlying package is presented in the `original` property:
@@ -352,7 +373,7 @@ soln.original
This problem can also be approached symbolically, using `ModelingToolkit`.
For example, we set up the problem with:
For example, we load the package and set up the problem with:
```{julia}
@@ -363,35 +384,29 @@ y = (P - 2x)/2
Area = - x*y
```
The above should be self explanatory. To put into a form to pass to `solve` we define a "system" by specifying our objective function, the variables, and the parameters.
The above should be self explanatory. To put into a form to pass to `solve` we define a "system" by specifying our objective function, the variables, and the parameters and then "compile" it through `mtkcompile`, in this case its macro form, as before:
```{julia}
@named sys = OptimizationSystem(Area, [x], [P]);
sys = complete(sys)
@mtkcompile sys = OptimizationSystem(Area, [x], [P]);
```
(This step is different, as before an `OptimizationFunction` was defined; we use `@named`, as above, to ensure the system has the same name as the identifier, `sys`.)
This system is passed to `OptimizationProblem` along with a specification of the initial condition ($x=4$) and the perimeter ($P=25$). A vector of pairs is used below:
This system is passed to `OptimizationProblem` along with a specification of the initial condition ($x=4$) and the perimeter ($P=25$). A dictionary is used below:
```{julia}
prob = OptimizationProblem(sys, [x => 4.0], [P => 25.0]; grad=true, hess=true)
op = Dict(x => 4.0, P => 25.0)
prob = OptimizationProblem(sys, op)
```
The keywords `grad=true` and `hess=true` instruct for automatic derivatives to be taken as needed. These are needed in the choice of method, `Newton`, below.
Solving this problem then follows the same pattern as before, again with `Newton` we have:
Solving this problem then follows the same pattern as before:
```{julia}
solve(prob, Newton())
solve(prob, NelderMead())
```
(A derivative-free method like `NelderMead()` could be used and then the `grad` and `hess` keywords above would be unnecessary, though not harmful.)
We used the derivative-free method `NelderMead()`. Other methods are more performant, but this one did not require a conversation about auto differentiation, as `Newton()` would have.
---
@@ -412,13 +427,11 @@ could be similarly approached:
@variables x
y = Area/x # from A = xy
P = 2x + 2y
@named sys = OptimizationSystem(P, [x], [Area]);
sys = structural_simplify(sys)
@mtkcompile sys = OptimizationSystem(P, [x], [Area]);
u0 = [x => 4.0]
p = [Area => 25.0]
op = Dict(x => 4.0, Area => 25.0)
prob = OptimizationProblem(sys, u0, p; grad=true, hess=true)
prob = OptimizationProblem(sys, op; grad=true, hess=true)
soln = solve(prob, LBFGS())
```
@@ -506,7 +519,6 @@ The package follows the same `problem-algorithm-solve` interface, as already see
The interface is designed for $1$-and-higher dimensional integrals.
The package is loaded with
@@ -519,7 +531,7 @@ For a simple definite integral, such as $\int_0^\pi \sin(x)dx$, we have:
```{julia}
f(x, p) = sin(x)
prob = IntegralProblem(f, 0.0, 1pi)
prob = IntegralProblem(f, (0.0, 1pi))
soln = solve(prob, QuadGKJL())
```
@@ -550,7 +562,7 @@ The `Integrals` solution is a bit more verbose, but it is more flexible. For exa
```{julia}
f(x, p) = sin.(x)
prob = IntegralProblem(f, [0.0], [1pi])
prob = IntegralProblem(f, (0.0, 1pi))
soln = solve(prob, HCubatureJL())
```
@@ -588,14 +600,13 @@ Using `Integrals` with `QuadGK` we have:
```{julia}
f(x, p) = sin(p*x)
function ∫sinpx(p)
prob = IntegralProblem(f, 0.0, 1pi, p)
prob = IntegralProblem(f, (0.0, 1pi), p)
solve(prob, QuadGKJL())
end
```
We can compute values at both $p=1$ and $p=2$:
```{julia}
∫sinpx(1), ∫sinpx(2)
```
@@ -624,13 +635,13 @@ The area under a surface generated by $z=f(x,y)$ over a rectangular region $[a,b
For example, the area under the function $f(x,y) = 1 + x^2 + 2y^2$ over $[-1/2, 1/2] \times [-1,1]$ is computed by:
XXX
```{julia}
f(x, y) = 1 + x^2 + 2y^2 # match math
fxp(x, p) = f(x[1], x[2]) # prepare for IntegralProblem
ls = [-1/2, -1] # left endpoints
rs = [1/2, 1] # right endpoints
prob = IntegralProblem(fxp, ls, rs)
prob = IntegralProblem(fxp, (ls, rs)) # tuple of (ls, rs)
soln = solve(prob, HCubatureJL())
```
@@ -702,6 +713,7 @@ So we have $\iint_{G(R)} x^2 dA$ is computed by the following with $\alpha=\pi/4
```{julia}
#| eval: false
import LinearAlgebra: det
𝑓(uv) = uv[1]^2
@@ -735,6 +747,6 @@ For a trivial example, we have:
```{julia}
f(x, p) = [x[1], x[2]^2]
prob = IntegralProblem(f, [0,0],[3,4])
prob = IntegralProblem(f, ([0,0],[3,4]))
solve(prob, HCubatureJL())
```

668
quarto/alternatives/makie_plotting.qmd
View File

File diff suppressed because it is too large Load Diff

192
quarto/alternatives/symbolics.qmd
View File

@@ -1,9 +1,5 @@
# Symbolics.jl
XXX This needs updating! XXX
XXX add https://docs.sciml.ai/SymbolicIntegration/stable/ XXX
There are a few options in `Julia` for symbolic math, for example, the `SymPy` package which wraps a Python library. This section describes a collection of native `Julia` packages providing many features of symbolic math.
@@ -386,7 +382,7 @@ ex = x^5 - x - 1
Roots.find_zero(λ, (1, 2))
```
### Plotting
## Plotting
Using `Plots`, the plotting of symbolic expressions is similar to the plotting of a function, as there is a plot recipe that converts the expression into a function via `build_function`.
@@ -425,7 +421,7 @@ The ordering of the variables is determined by `Symbolics.get_variables`:
Symbolics.get_variables(ex)
```
### Polynomial manipulations
## Polynomial manipulations
There are some facilities for manipulating polynomial expressions in `Symbolics`. A polynomial, mathematically, is an expression involving one or more symbols with coefficients from a collection that has, at a minimum, addition and multiplication defined. The basic building blocks of polynomials are *monomials*, which are comprised of products of powers of the symbols. Mathematically, monomials are often allowed to have a multiplying coefficient and may be just a coefficient (if each symbol is taken to the power $0$), but here we consider just expressions of the type $x_1^{a_1} \cdot x_2^{a_2} \cdots \cdot x_k^{a_k}$ with the $a_i > 0$ as monomials.
@@ -560,7 +556,7 @@ m,n = degree.(nd(ex))
m > n ? "limit is infinite" : m < n ? "limit is 0" : "limit is a constant"
```
### Vectors and matrices
## Vectors and matrices
Symbolic vectors and matrices can be created with a specified size:
@@ -616,7 +612,7 @@ R \ b
collect(R \ b)
```
### Algebraically solving equations
## Algebraically solving equations
The `~` operator creates a symbolic equation. For example
@@ -660,7 +656,7 @@ eqs = R*X .~ b
Symbolics.symbolic_linear_solve(eqs, [x,y])
```
### Limits
## Limits
Many symbolic limits involving exponentials and logarithms can be
computed in Symbolics, as of recent versions. The underlying package
@@ -683,7 +679,7 @@ limit(F(𝑥), 𝑥, Inf)
```
### Derivatives
## Derivatives
`Symbolics` provides the `derivative` function to compute the derivative of a function with respect to a variable:
@@ -764,164 +760,76 @@ eqs = [ x^2 - y^2, 2x*y]
Symbolics.jacobian(eqs, [x,y])
```
### Integration
## Integration
The `SymbolicNumericIntegration` package provides a means to integrate *univariate* expressions through its `integrate` function.
The `SymbolicIntegration` package provides two means to integration *univariate* functions using either the Risch alogorithm or a rules-based approach.
```{julia}
using SymbolicIntegration, Symbolics
```
The main entry point is the function `integrate`.
This is a substitution test:
```{julia}
@variables x a b
integrate(x*exp(-x^2), x)
```
This is an integration by parts example:
```{julia}
integrate(x * sin(x), x)
```
As is this. We
```{julia}
integrate(a * log(b*x), x)
```
Symbolic integration can be approached in different ways. SymPy implements part of the Risch algorithm in addition to other algorithms. Rules-based algorithms could also be implemented.
For a trivial example, here is a rule that could be used to integrate a single integral
The integration of rational functions (ratios of polynomials) can be done algorithmically, provided the underlying factorizations can be identified.
```{julia}
@syms x ∫(x)
is_var(x) = (xs = Symbolics.get_variables(x); length(xs) == 1 && xs[1] === x)
r = @rule ∫(~x::is_var) => x^2/2
r(∫(x))
integrate((x-1)/((x-2)^3*(x-4)), x)
```
The `SymbolicNumericIntegration` package includes many more predicates for doing rules-based integration, but it primarily approaches the task in a different manner.
#### Method selection
A third argument to `integrate` can specify the method, as in `integrate(expr, var, method)`.
If $f(x)$ is to be integrated, a set of *candidate* answers is generated. The following is **proposed** as an answer: $\sum q_i \Theta_i(x)$. Differentiating the proposed answer leads to a *linear system of equations* that can be solved.
The example in the [paper](https://arxiv.org/pdf/2201.12468v2.pdf) describing the method is with $f(x) = x \sin(x)$ and the candidate thetas are ${x, \sin(x), \cos(x), x\sin(x), x\cos(x)}$ so that the proposed answer is:
$$
\int f(x) dx = q_1 x + q_2 \sin(x) + q_3 \cos(x) + q_4 x \sin(x) + q_5 x \cos(x)
$$
We differentiate the right hand side:
This function does not get solved by the Risch method:
```{julia}
@variables q[1:5] x
ΣqᵢΘᵢ = dot(collect(q), (x, sin(x), cos(x), x*sin(x), x*cos(x)))
simplify(Symbolics.derivative(ΣqᵢΘᵢ, x))
ex = cos(5x)*sin(x)
integrate(ex, x, RischMethod())
```
This must match $x\sin(x)$ so we have by equating coefficients of the respective terms:
$$
q_2 + q_5 = 0, \quad q_4 = 0, \quad q_1 = 0, \quad q_3 = 0, \quad q_5 = -1
$$
That is $q_2=1$, $q_5=-1$, and the other coefficients are $0$, giving an answer computed with:
However, it can be done with the rules-based method:
```{julia}
d = Dict(q[i] => v for (i,v) ∈ enumerate((0,1,0,0,-1)))
substitute(ΣqᵢΘᵢ, d)
integrate(ex, x, RuleBasedMethod())
```
The package provides an algorithm for the creation of candidates and the means to solve when possible. The `integrate` function is the main entry point. It returns three values: `solved`, `unsolved`, and `err`. The `unsolved` is the part of the integrand which can not be solved through this package. It is `0` for a given problem when `integrate` is successful in identifying an antiderivative, in which case `solved` is the answer. The value of `err` is a bound on the numerical error introduced by the algorithm.
The example of integrating `(x-1)/((x-2)^3*(x-4))`, done above, is one where the `RischMethod` works, but not the `RuleBasedMethod`.
To see, we have:
Each method has different keyword arguments. For `RuleBasedMethod` the `verbose=true` argument will show which rules were applied. In this case, there is a single one:
```{julia}
using SymbolicNumericIntegration
@variables x
integrate(x * sin(x))
integrate(ex, x, RuleBasedMethod(verbose=true))
```
The second term is `0`, as this integrand has an identified antiderivative.
This example, shows two rules are applied:
```{julia}
#| eval: false
integrate(exp(x^2) + sin(x))
integrate(a * log(b*x), x, RuleBasedMethod(verbose=true))
```
This returns `exp(x^2)` for the unsolved part, as this function has no simple antiderivative.
To read the rules, there are different predicate functions involved. The commonly used `contains_var` predicate checks through pattern matching if the last variable is contained in any of the rest of the specified variables. In the pattern `c * x ^ n` the match is `c` is `b` and `n` is `1`. Neither depends on `x`, so this rule is applied to integrate `log(b*x)`.
Powers of trig functions have antiderivatives, as can be deduced using integration by parts. When the fifth power is used, there is a numeric aspect to the algorithm that is seen:
```{julia}
u,v,w = integrate(sin(x)^5)
```
The derivative of `u` matches up to some numeric tolerance:
```{julia}
Symbolics.derivative(u, x) - sin(x)^5
```
---
The integration of rational functions (ratios of polynomials) can be done algorithmically, provided the underlying factorizations can be identified. The `SymbolicNumericIntegration` package has a function `factor_rational` that can identify factorizations.
```{julia}
#| eval: false
import SymbolicNumericIntegration: factor_rational
@variables x
u = (1 + x + x^2)/ (x^2 -2x + 1)
v = factor_rational(u)
```
The summands in `v` are each integrable. We can see that `v` is a reexpression through
```{julia}
#| eval: false
simplify(u - v)
```
The algorithm is numeric, not symbolic. This can be seen in these two factorizations:
```{julia}
#| eval: false
u = 1 / expand((x^2-1)*(x-2)^2)
v = factor_rational(u)
```
or
```{julia}
#| eval: false
u = 1 / expand((x^2+1)*(x-2)^2)
v = factor_rational(u)
```
As such, the integrals have numeric differences from their mathematical counterparts:
::: {.callout-note}
#### Errors ahead
These last commands are note being executed, as there are errors.
:::
```{julia}
#| eval: false
a,b,c = integrate(u) # not
```
We can see a bit of how much through the following, which needs a tolerance set to identify the rational numbers of the mathematical factorization correctly:
```{julia}
#| eval: false
cs = [first(arguments(term)) for term ∈ arguments(a)] # pick off coefficients
```
```{julia}
#| eval: false
rationalize.(cs[2:end]; tol=1e-8)
```
The `contains_var(c, n, x)` call above checks if the constant `b` (matching `c` in the pattern) depends on `x` and if `1` depends on `x` (matching the power `n`, using a default for the variable, in the pattern). The rules comes from [`Rubi`](https://rulebasedintegration.org/) which has some 7000 rules, many of which are implemented in `SymbolicIntegration`.